You wouldn't have to use absolute paths everywhere - you could use paths relative to any file descriptor that pointed at a directory. The shell itself wouldn't seem to have any problem - it could maintain a logical CWD without assistance. Utilities would probably need some other convention - maybe fd 4 points at the directory they are to operate in at start?

In a sense, this is "still a CWD" - but the differences would be 1) you can maintain multiple at the same time, and 2) you could close it.