If you're not using Bundler and have a simple dependency chain, you can get further improvement from symlinking your gems into a single directory (see https://gist.github.com/975509 ).
This patch certainly improved my load time tremendously, but the core of the problem still lies with the way rubygems and bundler dump all directories of gems in the load path. The promise of $LOAD_PATH is that all the directories in it will be tried -- the most bang-for-buck optimization is thus minimizing its size.
If rubygems did this, everything would be much better.
There is one problem though, and it is that some libs are bad citizens and install stuff outside of lib|bin and depend on it (haml's VERSION file is an example of this), and that sucks.
This patch certainly improved my load time tremendously, but the core of the problem still lies with the way rubygems and bundler dump all directories of gems in the load path. The promise of $LOAD_PATH is that all the directories in it will be tried -- the most bang-for-buck optimization is thus minimizing its size.