Indeed, back in January I came across a directory traversal vulnerability in Rack (http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/...). They had lots of unit tests, but none testing that particular case. Unit testing is certainly no magic bullet.

On the other hand, the Rack team was incredibly responsive and put out an update within hours.