Greg Linden used to work at Amazon. From an older blog post of his:
"In A/B tests, we tried delaying the page in increments of 100 milliseconds and found that even very small delays would result in substantial and costly drops in revenue."
I find it immaterial considering the easier thing to speed up is the number of HTTP requests on the site, and that has clearly gotten worse over the years, not better. I pushed heavily to get HTML cleaned up and valid (showing an easy 10-20% reduction with HTML alone), reduce the amount of javascript, etc. It simply wasn't a priority. To be fair, at the time I was there the priority was to grow as fast as possible and deal with profits later.
"In A/B tests, we tried delaying the page in increments of 100 milliseconds and found that even very small delays would result in substantial and costly drops in revenue."
http://glinden.blogspot.com/2006/11/marissa-mayer-at-web-20....