+1 to your main point, but I had to respond to this side point, since I see it mentioned so often by Lispers:
"It's particularly amusing as a Lisp programmer to watch all these fads come and go because none of these ideas are new."
The takeaway from this is not that programmers are stupid and we should all be using Lisp, but that user interfaces matter. Social conventions matter. Installed bases matter. After all, most of Web 2.0 is just fancy rounded-corner reimplementations of 30-year-old UNIX utilities.
C succeeded because in one critical area - creating a fast, responsive UI on the hardware & compiler technology of the early 1980s - it worked and Lisp didn't. C++ and Java succeeded because programmers could take their C knowledge and syntax and apply it.
XML succeeded because it could leverage everyone's knowledge of HTML. HTML succeeded because at it's heart, it's just text - you could start typing a plain old text document and it was valid HTML. A good part of JSON's success is because every JSON document is also legal JavaScript and legal Python.
Aspect-oriented programming generates interest because it's in Java, and people already use Java. And on and on it goes...
On purely technical grounds, the Lisp solution from 25 years ago is almost always better than the modern-day solution we're just rediscovering now. But the modern-day solution has the benefit of building on top of all the social conventions that have arisen in the last 25 years. That's worth remembering, particularly in a forum devoted to entrepreneurship. If you tell your customers "our solution is better, but you'll have to throw away everything you already know and start from scratch", they probably won't remain your customers for long.
C succeeded because it was the language of Unix, and it rode to success on Unix's success. C++ did indeed succeed because so many people knew C. Java has very little to do with C, except for the surface syntax, which makes it look as if it is similar to C, but its semantics are not like C at all.
You do not need to throw away everything you know to use Lisp. 90% of what you have spent your time on when you learn programming applies to Lisp just as well as any other language: variables, procedures, iteration, and on and on. At ITA, when we hire new hackers who don't know Lisp, we just give them a copy of Peter Seibel's excellent book "Practical Common Lisp", and they quickly pick it up.
Common Lisp, as it is used in real-world practice, isn't nearly as strange as many people think. (Yes, we have a job to do to explain that to the world.) It's not a new paradigm the way Prolog or Haskell are. You can get started, doing the kinds of things you do in Java, very quickly. Now, learning all the more-powerful stuff, like Lisp macros and how to effectively and tastefully use them, and more advanced object-oriented features of CLOS, does take some time to learn. But, then, learning all the useful stuff in the Java libraries takes time to learn, too, even if you're the best C programmer in the world.
Over on the Lisp lists I sometimes make the point that Lisp failed precisely because it was so powerful that one person could be productive enough to get useful work done by themselves. As a result, Lisp tended (and maybe still tends) to attract people who don't work well with others, whereas if you program in C you have no choice but to work as part of a team if you want to get anything done at all. At the end of the day teamwork wins, even when hobbled by inferior tools.
That's an amusing idea, but it is not the case. I used Lisp extensively at the MIT AI Lab, at Symbolics (I was a founder), and now at ITA Software. The people worked together as teams very well in all places. The fact that you're using Lisp does not change the need to work together on design, conventions, architecture, etc, not to mention code reviews. The only top-grade hacker I ever worked with who could not work with others was using C++ (not that I think it matters).
"I used Lisp extensively at the MIT AI Lab, at Symbolics (I was a founder), and now at ITA Software. The people worked together as teams very well in all places."
That probably tells more about the people and environment of MIT AI Lab, Symbolics and ITA Software than the team dynamics of Lisp.
I don't claim this to be a hard and fast rule. It is obviously not impossible for Lispers to work together. My theory is just that Lisp tends to attract non-team-players more than C does, and the macro effect of this is that in the main C wins. There will, of course, be exceptions.
"It's particularly amusing as a Lisp programmer to watch all these fads come and go because none of these ideas are new."
The takeaway from this is not that programmers are stupid and we should all be using Lisp, but that user interfaces matter. Social conventions matter. Installed bases matter. After all, most of Web 2.0 is just fancy rounded-corner reimplementations of 30-year-old UNIX utilities.
C succeeded because in one critical area - creating a fast, responsive UI on the hardware & compiler technology of the early 1980s - it worked and Lisp didn't. C++ and Java succeeded because programmers could take their C knowledge and syntax and apply it.
XML succeeded because it could leverage everyone's knowledge of HTML. HTML succeeded because at it's heart, it's just text - you could start typing a plain old text document and it was valid HTML. A good part of JSON's success is because every JSON document is also legal JavaScript and legal Python.
Aspect-oriented programming generates interest because it's in Java, and people already use Java. And on and on it goes...
On purely technical grounds, the Lisp solution from 25 years ago is almost always better than the modern-day solution we're just rediscovering now. But the modern-day solution has the benefit of building on top of all the social conventions that have arisen in the last 25 years. That's worth remembering, particularly in a forum devoted to entrepreneurship. If you tell your customers "our solution is better, but you'll have to throw away everything you already know and start from scratch", they probably won't remain your customers for long.