Hacker Timesnew | past | comments | ask | show | jobs | submitlogin
The Myth of the Genius Programmer (code.google.com)
61 points by mlinsey on June 5, 2009 | hide | past | favorite | 59 comments


This is a bit of a diversion from Fitzpatrick and Collins-Sussman's talk, but I run into this belief a lot, and want to make a point.

I see little evidence here that Collins-Sussman has finally understood DVCS as a social entity, although he certainly seems to have the technical details down. The strongest argument for DVCS--any DVCS--that I have found is the periodic fights over "committer access" or project direction generally for projects like NetBSD, XEmacs, gcc/EGCS, or glibc/eglibc disappear.

I've seen more times than I can conveniently remember a perception of a cabal of committers who communicate exclusively with each other. This looks a lot like the structure of some sort of conspiracy, even though that is rarely the purpose. Even when it is, it's usually trying to work around some productive but extremely difficult person like Ulrich Drepper or Theo de Raadt.

DVCSes have the critically important property that there is no longer any inherent technical center of development but that the social center of development is retained (usually). So while there is a technical hierarchy of git repositories that goes to Linus, if you think he's an idiot and want to take it a different direction, it's easy to do that--and if he decides you have a good idea in the end, he can merge with you without huge amounts of effort.

Collins-Sussman has been concerned, publically, that there's some splintering process that DVCSes enable and that SVN prevents. But this perception is one you can only arrive at if you are in control of the SVN repository--if you are trying to submit patches to somebody doing this, and you don't have write access on the SVN server for branches or whatever (which imposes a nontrivial amount of effort), you more or less have to do this splintering on your own, without any tool support. In my experience DVCSes don't make this any more common than it was once; it's just that now these private splinters are visible, and can be integrated.

Another secondary thing that's very convenient for me is that I can make many small changes and push them all at once when I'm happy with them--because they're small they're easier to review. But this is a workflow issue, it's not that one couldn't do this with SVN.


DVCSes have the critically important property that there is no longer any inherent technical center of development but that the social center of development is retained (usually).

This is a really good point. It allows the social structure of the project to grow and change in a very natural way which makes it not only easier to adapt as committers come and go, but effortless.


The social center is also decided by a kind of marketplace.

With central control of a repository, you can have what amounts to an "irrigation rights empire" -- a totalitarian regime (perhaps with some democratic window dressing) based on one party's control over some sort of infrastructure.


Another secondary thing that's very convenient for me is that I can make many small changes and push them all at once when I'm happy with them--because they're small they're easier to review. But this is a workflow issue, it's not that one couldn't do this with SVN.

This is part of the problem with DVCSes - you do your work off in a cave, and then when it's "perfect" you submit it for inclusion.

Note that SVN doesn't lack this problem - it can even make it worse. It's just far more common in my experience for people to do work in a cave somewhere with a DVCS and then throw it all over the wall in a huge hunk than with SVN, because it's so easy to maintain that fork off in neverland.

After all, just look at the comments (on this article no less!) about how "once I clean X up, I'll put it on {github,bitbucket,google code}." That's the kind of attitude that means it'll almost never (statistically) be seen by anyone other than the original author (which is likely true anyway, only one of my projects has ever really attracted anything resembling attention).

Note that Ben isn't convinced that this risk of DVCSes outweighs their (admittedly large) benefits. It merely aggravates a very serious problem to a point where it becomes a complete nightmare.


My experience has been that the alternative to many small commits, occasionally batched (because who really wants to look at "cleaned up Checkstyle's warnings" endlessly?) is one big commit two or three times a day, which is significantly harder to digest. But in any case, as you allude to in your response, this is a social problem. I contend that the sort of people who work in a cave somewhere with a DVCS and then throw it all over the wall in a huge hunk would either do that anyway (see Linux kernel features before they make it into the mainland trunk for evidence; this was going on well before git, and if anything git has made it less of a problem) or wouldn't contribute at all.


These guys look like being advocates of mediocrity. I haven't seen such radical egalitarianism promotion even from junior managers.

I especially disliked the moment when they interrupted the criticizer who was talking that there are a lot of evidences of geniuses existence and highly productive programmers in particular.

And rather than advising to learn from the best programmers, they are advising to learn from ephemeral "community" and trimming ones ego and ambitions.


You seem to be making the same effort to miss the point as the man that ran through a litany of objections starting at 45:18.

I don't like the "there are no geniuses" phrase, especially since they then have to back down and say, "Well, it isn't that there are no geniuses -- just that they're extremely rare," which just obscures the point further. The point I think they're trying to make, though, is that the myth of the hero programmer who writes a perfect program and releases it to a world full of unhelpful clods is counterproductive.

If you want to dispute their point, then answer their question: What successful, widely-used project was created entirely by one lone genius?


TeX, by Donald Knuth, certainly qualifies. But I have a hard time thinking of another one so your point is taken.


Git and Linux started out as creations of Linus.


And they only took off because Linus is not a total dick, and he published his code very early so that collaborators could hack on it. Those two things were important in Linux and Git's success.

Thats all the talk is about. And the only reason anyone is arguing with it is the very insecurity the talk seeks to address.


> And they only took off because Linus is not a total dick

Have you ever seen Linus on usenet? I'd argue that his Real Programmer protective stance towards Linux was more helpful to it taking off than being a really Nice Guy.


>> And they only took off because Linus is not a total dick

> Have you ever seen Linus on usenet?

"Not a total dick" in Internet terms is something like "A little to the left of Mao." However, Linus is also very astute and capable. People will tolerate "does not suffer fools gladly" if they are not a fool, or if they like the person's work.

Every programmer with an ego should ask themselves, "Am I an unjustified asshole?" Consider:

    - Do a lot of people mostly like your work?
    - Does the cost/benefit of alienating people vs. 
      shutting down the fools generally work out positively?
      (For the community not just for your personal 
      gratification.)
    - Do people complain about you profusely, but for 
      some reason still grudgingly respect you?
Answer yes 3 times? Congrats: you are not an unjustified asshole!

Are you a delusional asshole?

    - Do only a select few understand your work, and do you
      think even they are all inferiors?
    - Are you *always* right?
    - Do the people who are trying to do something valuable 
      generally try to ignore you?  
Answer yes 3 times? Well, don't worry about it, you're probably one of the few who are genuinely ahead of their time and misunderstood. ;-)


Linux was a copy of Minux.


What successful, widely-used project was created entirely by one lone genius?

This position is a little oxymoronic since anything successful and widely-used must benefit from the non-zero amount of direct and indirect community feedback. Every project exists in relation to others and if one is superior in some way it's often because the creator decided to improve on something they liked or disliked in another system.


BitTorrent, by Bram Cohen. Napster. Ruby on Rails (maybe; I'm uncertain how much it was just dhh in the early days).


3 clever ideas with very poor implementations, heavily refined by many others over years.


I can do go for that for RoR; but Napster was a much bigger leap in its time than we give it credit for now, and the same for Bram/BT. The BitTorrent protocol was well done, even once P2P was far from an obscure idea.


And all of them were "innovations", not inventions. None of them were original ideas, they were just rehashed ones that were right for the time in which they became popular and widely known.


To be fair to Bram, he had the technical insight to deliver a workable P2P protocol with the ability to distribute demand at a nice granularity. Because of that, his protocol was technically superior to the competitors. Also, he did not try to tackle search and do it badly.

Napster was a bad implementation at the right time.

RoR - credit should be given. Sometimes just getting off your duff and starting something is what the world needs. Motivating many others to refine code into something useful deserves credit.

(EDIT: I am not saying anything was/wasn't a work of genius. I'm just saying it's truly creditable, whatever it was.)


To be truly fair to Bram, he had the insight to take an existing marginally working p2p system that he had previously been employed to develop and strip it down to the essential components needed to shift bits quickly. Innovation, not invention.

I definitely agree with you on Napster. I am less sure about RoR. It was a a very nice improvement over existing systems, but how much of that was DHH genius and how much was simply due to Ruby providing a better platform upon which to build this particular framework I do not know. It was definitely better than existing alternatives in other scripting languages, but I was not a part of the Ruby world at the time and have no idea where the initial RoR work fit in to the whole Ruby ecosystem at the time.


Well, you can write fortran in Ruby, but it took DHH to show where it does shine. It is easy to underestimate the initial insight in hindsight "oh, that's obvious", but it only becomes obvious when someone does it. And does not definition of geniality has something on having that insight for the "obvious"?


Others had shown were it shines as well; there was Nitro, IOWA, a few other apps that were solid examples of Ruby qua Ruby at the same time as (or prior to) Rails.

Why do some people/code get attention and accolades, while others, equally deserving, do not? Timing? Personality? Bluster?


The ability to mobilize people into a large, cohesive community.


Sure, but how does that happen?


DHH said (more or less, I forget the exact words) that Rails had no new ideas, just implementations of existing good ones.

Nitro, another MVC-ish Ruby Web framework released at the same time as Rails, always stuck me as being a much better approach to the same problem, and better engineered (thread safe, less "magic" and munging of core Ruby classes, a form of pipeline transformations somewhat similar to Rack middleware today), but for whatever reasons Rails managed to capture far more attention.

The big win for Rails was not the initial code itself, but in capturing attention and motivating people to contribute, to fix it up, and make it better.


Sid Meier and Will Wright


Genius exists, but is not a sufficiently good excuse for being an asshole.


Bravo. Its amazing that so many people took something other than this away from their talk. Really drives the talk's point home, doesn't it?


These guys look like being advocates of mediocrity.

Yup. They wrote Subversion and continue to encourage people to use it.

Perhaps they should tone down their egos and realize that Subversion was not really that great of an idea, and there are now several competing tools that do everything Subversion does, but better.


You say that "Subversion was not that great of an idea" but if you go back to 2001 (which is when svn became selfhosting), or even 2004, CVS was such a disaster for some use cases that the need for a replacement was readily evident. Add to that the fact that svn was a compelling replacement for Visual Source Safe, and it was definitely a worthwhile idea. Sure, it's now the "old safe standard" of version control, but there's a lot of value in that for many companies. Several of the original svn team (including these two) have admitted that some of the design decisions (the handling of tags and branches most notably) were probably mistakes, given the value of hindsight. They got a lot right too, though. The delta editor interface is nicely abstracted to the point that hgsubversion uses the same delta editor as the svn command line client to do something the original design spec never would have anticipated. Yet it works extremely well.

Subversion is much simpler for people to "get" than other systems. I've been a heavy evangelist of DVCS tools and workflows for a while at work, and it's taken since October (8 months) to get any critical mass behind actually even investigating switching to those from SVN.

Let's face it: Subversion is many many times better than nothing, which is what a ton of people do. Most college graduates in CS or CSE come out of school never having touched version control in my experience, and the ones that have seen it were exposed via open source, not their coursework. Evangelizing Subversion is (really lousy analogy warning!) like convincing people to smoke cigarettes instead of doing illegal drugs. At least it's less bad for them, and the barrier to entry might be a bit lower. When we had a poll of the local Python users group one meeting, "email" was the most popular version control solution, followed by a basic dead heat between svn/bzr/git/hg. The fact that people still view email as a valid answer is far more troubling to me than evangelizing an old and very mature tool instead of one of the new upstarts.


I think SVN is a classic case of overreacting. "CVS is bad, and it has first class branches and tags. Those must be bad too!" The result was a very poor design ("everything is a file!") and a very poor implementation. The implementation can be fixed (and it is going in that direction), but the design is broken to the point of being unusable.

I am glad that CVS died, but it was "replaced" with something not-much-better.


A lot of the problems SVN ran into was trying to use cutting edge technology and standards before they were really ripe. WebDAV, for example, or the endless, endless iterations of Berkeley DB corruption. It is, in a lot of ways, an excellent illustration of the dangers of trying to solve the wrong problem.


What's innovative about WebDAV ? It's basically a design by committee standard pushed by Rational to web enable clearcase; it's a very verbose protocol and there was an entry in Sussman's blog about replacing WebDAV with a better wire protocol like Mercurial's. Getting WebDAV into SVN was basically Greg Stein's idea - he had worked at Microsoft and had observed them transitioning over to WebDAV in a big way and started pushing for its usage in SVN as well - a move that was not well thought out. WebDAV is not very popular even with some of the SVN hackers.

Berkeley DB was certainly old when it was adopted by the SVN project. It's not like the NFS issues with BDB were unknown.


I never said WebDAV was innovative, just that it was brand new and full of problems. And as for BDB, IIRC they were having issues with the database getting wedged constantly even when not using NFS. But this was long after I had stopped paying a great deal of attention to the project.


"he had worked at Microsoft and had observed them transitioning over to WebDAV in a big way and started pushing for its usage in SVN as well"

That's why I am suspicious anyone who ever worked for Microsoft, specially when you talk about open-source. You never know when they will send a saboteur.


Linus was making fun of SVN in his talk ( http://www.youtube.com/watch?v=4XpnKHJAok8 ). One of his point was, that SVN chose the wrong way right from the start and there is no way to have "CVS done right" because CVS is hopeless by design.


CVS may have had serious flaws, but I find Linus' original revision control practices to be less than optimal:

"The Linux kernel source code used to be maintained without the help of an automated source code management system, mostly because of Linus Torvalds' dislike of centralized SCM systems.

In 2002, Linux kernel development switched to BitKeeper, a SCM system which satisfied Linus Torvalds' technical requirements."

Nearly a decade without a revision control system?

http://en.wikipedia.org/wiki/Linux_kernel#Revision_control


In Linus' git talk, he says that he finds creating different tarballs is much better version control than using CVS.


If you compare granularity of tracking and utility of historical version control across different projects of that era, such as FreeBSD vs. Linux, I think you'll find that Linus' position is clearly false.


While I don't have the time to watch a 50 minute video at the moment, a common theme I see in the PDF is that people are reluctant to share source code until it's reasonably "done". A lot of things that seem impressive as completed works have spent most of their development messy and (by definition!) incomplete. When coupled with potentially endless anonymous criticism via the internet* , this can be pretty intimidating.

I get where they're coming from. I have several projects that I really need to polish up and upload, and I want them to feel presentable; I don't really understand how people get used to working in public (on github, for example). I also do a fair amount of maintenance programming, and ... I want to make sure my code is better than that. Just because I change my mind, make mistakes, etc., it doesn't mean I need to publish sloppy code.

* Look at the comments in any discussion on Reddit. About anything. People get nasty, often just because they can.


I see where they are coming from too, and I often feel the same way; sometimes I have found myself starting on side-projects and not even telling close friends what they are about because I am scared or embarrassed to have anyone see my work in an embarrassingly incomplete state.

I think one of the key lessons for me, which is very easy to grasp intellectually but much harder to internalize, is that any nastiness you receive in the short term doesn't really matter (and often comes from people who don't really matter). If you succeed in the long-term, that's what you will be known for, and all of the scorn directed at your early works will seem irrelevant. You should therefore do whatever maximizes your chances at long-term success, which includes taking the risk of exposing yourself to humiliation so that you can get better faster.

It's one of the things I really admire about, say, Jeff Atwood. He's consistently bashed here and elsewhere for being incompetent, and it's not undeserved: even though I consider myself a pretty poor programmer at the moment, I can often tell when he says things that are fallacious or ignorant. And yet, with Stack Overflow he has been involved in a project more successful than most people's projects here. I would never mistake him for the mythical genius programmer, and I may be puzzled at the attention he receives or his persistence about writing topics he doesn't know much about, but when I see how he keeps writing and keeps building useful things even when everyone is publicly explaining to him all the ways he sucks, I wish I had his cojones.


I don't really understand how people get used to working in public

It's easy. Until your code is good, no one gives a damn about it. Even if it's good, you'll still have to shove it down people's throats before they care. So don't worry that "someone might see your bad code". They won't. Nobody cares.


Successful projects are not determined by the quality of the code, but the utility of it. It's quite possible to have a very successful project, that a lot of people use, made up of embarrassingly bad code.


Alternative phrasing: to make something good I find I have to be thinking about it all the time. If I think about it all the time I talk about it every chance I get, ask for help, comments, feedback. It's easier to ask for help if the thing you want them to help you with is a click away.

So make it public.


Re: a bit tangential on nasty criticism. An approach I try to have (it's not for everyone) is to see the code as trying to do something worthwhile, rather than for the greater glory of 10ren. Without my ego on the line, I have more attention for the code itself. :-)

With this attitude* , only criticism about how the code is serving its purpose is relevant. If someone criticizes it (or I imagine they will) because it's Lisp (or because isn't), or because it uses design patterns (or because it doesn't) and the criticism doesn't seem to make a difference to its purpose - it's just kind of bemusing.

* Hard to do.

EDIT ah... having now read the slides, this is one of the things they're saying.


You don't have to work in public using github. That's the nice feature of DVCS — you can have it all your code for yourself and at the same time enjoy all the benefits of VCS. Using centralized system you don't have much choice - either you don't commit, or you commit and everybody sees your ugly code. With DVCS you can have your private branches and work on your code as long as you want. When you a happy with your work, just push it to the public repo (hosted on github for example). Even more, you can have a lots of tiny working commits and then "squash" them into one big and nice final commit.


Love this: "The failures tend to get smaller over time, and successes get larger. That's a trend you'll see, especially if you're learning, over time"


Wouldn't this be true only if you're doing the same thing over and over?


From my point of view the video is basically saying collaboration is most important as a programmer. Unlike athletes, programmers have to work together to achieve something great. Anything that is really impressive/great will come from the contribution of many programmers (eg linux). The question is: what is the most effective way to collaborate? Is there something about the ego that makes us ineffective? It would be really interesting if they had elaborated more on what makes a "healthy ego". Personally, as one of the critics has said, "people have egos no matter how humble they appear to be." It is simply human nature. Perhaps sometime it is more important to not to dumb down our own egos (or other people's) but to be more empathetic and reasonable working together.


In all fields, there are soloists and there are teams. There are team sports and individual sports. Michael Phelps is a soloist. S. Wolfram is a soloist... he has a backup band though...


Don't have time to watch the video, but I read through the PDF (which, btw, consists of very brief notes, so it's not really worth reading. However I think I got their basic thesis.)

I find it an interesting point of view, since I'm a total noob when it comes to programming - first year CS student, only done a few tiny VB projects before that. However, I really enjoy programming, work on a few projects in my spare time, and plan to get some decent languages (a scripting language (Perl/Python/Ruby), and maybe a functional language (Clojure?) under my belt soon. I've used OO in class projects, but I want to figure out how to use it in my own projects. I want to learn software engineering practices. I want to learn graphics programming. That's just over the next couple of years! In 10, 20 years time, who knows?

The appeal to me is that I can never peak in programming, their will always be new stuff to learn. Anyway, where was I? Oh yeah, my point was that I'm a noob now, I'm very willing to be modest and learn from others, but the hope of becoming a genius programmer I find very motivating.


Ward Cunningham is a great role model. He's super-smart, darn clever, and just a really soothing sort of laid-back guy. He doesn't have to struggle with the alienate/suffer fools conundrum because he's aware enough to be learning something from the majority of his interactions with other people. If you can achieve that, you can conserve all the energy you'd waste swatting at fools.


There's a flaw in the video around 27:55 that is obnoxious, but it seems to just be local corruption.


When you finally reach enlightenment or a state of "genius" you won't be thinking about what other people think of you.


Looking at the handout too...,

I am a bit disappoint this talk was not an effort to address the old cliche that some programmers are ten times as productive as other programmers and similar things.

I already hate kind of thinking but I would like to have hard figures to use in arguments against the kind of people who believe this bolox. I agree with statements like "lose the ego" but I don't think that would fly in arguments with my manager.


It may be an "old cliche", but also happens to be true. Maybe you can quibble over whether it's a 10X, 100X, or merely a 5X difference, but some programmers are clearly much more productive than others.


It's also true that some programmers are more productive collaborators than others. (In fact, some are effectively anti-collaborators -- less work gets done when they pitch into a project.) If you are in a collaborative enterprise, this might be something to consider.


Wow, love the opinion based moderation here.

Minus four used to be for trolls but now it seems it for disagreement? Even if you think I'm wrong here, does this post merit this treatment? (I moderate base on opinion for posts in the positive range but negative has, uh, negative connotations).

I would say that I've have been in a number of situations where one team member was indeed more far more productive than another. But as far as I can tell, this was a product, itself, of the breakdown of team-process. Either the productive programmer wouldn't tell the unproductive programmer how they did their magic or alternatively, the unproductive programmer wasn't interested in finding out.

One way to see team organization is as a way to prevent one person from always being "on the critical path". The most effective way is give skills to those who don't have them.

There might indeed be no simple metrics that will support my view but I suspect this is because it is complex than such metrics can describe.

There have been some studies lately showing that, in general, few people gain their abilities through "raw talent" rather than hard work see http://www.scientificamerican.com/article.cfm?id=the-expert-... . I guess these are what I'd fall back on for any proof. Whether person is willing to learn a given skills is whole other problem but I think these studies show that if a person is willing to learn, they are able.


No way I'm watching a 50-minute video I'm not even sure will be worthwhile. I didn't find a transcript. I had a look at the PDF and there's nothing groundbreaking in there IMHO...




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: