> unless you tell it in painstaking detail what to do and what to avoid, and never ask it to do a bigger job at a time than a single function or very small class.

This is hyperbolic, but the general sentiment is accurate enough, at least for now. I've noticed a bimodal distribution of quality when using these tools. The people who approach the LLM from the lens of a combo architect & PM, do all the leg work, set up the guard rails, define the acceptance criteria, these are the people who get great results. The people who walk up and say "sudo make me a sandwich" do not.

Also the latter group complains that they don't see the point of the first group. Why would they put in all the work when they could just code? But what they don't see is that *someone* was always doing that work, it just wasn't them in the past. We're moving to a world where the mechanical part of grinding the code is not worth much, people who defined their existence as avoiding all the legwork will be left in the cold.

> This is hyperbolic

Maybe a bit, but unfortunately sometimes not so much. I recently had an LLM write a couple of transforms on a tree in Python. The node class just had "kind" and "children" defined, nothing else. The LLM added new attributes to use in the new node kinds (Python allows to just do "foo.bar=baz" to add one). Apparently it saw a lot of code doing that during training.

I corrected the code by hand and modified the Node class to raise an error when new attributes are added, with an emphatic source code comment to not add new attributes.

A couple of sessions later it did it again, even adding it's own comment about circumventing the restriction! X-|

Anyways, I think I mostly agree with your assessment. I might be dating myself here, but I'm not even sure what happened that made "coding" grunt work. It used to be every "coder" was an "architect" as well, and did their own legwork as needed. Maybe labor shortages changed that.

> It used to be every "coder" was an "architect" as well, and did their own legwork as needed.

I disagree. I remember in the days before "software engineer" became the rage that the standard job titles had a clear delineation between the people who thought the big thoughts with titles like "analyst" and the people who did the grunt work of coding who were "programmers". You'd also see roles in between like "programmer/analyst"

Might be a big company thing then, but I'm not wholly convinced. There's a big gap between designing the outline of a big system and coding instructions that can be followed without having to make your own decisions. The question of how much of that gap is filled by the "design" vs "coding" levels is a spectrum.

I think I see what you're saying and if so we're talking past each other a bit and I agree with what you're saying as well.

The point I was raising is by the time an IC developer sees something, there's already been a process of curation that happens that frames the possible solutions & constrains branch points. This is different from saying that an IC makes 0 implementation decisions. The C-suite has set a direction. A product manager has defined the shape of the solution. A tech lead, architect, or whatever may have further limited scope. And any of these could just already be in effect at a global scale or on the specific problem at hand. Then the IC picks up the work and proceeds to make the last mile decisions. And it's turtles all the way up. At almost all levels on the career ladder, there are people above and/or upstream of you who are pre-curating your potential decision tree.

As an analogy, I once had a fresh tech lead under me where they didn't understand this. Their team became a mess. They'd introduce raw tickets straight from the PM to their team without having thought about them at all and things ground to a halt due to decision paralysis. From their perspective that's how it was always done when they were an IC in that group. The team tackled the tickets together to work out how to accomplish their goals. It took a lot of effort to convince them that what they *didn't see* was their prior tech lead narrowing down the search space a bit, and then framing the problem in a way that that made it easier for the team to move forward.

I'm on board with that framing of the process, and I see how my original formulation was too rough.

I was reacting to "We're moving to a world where the mechanical part of grinding the code is not worth much". I have the impression that in the past just mechanically grinding the code was less of a thing than it apparently is today. Guidance, sure, but not as much as seems to be common (often necessarily so) today. But I'm sure that varies with a lot of factors, not just the calendar year.

Exactly. I was channeling the stereotypical dev that says they "just want to write code". To your point they're not literally *only* writing code, but this was the sort of person/mentality I was calling out.

What it says to me is they've actively avoided what appears to be becoming the most important skills in the new world. They're likely to find themselves on the short end of the stick.

I'm with you, it's constantly doing stupid shit and ignoring instructions, and I've always been responsible for determining architecture and doing the "legwork." Unless the task is so small and well defined that it's less typing to tell the LLM (and clean up its output) then i may as well just do it myself

> what happened that made "coding" grunt work

Modern human programming has devolved to nothing more than modeling problems and systems using lines of code, procedures, sub-routines and modules, utilizing a “hack it till it works”(tm) methodology.

> utilizing a “hack it till it works”(tm) methodology.

Your post describes my coding perfectly. I don't have CS training of any type, never been formally involved in software development (recently started dabbling in OSS) and never used an LLM/agent for help (do use a local SLM for autocomplete and suggestions only).

Yet I can "code." I suspect a (pre-2023ish) software developer would likely tell me "go learn to code" if i asked for review. I don't know the formal syntax people expect to see and it has organization more typical of raging dumpster fires. Doesn't mean it's not code.

> The people who walk up and say "sudo make me a sandwich" do not.

My personal beef is the human devs get "make me a sandwich", and the LLM superfans now suddenly know how to specify requirements. That's fine but don't look down your nose at people for not getting the same info.

This is happening now at my company where leadership won't explain what they want, won't answer questions, but now type all day into Claude and ChatGPT. Like you could have Slacked me the same info last year knuckleheads...

Absolutely. Merely being a member of the business class does not magically mean one has the ability to specify business requirements much less product specifications. These are *not* the people I'm talking about now having superpowers.

I am picturing people who blend high level engineering and product skills, ideally with business sense.

  > Merely being a member of the business class does not magically mean one has the ability to specify business requirements much less product specifications

Is this not why COBOL failed? Common Business-Oriented Language sure does look much more like natural language than a lot of other code, but it could never solve the abstraction needed to do the complex things.

I don't think LLMs will ever get rid of coders. Business people can no more tell an LLM what to build than they can a team of programmers. I've long argued that the contention between the "business monkeys" and "coding monkeys" is a good one. That the former focuses on making money and the latter focuses on making a better product. The contention is good because they need each other (though I do not think the dependence is symmetric).

Maybe one day AI will get there, but I don't see how it does without achieving AGI. To break down intent. To differentiate what was asked from what was intended. To understand the depth and all the context surrounding the many little parts. To understand the needs of the culture. The needs of the users. The needs of the business. This is all quite complex and it's why the number of employees typically grows quite rapidly.

How do we move forward without asking how we got here? Why we got here? How optimizing for decades (or much longer) led us to these patters. Under what conditions makes these patterns (near) optimal? I've yet to see a good answer to how LLMs actually address this. If typing was the bottleneck I think we would have optimized in very different ways.

My intuition tells me that llm’s combined with SWE’s with really amazing fundamentals will kill the code monkeys.

And frankly? That’s the best outcome. Code monkeys (in my view that’s an individual who writes out code just to complete a jira ticket) are a liability. Not only that but each additional person you have in an org means more noise creation.

If this forces the code monkeys to level up to compete… again a good thing.

The code base should not be elongated nor complicated. I’m not even a SWE by trade, rather a CEO, and this is my preferred outcome.

I agree with your first paragraph but not the second one. In many cases it's easier for me to directly write the code that satisfies the unwritten acceptance criteria I have in my head than to write those criteria down in English, have an LLM turn them into code, and then have to carefully review that code to see if I forgot some detail that changes everything.

> easier for me to directly write the code that satisfies the unwritten acceptance criteria I have in my head than to write those criteria down in English

Yes, and for team or company code, "there's the problem".

Those acceptance criteria are guardrails for the change that comes after, and getting those out of your head into English is more important over the long haul than your undocumented short-term solution to the criteria.

Virtually all teams — because virtually all PgMs, PjMs, TLs, and Devs — miscalculate this.

Easier for you, not better for team or firm.

• • •

FWIW, perpetuation of this problem isn't really a fault of culture or skill or education. It's largely thanks to "leadership" having no idea how to correctly incentivize what the outcome should holistically be, as they don't know enough to know what long-haul good looks like.

FWIW, you can make that easier for them by having the LLM derive your acceptance criteria into English (based not only on code but on your entire conversation+iteration history) and write that up, which you can read and correct, after the countless little iterations you made since your head-spec wasn't as concrete as you imagined before you started iterating.

Even if you refuse to do spec driven development, LLMs can do development-driven spec. You can review that, you must correct it, and then ... Change can come after more easily — thanks to that context.

> Those acceptance criteria are guardrails for the change that comes after, and getting those out of your head into English is more important over the long haul than your undocumented short-term solution to the criteria.

I have a lot of context about the system/codebase inside my head. 99.9% of it is not relevant to the specific task I need to do this week. The 0.1% that is relevant to this task is not relevant to other tasks that I or my teammates will need to do next week.

You're suggesting that I write down this particular 0.1% in some markdown file so that LLM can write the code for me, instead of writing the code myself (which would have been faster). Chances are, nobody is going to touch that particular piece of code again for a long time. By the time they do, whatever I have written down is likely out of date, so the long term benefit of writing everything down disappears.

> after the countless little iterations you made since your head-spec wasn't as concrete as you imagined before you started iterating.

That's exactly the point. If I need to iterate on the spec anyway, why would I use an intermediary (LLM) instead of just writing the code myself?

You also write it down so that the bus factor goes down.

And if nobody touches the code for a long time, how can the documentation be out of date?

  > getting those out of your head into English is more important over the long haul than your undocumented short-term solution to the criteria.

I think there may be miscommunication going on, or I may be misreading the conversation. What I do not know is what valicord means by "satisfies the unwritten acceptance criteria".

In one interpretation, I think they make a ton of sense. We invented formal languages to solve precisely this problem. The precision and pedantic nature of formal languages (like math and code[0]) is to solve ambiguity. If this is the meaning, then yes, code is far more concise and clear[1] than a natural language. That's why we invented formal languages after all. So they may be having trouble converting it to English because they are unsatisfied with the (lack of) precision and verbosity. That when they are more concise that people are interpreting it incorrectly, which is only natural. Natural languages' advantage is their flexibility, but that's their greatest disadvantage too. Everything is overloaded.

But on the other hand, if they are saying that they are unable to communicate the basics (it seems you have read in this way) then I agree with you. Being able to communicate your work is extremely important. I am unsure if it is more important than ever, but it is certainly a critical skill. But then we still have the ambiguous question of "to who?" The type of writing one does significantly differs depending on the audience.

Only valicord can tell us[edit], but I think we're just experiencing the ambiguity that makes natural languages so great and so terrible. I think maybe more important than getting the words out of ones head is to recognize the ambiguity in our language. As programmers this should be apparent, as we often communicate in extremely precise languages. But why I'd say it is more important than ever is because the audience is more diverse than ever. I'd wager a large number of arguments on the internet occur simply due to how we interpret one another's words. The obvious interpretation for one is different for another.

[0] Obviously there's a spectrum with code. C is certainly more formal than Python and thus less ambiguous.

[1] Clear != easy to understand. Or at least not easy to understand by everyone. This is a skill that needs training.

[edit] Reading their response, I think it is the first interpretation.

This is the point I'm raising. I agree with you, but what I'm saying is I think the skillset you describe is the next on the chopping block.

The acquaintances of mine who are absolutely *killing* it with these tools are very experienced, technically minded, product managers. They have an intimate knowledge of how to develop business requirements and how to convert them into high level technical specifications. They have enough technical knowledge to understand when someone is bullshitting them, and what the search space for the problem should be. Historically these people would lead teams of engineers to develop for them, and now they're sitting down and having LLMs crank out what they want in an afternoon. They no longer need engineers at all.

My contention is that people with that sort of skillset will have an advantage due to their experience with skills like finding product fit, identifying user needs, and defining business requirements.

Of course, the people I'm talking about were already killing it in the old paradigm too. I'll admit it's a bit of a unicorn skillset I'm describing.

It's almost as if architecture and code quality mattered just as before and that those who don't know proper engineering principles and problem decomposition will not succeed with these new tools.

If you're using words in English to "tell it in painstaking detail", you're doing it the hard way.

You can provide it with a tool to do that. Agents run tools in a loop, give it good tools. We have linters, code analysers, fuzzers and everything else.

Configure them correctly, tell the agent to use them (in painstaking detail) and it can't mess things up.