> I think it's revealing how limited the LLMs were in algorithmically optimizing this problem vs approaches like "add some parallelism" or "use numpy", for which they were quite decent in Python and somewhat less decent in Rust. It's surprising to me that unless I prompted it overly-specifically, GPT 4o didn't suggest using a faster rand implementation, as that was a pretty effective, easy step.
As a human performing optimization one of the steps might be to generate a list of candidate optimizations. After that you might explore each idea in parallel. By using the LLM iteratively, you're side stepping that approach and constraining the LLM to instead picking a certain optimization first and then build on that. So one cure might be to adjust your prompt and iteration strategy to take into account that there are multiple candidate solutions rather than just one. This leads to more of an agentic / chain approach than a completions approach to the task though.
The other missing part of this is that each time you optimize the code (even if it's a failure), you learn something more about the optimization. Again this context seems missing when the LLM looks at the code (unless that history is provided in CoPilot completions automatically - it probably depends on what approach you're taking there).
I wonder if there are any AI tools out there doing context + planning + history + RAG + test + evaluation + tool use + feedback approaches. I've seen various tools do 2-3 of those. It's not a stretch to think that we might eventually see tools that start to emulate more of the developer inner loops.
> I wonder if there are any AI tools out there doing context + planning + history + RAG + test + evaluation + tool use + feedback approaches. I've seen various tools do 2-3 of those. It's not a stretch to think that we might eventually see tools that start to emulate more of the developer inner loops.
As a human performing optimization one of the steps might be to generate a list of candidate optimizations. After that you might explore each idea in parallel. By using the LLM iteratively, you're side stepping that approach and constraining the LLM to instead picking a certain optimization first and then build on that. So one cure might be to adjust your prompt and iteration strategy to take into account that there are multiple candidate solutions rather than just one. This leads to more of an agentic / chain approach than a completions approach to the task though.
The other missing part of this is that each time you optimize the code (even if it's a failure), you learn something more about the optimization. Again this context seems missing when the LLM looks at the code (unless that history is provided in CoPilot completions automatically - it probably depends on what approach you're taking there).
I wonder if there are any AI tools out there doing context + planning + history + RAG + test + evaluation + tool use + feedback approaches. I've seen various tools do 2-3 of those. It's not a stretch to think that we might eventually see tools that start to emulate more of the developer inner loops.