- Human describes the problem specs in vague English
- LLMs generate some solution based on the texts it has seen
(LLMs may generate non-factual information here but assume the output is factual)
- Human refines the prompt to get closer to "the thing he has in mind"
- LLMs still tries _to the best of its effort_ generate a "close" solution
- and we loop here
Assume the specs of the problem is fixed. Why LLMs couldn't generate the output the Human "had in mind"?
Because English description of the problem was not deterministic?
What can we do? Invent a language for automation specification. Then LLMs can accurately churn up a solution in one shot.
But..that "deterministic language" is what programming languages are. So LLMs at this current state are basically a transpiler from rough-English to a deterministic-English.
But, can LLMs be thought only the specs of a programming language (like C) then generate assembly code from a valid C code? In effect, can LLMs be exact compilers not English transpilers?
- Human, named ProductDesigner, describes the problem specs in vague English
- Another human, named Programmer, generate some solution based on his expirence.
- ProductDesigner, refines the spec to get closer to "the thing he has in mind"
- Programmer, while annoyed, tries to his best effort to write a close solution
- And we call it iteration of development
Is it really that different from your experience? Ok, maybe not your experience, but in the world of outsourcing works it's really kinda like that.
The solution to the above is never "ProductDesigner should learn to speak a deterministic language." It's "when Programmer feels the spec is too ambiguous, he should give feedback to ProductDesigner."
You can't eliminate the "vagueness of natural languages" from the equation. The only differences are, to me, 1) AI is really not that smart yet. 2) AI doesn't admit it doesn't know something. Humans are reluctant to admit it as well, but as least we do it when it's necessary.
I haven't had this from ChatGPT and haven't used paid ChatGPT to test GPT 4.0 there, but with Bing (Creative) specifically I've had it regularly tell me it doesn't know something. It still hallucinates other things, but it's no longer just a lying yes man. If you ask it for information it can't find or a task it can't do, there's a very good chance it says "Sorry, I can't do that."
The other day I was trying to find out what percentage of Konstantin Tsiolkovsky's works were published before the age of 60 vs after the age of 60, something that's almost impossible to find on the internet (I don't speak Russian). I spent a lot of time trying to get Bing to answer it for me (didn't succeed), and one of the times I asked it to look up all the bibliographic information it could on Tsiolkovsky, go through each file and count the dates of publication, then tell me how many were after his 60th year. It said "Sorry, that would require me to download and analyse a large number of PDFs, which is outside my current capabilities. I'm sorry I couldn't be of more assistance this time. "
They've improved a lot on the Yes Man aspect of it.
It will still often tell you that it can do things that it can’t or vice versa, it’s like it’s guessing whether it might be able to do things or not based on what humans told it that it could do in the training process rather than actually being aware of its own capabilities
Traditional computing is good at being precise at the cost of being rigid, ml computing is good at being flexible at the cost of being ambiguous.
I don't trust ml computing to be precise in the same way I don't trust humans to be precise (we can all potentially write code with bugs) because the process is fundamentally non deterministic.
The way we solve this with humans is to create tools that test verify what we make, backed by some sort of proof.
I guess once ml computing can get feedback from these tools, the situation will improve?
thats not quite it.. it is non-deterministic plus the way the parts are re-ordered is flexible in ways that are not obvious, and do result in nonsense sometimes. More specifics are according to the engine(s) in use..
I believe this framing is a bit over-simplified.This description presumes that the Human does indeed have the entire solution in their head, and the only problem is expressing it in a language.
In reality, Human only understand a fraction of the necessary solution (maybe even a fraction of the full problem!) when they begin to program the solution. A big part of the iteration is discovering things the Human did not know about the problem or the necessary solution. Eventually we converge on A solution, which may not look very much like the originally envisioned solution at all.
So no, I do not think the LLM is functioning as a transpiler for english. I think its a tool providing feedback in a necessary iterative cycle or discovery and development. The iteration itself is the "transpiler for english", in a crude way.
The problem is that the "human" cannot accurately express himself in English, or cannot define the whole problem scope. Mastery of the English (or whatever) language that the Enterprise is using is more important than technical expertise. This cannot be stressed enough. Correct and extensive technical specifications are as hard as the code/stack itself.
The human here is acting as a compiler for the real world. In each iteration, it's raising exceptions and errors for the AI. The iterations will continue until the AI compiles the correct output.
You are absolutely right that using natural language to specify problem means LLM has to guess a bunch of things. And obviously the guesses are not fully accurate. So we go back and forth. But guess what, it gets 80/100 (if not more) assumptions right in a typical code problem. Majority of them boilerplate code assumptions. Thus you end up focusing on just the most important decisions and the LLM acts as a companion to you in a way a pair programmer does. Thus the ambiguity of the language is what benefits the system here.
The power of LLMs is the fact that they use natural language. It’s like bashing on pipe based string processing that enabled all the Unix applications in the beginning. The simplicity of the system is what powers the entire thing. You cant take it away.
What LLMs solve for programming are a class of low-hanging-fruit problems: things that in many respects could be done by hand, but are too expensive to bother with relative to the problem's size.
If you have a one-shot problem, you may be able to avoid making the actual program and just jump from problem to solution. A lot of the systems that programmers design are potentially representable instead as new iterations of that one shot problem, and therefore something no code need be written for.
This does run a risk of getting a wrong answer infinitely fast, but that has never not been the case with computers, it's just a case of now having a kind of "level 3 autonomy" where you may need to abruptly grab the wheel and get the answer yourself.
In a recent conversation with a buddy after I tried ChatGPT4:
-ME: It took my requirements for a simple webapp, gave me the front-end and back-end code. Then it tweaked the code appropriately based on my comments and bug reports, without arguing back or asking why I'm so dumb. In 10 minutes I went from idea to prod potential.
-HIM: Effing thing is going to put me out of a job...
-Me: PMs who jump on the bandwagon and train people in how to write good, detailed and precise specs will be fine.
-Him: Hahahaha. No one writes good specs. I’m safe for now.
I asked ChatGPT “Let’s say I want a program to delete unused files. Can you write a detailed specification that a developer could use?”. I was as vague as possible on purpose.
Here’s what it gave me:
“Program Name: FileCleaner
Overview: The FileCleaner program is designed to identify and delete unused files from a specified folder or directory. The program scans the directory for files that have not been accessed for a specified period of time and prompts the user to delete them. The program is intended to free up disk space and improve system performance.
Requirements:
The program must be able to scan a specified folder or directory for files that have not been accessed for a specified period of time.
The user should be able to specify the time period after which a file will be considered unused.
The program should prompt the user to delete the identified unused files.
The user should have the option to select which files they want to delete.
The program should permanently delete the selected files from the system.
The program should log all deleted files, including their location and name, in a separate file for future reference.
The program should have a user-friendly interface that is easy to navigate and understand.
Functional Specification:
The user launches the FileCleaner program.
The user is prompted to select the folder or directory they want to scan for unused files.
The user is prompted to enter the time period after which a file will be considered unused (e.g., 30 days).
The program scans the specified folder or directory for files that have not been accessed for the specified time period.
The program displays a list of unused files, including their location and name. The user is prompted to select which files they want to delete.
The program permanently deletes the selected files from the system.
The program logs all deleted files, including their location and name, in a separate file for future reference.
The program displays a message indicating that the selected files have been successfully deleted.
The user can choose to exit the program or scan another folder or directory.
Non-Functional Specification:
The program must be designed and implemented using a high-level programming language, such as Python or Java.
The program should be able to handle large files and directories without crashing.
The program should have a user-friendly interface that is easy to navigate and understand.
The program should be able to run on multiple operating systems, including Windows, MacOS, and Linux.
The program should be secure and not pose a risk to the system or user data.
The program should be well-documented, with clear and concise instructions for use and troubleshooting.
By following these specifications, a developer can create a FileCleaner program that efficiently identifies and deletes unused files from a specified folder or directory.”
This could be improved even more by having it ask questions to narrow down exactly what is needed
Jesus fucking christ. I gave it these specs and it spat out a perfectly fine CS101-style python program that worked.
Then I told it I wanted a GUI, so it started writing a perfectly fine CS101-style tkinter program. I couldn't get pip to work, but chatgpt solved that for me too.
Then I told it to rewrite it in rust. It generated plausible code, but it couldn't finish the problem before getting cut off. And I was unable to get it to continue where it left off.
I think you are taking a very major assumption that LLMs are deterministic infact they are exactly opposite. They are probabilistic systems.
- They do not transpile Rough English to Deterministic English. Infact they do not do any transpiling at all.
- LLMs learn the probabilities of words in the dataset for all contexts from that dataset (context is a ordered set of words). This is called training.
- Once training is done, LLMs can generate text given a prompt and the probabilities that it has learned. The analogy of LLMs being auto-complete on steroids is very apt.
- Whether the text generated by LLMs is factual or not is purely coincidental.
A lot of things which seem like magic about LLMs will get demystified.
Now one can argue that LLMs are showing human like reasoning/intelligence/sentience as an emergent behavior. This is hard to argue against because all these terms are extremely hard to define.
IMO, the only emergent behavior that LLMs are showing is the output they generate looks like it might have been generated by a human which should not be surprising given that LLMs like ChatGPT has trained on a large amount of human written text available on internet.
On your first point you couldn't be more wrong. LLMs are deterministic. They are run on a deterministic machine and the entailment of that forces them to be so. Also probabilistic does not mean the absence of determinism. For example, for every non-deterministic/probabilistic finite automata there is an equivalent deterministic one.
LLMs are fully deterministic in that sense: same input, same outputs.
Because full determinism is not always desirable, the researchers have implemented an explicit "temperature" parameter that you can use to inject randomness to the outputs. If you set that to 0.0 you will always receive the same output for the same input and model version.
LLMs can be implemented to be formally deterministic but if you ask them to solve a specific problem instance you have not seen before, you cannot generally guarantee they will do so reliably. So you're correct in a pedantic sense but I think GP's perspective is more useful if you are problem solving.
It's not pedantry. The parent commentator is simply stating something incorrect and its very misleading at a conceptual level for people who won't think about it too much. And you can guarantee they will do so reliably unless you parameterize the input with some "truly" (yea i know) random input.
> - Whether the text generated by LLMs is factual or not is purely coincidental.
No, because the probability of a word on the internet being factual is not coincidental. Factuality compresses the corpus; the truth is generally the simplest explanation for a set of observations. (The collected text of the internet is a set of observations about reality.)
> IMO, the only emergent behavior that LLMs are showing is the output they generate looks like it might have been generated by a human
The whole point of the Turing Test is to stop people from asking "yes, it acts indistinguishable from a human but is it human?" "Generating output that looks human" is in fact the entirety of AGI.
> > it wouldn't be so incredibly difficult to determine.
> Never said it was easy.
My point is that this process of determination happens over time and in a non-linear fashion, so the corpus contains tons of noise around any given truth statement.
> > The corpus of written language is full of ambiguity and contradictory statements.
> Right, but the truth is the one set of information that logically cannot be contradictory.
I have no idea if an LLM specifically is the tool for the job or not, but presumably with the input of the source code of millions of compiled executables and the compiled output, "an" AI system can learn to be a compiler. If it can then analyse the generated programs for things like performance, size or other metrics of compiler goodness, it can compile "better".
Hopefully it won't make any mistakes when you come to use it because that will be the devil's own job to track down: compiler bugs are bad enough as it is!
But then we can conceive of a compiler bug-finding AI...
Every new generation of LLMs can write code better than the previous ones. It's really just a matter of time as the models keep improving. They can already generate basic solutions in one shot if simple enough and if enough examples were present during training.
Complexity is the problem. This is what humans, and now LLMs, struggle with when it comes to designing and coding solutions. Managing complexity is why we've attempted to invent paradigms such as OOP, which some claim only makes things worse.
I think it can be easier to give a napkin sketch / whiteboard image / wireframe to get to the right UI faster, than having to verbally give very specific directions for every little element.
I was wondering the submit pic option on GPT-4 actually would be this well and perhaps out HTML/CSS to do it - maybe not possible today, haven't tried it yet.
Or who knows - maybe it will be be a like a senior product design who will just figure it all out, but that would require a long convo / submitting a design brief / customer profiles, etc.
You don't need to loop with the LLM multiple times - after a couple of prompts and adjustments you can accept the suggested code and make any modifications you want yourself because at that point editing the code by hand would be faster and more precise that expressing every detail to the LLM in English.
That way you're gaining productivity from the all the code generation the LLM does for you and you have final say on all the committed code, it seems like a win-win.
Even giving instructions to a human programmer, a few iterations will be needed to refine the requirements, clarify ambiguities, and get the desired system.
Well... yes? I don't know about this 'flow based programming' per se, but what you describe is indeed how one uses ChatGPT for programming. I don't understand the pushback on every article like this. How aren't there obvious major productivity gains when a computer can generate 80% or more of your solution, and you just have to fine tune it?
Let me give you an example of what I did just this morning. In fact, ChatGPT is generating code in another tab as I'm typing this - GPT4 is quite slow (well, at least it takes longer than my attention span which has been reduced to that of a sea urchin by years of social media and usage of internet in general), especially when you let it create 50 to 100 lines at a time.
I'm working on a specialized ETL tool for geospatial data. In fact, I've had several iterations of this tool over 5 or more years, but for the last few months I've been working on a new version that incorporates everything I've learned about the domain in that time. This conceptualization, the learning of what the tool should do and how all components fit together, is not something a computer can do. I would have to be able to describe it to get a computer to generate it, of course, but I couldn't a few years ago because I was still finding out what it should do to provide the most value. The reason I had several versions is because I kept learning more about what I need it to do.
But many of the boring machinations I absolutely can have a computer do. For example, a part of this tool is a tiny DSL which is basically a way of specifying combinations of parameters with some dress-up. I'd been dreading making a proper parser for weeks, since a) the 'design' of this DSL has fluctuated quite a bit, and b) I'd have to dreg up my knowledge of parsers from the last time I wrote one 10 years ago. So I've been getting by using a combination of regex, split() and some hand massaging of strings.
But this morning, I had ChatGPT write me the biggest part of a parser using three questions:
- "Write a parser in python for the following format: [optional identifier] [open square braces] [either a list of values, separated by commas; or a list of key-value pairs, separated by commas, the keys and values are separAted by equals signs] [close square braces]. The identifier is optionally followed by an 'at' sign, and a qualifier after that. Show me the EBNF and a parser in python that implements this grammar."
- "The qualifier should come before the square brackets, it's part of the identifier."
- "Write some unit tests for this grammar. Include corner cases for all varieties of the allowed input, so with or without identifier, with or without qualifier, with plain lists and with lists of key/value pairs. Include cases for 0, 1 and 3 values in all places where more than one element can appear."
(try it, I used GPT4 but you'll probably get similar results if you use the free version)
The result I got from this has 2 mistakes: in one case I had to swap two tokens in the EBNF. This one I found by reading the docs while waiting for the output of whatever ChatGPT suggested as a fix after me copy and pasting the error message I got. So this was fixed, by me, in a few minutes (ChatGPT's suggestion for a fix made no sense). The second one, ChatGPT suggested a correct fix after I copy and pasted the error message also.
So, ChatGPT gave me a skeleton grammar, complete with the syntax of the library which I had never heard about 2 hours ago. The second of my first three questions was because I gave it ambiguous input, I only thought of that part while I was writing the question and didn't bother to go back and edit it in in the right place where it would have been more obvious. Then I copy/pasted the raw output from ChatGPT in a new file, ran it, I got the issues fixed (in collab with ChatGPT) in 20 minutes or so. This would have taken me all morning and probably all day if I include all unit tests and all other details just a few months ago. How is this not a massive game changer?
Now, I would not have been able to do any of this hadn't I already known what an 'EBNF grammar' was, and hadn't known how to read and implement one. This tool will not replace programmers. It will however eliminate the need for the very low skill ones, and make the rest (much) more productive.
What would you estimate as the industry-wide productivity increase for programmers from this tool? I've seen estimates anywhere from 1.5x to 10x from friends, which astounds me because in any other context whatsoever a 50% productivity increase is absolute gold, let alone the wild and frankly unbelievable higher numbers.
Something I'm really interested in, after we've gotten the full 32k context and multi-modal capabilities of GPT-4 and done all the Langchain/ReAct shenaniganery we can, is GPT-5, or 6. These things are coming so fast and delivering step changes in productivity, it really does feel like something new.
Oh I don't know about that, I can only comment on my own experiences. I don't have sufficient breadth of experience to be able to say how much of this would be applicable to other developers in other subfields. I do however, like you, look forward to multi modal capabilities. I fully expect it to be able to take a Balsamiq mockup and generate the code to set up the UI for that mockup, with handlers for widget events and all. I've already had it generate UI's from text descriptions, which was cumbersome (relatively speaking - hedonistic adaptation strikes hard here).
Wrt to productivity - I do think it's misleading to look at it like 'it used to take me 5 hours, now only 1, so that's 5x' or some other (imo naive) quantification. It's much more about removing drudgery and thus freeing up mental resources for interesting things. Say I do a task with a certain tool in the same time it took me without it, but in a much less mentally taxing way (because that's what drudgery does to me - activate bore-out mode). From the naive metric pov there is no productivity improvement. But if that leaves me capable of doing more in the rest of the day/week because I don't have to fight myself to move on with something, I'd argue it's still a productivity gain.
Does typing with a nice keyboard and in a good chair make me more 'productive'? Well, I don't think anyone will argue that I can't type on a e5 Chinese crap one while sitting on a bucket before a door on two sawhorses. I can even type and think equally fast in those circumstances. But better tools still make one more 'productive' in the larger picture, I don't think anyone with experience will deny this either. This 'productivity gain' is in that sense somewhat ethereal, in a poetic sense of that word. To me that's where these tools are at now.
One of the interesting things with with programming as field is that 2x-10x productivity gains are _common_. New abstractions and tools regularly allow people and teams to build more, quickly. The goalposts of what is expected to be delivered, and the complexity of systems expands to fill the time available to software development teams, but they are delivering more, especially if you step back and look at things on any given 10-20 year timespan. So Codex will provide big productivity boosts, but so do moving to C from assembly, or Ruby on Rails, or react vs jquery-spaghetti. Open-source and commercial APIs extend this further, especially when interacting with things like money or the physical world.
IMO the biggest unknown is the subset of people who will have near infinite increase in productivity because it opens a door that was completely closed to them previously. Otherwise incredibly talented people in art/manufacturing/education/etc. who have a lot of domain expertise could be the ones who really come up with the coolest new uses of the tech.
Yes: https://platform.openai.com/docs/guides/chat . I wrote a small proof of concept plugin for vim last week, was quite easy to do with their Python library to access this API. I have no experience accessing the raw endpoints.
Not sure why I'm downvoted - I literally used it last week. I got an API key from my account and made several calls to it, using the OpenAI Python library.
That is more or less the state of all plans on putting llms into anything useful. I think the lack of and education of language parsing ambiguity is the main reason why so many people are buying the hype here.
>Siri call me an ambulance.
Is not a solved problem and will never be if you're using English as your input language.
I was not bullish on AI generated code until last week.
Last week I asked GPT to write a C version of dirbuster using the POSIX api. It returned a nearly correct solution (paths weren’t prefixed with /).
Then I asked it to rewrite the program using threads. And again, it kicked out a nearly correct solution (same path bug).
For each it gave me instructions for compiling and told me how to run the binary.
Then I asked it for a word list, and it gave me a GitHub repo to clone and told me how to use it with the binary I just compiled.
Then I described the behavior of the service I was scanning (always returns 200 with status=ok when another service would have returned 404) and asked it for a git diff of the threaded version of the program that worked with this service.
Then I described the missing / prefix and it gave me a gif diff for that too.
In both cases the git diff was valid.
And, after applying the git diff and running the binary against the word list, the program scanned the service finding the /ping endpoint.
My limited experience so far with ChatGPT for programming is mixed.
On the positive side, I found it is like a search engine on steroids. It works SO much better than Google at helping me find something.
Here is a really, really dumb example. I am coding in PHP and I know Laravel has a dd() and dump() methods. I know that from doing a simple Laravel repo once.
So I'm like, okay, there's gotta be a package that I can just add with composer. But what is it? I ask Google:
"what package do I need to use the dd method in php?" (or something along those lines I don't reemember exactly)
Google just gives me a bunch of StackOverflow posts that are related to Laravel, that sometimes mention the dd() method. None of those links help me.
So I was like, what the heck letś try ChatGPT. I ask the same thing and this mofo just straight up tells me:
- I can use the symfony/var-dumper package
- I can include it with the following composer require command...
The hell?
So I know some of you will be like, "you could have searched xyz"
Thing is, everybody is different. I can´t at any one time know all the possible "ideal" ways of finding something. Every website like packagist has their own search.
So for me at least ChatGPT hasn't helped me much producing code, because it takes too much effort to prompt it.
However, it works really well at helping me find answers.
Things like "show an example conditional in bash that tests for..." works really well.
So long story short I think the immediate value of these LLM for us developers is time saving, taking out the frustration of trying to find information. As a developer you need to trawl through so much APIs and frameworks and docs and searching is a huge time sink.
On the other hand, I used ChatGPT to try and find the name of a book I'd forgotten, and it straight up lied (listed the synopses of a few similar books in the genre, with the plot points I asked it to search for inserted in the middle of the synopsis).
I haven't tried it with code yet, but I have heard of similar things happening where it fabricates method signatures, packages to import, and so on, wholesale.
It's nice if it has helped you, but I remain distrustful of LLMs.
Yeah for searching movies, for example, it is terrible. I also don't really see how it is going to get much better than it is since it inevitably seems to try and steer its way to non-obscure stuff even when you are explicitly asking for something obscure.
Just for example. I was trying to remember a film that is basically a knock off of war games involving two teenagers using a scientists small helicopter to do stuff. I typed into Google "film with small helicopter" and it was not only the first result, it was in a little cut out section highlighted at the top. The film is called Defense Play and it has a grand total of 144 votes on IMDb. Incredibly obscure.
I thought it would be fun to ask ChatGPT roughly the same question and at first it just gave me a list of very popular movies with large helicopters. So I said, no this has a small helicopter or a toy helicopter in it. And it gave me a list of films with helicopters but three in Toy Soldiers, Small Soldiers (which probably does have a small helicopter in it) and GI Joe (which is based off a toy and has a helicopter probably).
No matter how much I insisted that this movie is obscure it can't quite figure out how to do anything about returning things that match something as vague as "obscurity" even though there are measures (e.g. IMBd votes) available that could reasonably give it such a notion. It probably does know about the movie, but whenever I ask about obscure movies while it often can give me a year and the first few actors in it it does seem to make up the plot even when asked to just paraphrase whatever plot synopsis it has available to it.
I do see it improving but I also wonder if a model which is heavily weighted towards guessing the next section of text will have difficulty with the unlikely scenario of someone being specifically interested in an obscure film a handful of people have ever bothered to watch. Seems really useful for searches where a person doesn't really know what they want, and very not useful in searches where the person knows they need a specific and obscure piece of information potentially.
Well I heard this argument on ThisWeekInTech recently, someone made a good point: we will learn that LLMs, just like search engines are NOT a repository of truth.
Do you expect Google to return only "truthful" and "factual" results? I hope not! Do you want a live in a one sided world where you can only hear things that have been vetted for approval by some government/org?
We are pretty used today that Google returns a bunch of stuff, and we pick the things we find useful and we have to chose ourself what is reliable sources and so on.
I think part of the distrust here may come from the fact that they design this as a human-like chat interface and make it seem like ChatGPT is a person. And that sets the wrong expectations. So I'd agree with your sentiment in that the providers of these LLMs need to stop trying to make them look like reasonable people.
Unfortunately right now it's really exciting tech and the media especially is guilty of passing it off for what it isn't and trying to get all the clicks with their doomsday articles about AI taking over all our jobs, or AI teachings us "false" things and whatnot when LLMs are really just an sort of multiplication matrix that can find data points in a huge data set and interpolate in between in a useful way.
I don't understand the insistence on anthropomorphzing ChatGPT. It didn't "lie", the algorithm just didn't translate the prompt you gave it to the result you desired. Doesn't that happen all the time with google searches also or with regular expression patterns that don't quite turn out to match what you wanted?
ChatGPT, Google's search engine, and a regular expressions algorithms are tools. Assessing them in turns of "trust" seems strange to me. I think it is better to think of them as being useful (or not) for particular types of problems.
This probably can be solved with better tooling. ChatGPT is probably "less confident" when it comes to these kind of false positives and could say so explicitly.
What I think is happening with people though, is there is a missing context. With a google search that doesn't return the expected results, you can clearly see it. The results are for unrelated things and you know you need to amend your approach to try something else. When you get linked to some kind of misinformation article ... people do complain and are upset. With ChatGPT you sometimes are missing the feedback to know that further work is required to get the desired outcome.
I don't disagree about context being helpful but I'm not going to get upset at ChatGPT as if a person had "lied" to me by withholding information that clearly would have helped. I have different expectations for a person than for an algorithm.
Because it's not a search engine, and it is lossy intentionally. Things that happen statistically more often occur in higher weights in the neural net, things occur less often can get lower weights.
- Flow-based programming enables a more composable way of developing software which avoids a lot of errors coming from incorrectly connecting parts of code together, making it more safe for allowing AI to the loop, and designing the graph of connected components.
- The graph is also more easily understood by humans, making the verification of the produced program easier to do for a human.
So in summary: Having an easier to understand and more composable way of developing software helps both for collaboration between humans, but perhaps extra much so when letting "still kind of unproven" AI take part.
In some ways, this has always been good programming design. Have clean interfaces. Encapsulate state, and have clear propagation of state between components. Identify appropriate abstractions that represent the components of your system such that they can be composed, potentially reused, and can be clearly reasoned about and tested. Pulling this into the explicit programming framework rather than just being a set of good design patterns does seem like it help when collaborating with semi-trusted participants. This is how most "low-code" systems work.
Yup! Similar approach would certainly work with more traditional programming methods as well. But FBP enforcing that separation of concerns makes it easier to split the workload between GPT and human.
I like to first ask Chat GPT to write a prompt that would help it accomplish <task x>, then I check the prompt and edit it if needed, then use the prompt to accomplish the task.
I'm realizing people code for different reasons. There's code used to achieve means and code written for self. I fall into the latter category which makes the thought of AI assisted code offensive.
From my perspective, computers are ideal for automation and calculation. Humans are (generally) better suited for problem solving. One requires precision and the other approximation.
Until recently, I would have taken your second paragraph for granted; but, somehow, we have ended up in a dystopian future where humans are going to have to be directly involved with all the boring tasks that require careful correctness and we'll get to outsource many of the high-level creative tasks to computers (lol).
The output is an approximation. However, the intention is to produce something exact. Further, the computer is not actually approximating. The relationships between input and output are imperfect which results in an approximate output despite exact calculations.
You're not alone. I program for fun, mostly, and I'm completely uninterested in these talking computers. I'm a little wistful that my hobby has gone from a viable career path to just a hobby, but I believe the talking computers systems will write better software, so I can't be too sad, eh?
Now that the machines can think and talk, the only real problem left (to a first approximation) is selection of the options they generate, as describe sixty years ago by Engelbart in his classic, "Augmenting Human Intellect: A Conceptual Framework" https://dougengelbart.org/pubs/augment-3906.html
I'm going to repeat that because it's as important as it is counter-intuitive: the machines will think better than we can, so all [solvable] problems are solved except the problem of intelligence itself: "What is the best option?"
"but I believe the talking computers systems will write better software, so I can't be too sad, eh?"
I hear you and actually agree that most coding will eventually be performed by AI. However, I'd bet against AI/ML ever being capable of original thought. One must first move past convention; something that is diametrically opposed to the current AI/ML approach. Time won't change this. A new paradigm might.
Either way, the most powerful commodity (and scarcest resource) is and has always been... original thought.
From Information Theory we have the result that the unpredictability of a message is a measure of it's information content. Right now we are concentrating on making systems that generate predictable results (the novelty is not just that they can generate realistic "thought", but also that they can do it "to spec", which is another way of saying that we can predict some of the aspects of the output of the machines: it looks like they understand us and are providing salient responses.
So it's not that computers can or cannot make original thought, it's that we have to decide what parts of the outputs are original: it's an observer-dependent variable, so to speak? What do you think?
Fair play. I'm not familiar with the "cut up" technique. Thanks for the share. Your point is definitely valid. Might the phrase "novel thought" better describe the intention than "original thought"?
"So it's not that computers can or cannot make original thought, it's that we have to decide what parts of the outputs are original: it's an observer-dependent variable, so to speak? What do you think?"
This is a complicated statement. Novelty is certainly observer dependent. I suppose there should be two classes of novelty that consider context and perspective. Something can be new to the user (novel) and new as a solution (Novel). Though, there are also permutations to consider. For example, every year new users get online, but the Internet isn't a new solution.
I think we'd have to define what merits the designation of "novel thought" and the context before discussing the possibility.
It's an intriguing thought to consider regardless.
I feel nobody seems to be talking how LLMs can be used in training humans to teach better. fundamentally the process of getting a desired outcomes from such a model involves nudging it to ask and answer better questions right? so why don't we use it as a training tool across domains. Teacher training across multiple subjects could really use this!
I have built humans in the loop feedback models before and they are always very targeted to a specific task. This approach modular and intuitive. I think the scope is too small though.
I spent the weekend starting a project to use this approach using a GPT model and FlumeJs (https://github.com/chrisjpatty/flume) but now that I see noFlo i am excited to try it
No thank you. Tried to get help from ChatGPT twice to get help solving some issues. Produced code both times that did not solve the problem and did not compile at all. At least it nicely described what the functions and the macros were supposed to so. Also have read somewhere that 70% of its output is valid, while the rest is bullshitting. Don't have time to fact-check and trying to fix the bad code.
Yeah it’s not there yet, it only works for very basic things you may as well write yourself. I don’t think we’ll get there till we find a generic way for LLM type models to run experiments themselves and understand the changes.
If you give it a reasonably well specified and well understood problem, solvable in a page or two of code and let it run in a loop (giving feedback) it will write and debug the code by itself. The output might not be "great" but it tends to be serviceable.
The problems with copy/pasting from a ChatGPT window are often environmental; modern computing environments are diverse and not well suited to one-shotting problems. I find it quickly corrects its own hallucinations under feedback and can understand what it needs to fill in. My experience has been that the "bullshit" problems go away when there's a concrete problem and reliable feedback.
This is not hard to automate and I imagine we will see TDD-based LLM "programming loops" where the human specifies problem, LLM generates tests, human supervises / extends tests, then LLM generates code to solve problem in the next 12mo. I feel like I can guarantee there are people working on frameworks for this right now.
Bear in mind that we are very early on the whole journey and that LLMs are likely to be only one part of a solution that can solve bigger problems.
The obvious problems using an LLM naively are the "context window" and deciding on appropriate sub-problems. Being able to create good working memory seems like a priority, whether that is vector databases or some other thing. I don't think it will be able to work in "novel" domains for some time and will not generally compete with skilled programmers yet.
But with the right architecture invoking it I'm highly confident that even today's LLMs will be able to bash out crud apps up to say 50kloc from specifications alone, jankily this year and reliably within 3 years. It's hard to predict just how good it will get and how fast, without understanding what further fundamental advances are possible.
[Note I'm just describing solution domains where "traditional" code is the target. I think this is where a lot of programmers suffer from a "skill curse". They think code is needed to solve problems and forget how a McDonald's can profitably emit burgers using nondeterministic teenagers, even if they occasionally forget your fries. It may be the case that "code" itself turns out to be a dead end or "expensive optimisation" for very many business problems. For example, a lot of business runs on forms and they are a staple of in-house applications. For many form processing applications it may make more sense for an LLM to just process a form or request directly and produce side effects and outputs based purely on instructions from the business without bothering with the nuisance of code at all. It will be interesting to see which direction gets more love - LLM just solves problem instances at runtime or LLM doing work at compile time for cost reduction and determinism.]
I was able to get Bing to produce those, but only pretty simplistic ones. And as mentioned, the results were better in the fbp language than in JSON (most JSON wasn't even fully valid).
I wonder if it makes sense to produce some sort of neo-COBAL that produces a bridge between flow and declarative syntax that better aligns with expressing intent that maps from human input. That is harder to do with lower-level languages, but if we are optimizing for gluing together components it seems like it is very common for systems to build their own "DSLs" to solve a task. Is there a more generalizable DSL (which would no longer be domain-specific, I suppose) but still maintains the properties of such a system without generalizing to any kind of programming task?
It could increase flow for this flow paradigm though. Instead of making a request to a human that makes the component in a few days you can get the component in a few seconds. Although the person working with the graphs does need to verify/cleanup components.
Alpaca [1], perhaps. It's based on facebook's model (LLaMA) and its been trained on a conversational style, same as chat gpt. I don't know if it can produce code, though.
From what I can tell setting things up to train the model is easy. Then you have to generate the training data (can be done using GPT3's API), fine tune Alpaca, and then evaluate it.
Haven't done it myself but I believe you can find more info here [1]
Your comment made me think about an IDE built around AI code gen. Basically would have tools to generate and debug, test, and overall validate the code. I guess I’m not sure what features that would constitute - maybe it wouldn’t even need a nee IDE. Just thinking about a playground designed for codegen from the ground up.
One approach I've been thinking about is focusing on enhancing the Behaviour-Driven Development (BDD) workflows. I was was really into cucumber and its tooling ecosystem a decade or so ago. One of the challenges is maintaining these "user story" like specifications and then creating boilerplate mapping/translation between the concepts in the specification and unit tests and sometimes also the implementation. Keeping that updated as a project grows in complexity (and bringing on team members that might be less dedicated to BDD purity and related practices like rigorous refactoring) becomes a burden. But if you used an ai-code-gen augment to help maintain the environment, and autogenerate both the code and tests, really all you would be doing is clarifying mistakes in intent, or insufficient specificity. So the question would be how do you build tooling to optimize for highlighting and reviewing the changes to the system (diffs - but maybe semantically enriched in some way? Maybe start with smarter comments?) Maybe generating synthetically workloads and reviewing how that changes the system (sorta like fuzz-testing but with a smart-agent that understands how the system works without you needing to specify all the details). Importantly you could also work backgrounds if you change the code directly, you could have it update the spec, and then review that (does my change do what I think it should do, at least a higher-order domain level?)
- Human describes the problem specs in vague English
- LLMs generate some solution based on the texts it has seen (LLMs may generate non-factual information here but assume the output is factual)
- Human refines the prompt to get closer to "the thing he has in mind"
- LLMs still tries _to the best of its effort_ generate a "close" solution
- and we loop here
Assume the specs of the problem is fixed. Why LLMs couldn't generate the output the Human "had in mind"?
Because English description of the problem was not deterministic?
What can we do? Invent a language for automation specification. Then LLMs can accurately churn up a solution in one shot.
But..that "deterministic language" is what programming languages are. So LLMs at this current state are basically a transpiler from rough-English to a deterministic-English.
But, can LLMs be thought only the specs of a programming language (like C) then generate assembly code from a valid C code? In effect, can LLMs be exact compilers not English transpilers?