> My mantra became "products not papers." Maybe that applies downstream of the LLM now.
I agree, in the sense that my intuition is fed by having as many available examples as possible. I am not even sure one could quantify the difference between intelligence levels in a way that I would be satisfied with in a paper.
For example, I find it intuitive that a mixture of 10 narrowly fine-tuned GPT-3s are better at a task than 1 broadly fine-tuned GPT-3. But I don't have a real intuition about how many GPT-3s you would have to mix to match the quality of GPT-4, or if there even is any number of GPT-3s you can mix to achieve the result of GPT-4. I think we just need to start building systems and see what happens.
I agree, in the sense that my intuition is fed by having as many available examples as possible. I am not even sure one could quantify the difference between intelligence levels in a way that I would be satisfied with in a paper.
For example, I find it intuitive that a mixture of 10 narrowly fine-tuned GPT-3s are better at a task than 1 broadly fine-tuned GPT-3. But I don't have a real intuition about how many GPT-3s you would have to mix to match the quality of GPT-4, or if there even is any number of GPT-3s you can mix to achieve the result of GPT-4. I think we just need to start building systems and see what happens.