AlpaGasus: Training A Better Alpaca with Fewer Data https://arxiv.org/abs/2307.08701
Textbooks Are All You Need II: phi-1.5 technical report https://arxiv.org/abs/2309.05463
Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models https://arxiv.org/abs/2307.14430
I can very well believe that empirical this can work. (I haven't checked the literature.) My point was merely that given my priors, this isn't intuitive.
AlpaGasus: Training A Better Alpaca with Fewer Data https://arxiv.org/abs/2307.08701
Textbooks Are All You Need II: phi-1.5 technical report https://arxiv.org/abs/2309.05463
Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models https://arxiv.org/abs/2307.14430