Does anyone know why there hasn’t been more widespread adoption of OpenAI’s Harmony format? Or will it just take another model generation to see adoption?
It's a good question, opinionated* answer: it's the whackiest one by far. I'm not sure it's actually good in the long run. It's very much more intense than the other formats, and idk how to describe this, but I think it puts the model in a weird place where it has to think in this odd framework of channels, and the channel names also shade how it thinks about what it's doing.
It's less of a problem than I'm making it sound, obviously the GPTs are doing just fine. But the counterexample of not having such a complex and unique format and still having things like parallel tool calls has also played out just fine.
When I think on it, the incremental step that made the more classical formats work might have been them shifting towards the model having tokens like <parameter=oldText>...</parameter><parameter=newText>...</parameter> helped a ton, because you could shift to json-ifying stuff inside the parameters instead of having LLM do it.
Also fwiw, the lore on harmony was Microsoft pushed it on them to avoid issues with 2023 Bing and prompt injection and such. MS VP for Bing claimed this so not sure how true it is - not that he's unreliable, he's an awesome guy, just, language is loose. Maybe he meant "concept of channels" and not Harmony in toto. Pointing it out because it may be an indicator it was rushed and over-designed, which would explain it's relative complexity compared to ~anyone else.
* I hate talking about myself, but hate it less than being verbose and free-associating without some justification of relevant knowledge: quit Google in late 2022 to build a Flutter all-platform LLM client, based on llama.cpp / any 3rd party provider you can think of. Had to write Harmony parsing twice, as well as any other important local model format you can think of.
Yeah but this seems like a bolt-on and not something they train their model to understand at the token level like how they do tool calls. Maybe Anthropic has a token-level skills support (e.g. <SKILL_START>skill prompt<SKILL_END>).
Long-running tasks are an open topic of discussion, and I think MCP intends to address it in the future.
There are a few proposals floating around, but one issue is that you don't always know whether a task will be long-running, so having separate APIs for long-running tasks vs "regular" tool calls doesn't fully address the problem.
It'd be nice to more closely integrate MCP into something like Airflow, with hints as to expected completion time.
Real world LLM is going to be built on non-negligible (and increasingly complicated) deterministic tools, so might as well integrate with the 'for all possible workflows' use case.
RSC is indeed very cool. It also serves as a superior serialization format compared to JSON. For example, it can roundtrip basic types such as `Date` and `Map` with no extra effort.
One thing I would like to see more focus on in React is returning components from server functions. Right now, using server functions for data fetching is discouraged, but I think it has some compelling use cases. It is especially useful when you have components that need to fetch data dynamically, but you don't want the fetch / data tied to the URL, as it would be with a typical server component. For example, when fetching suggestions for a typeahead text input.
I discussed it with a couple of Next.js maintainers (https://github.com/vercel/next.js/discussions/75930), and they indicated that it's only a problem for "standalone" deployments (i.e. not on Vercel). However, I'm not entirely convinced that is true. I wonder if there are major optimizations that could be made to, for example, the routing system.
I have no particular views on performance but the thing i keep hearing about next of 'this is solved by using one particular hosting provider' really surprises me as something people are ok with.
I think people reasonably expect, say, an aws lambda to be aws specific.
That's a very different story to React, which is supposed to be a library for general application ui development, and the official react documentation recommending Next as the way to use it.
I've been using React on and off since 2017 or so. Back in the day it was typical to bundle your React code and integrate it with the backend framework of your choice. It was stable and cheap. In the 2020s frontend tech took a bizarre turn...
Is he wrong though? Next.js reason of existence is essentially that .json file navigation. The addition of server side compute (ie: middlewares, router, etc...) is mostly a business decision that the framework has nothing to do with as it breaks your back-end/front-end separation (unless your application is very simple to fit in a nextjs function).
> and they indicated that it's only a problem for "standalone" deployments (i.e. not on Vercel)
The fix is clearly to take out your wallet /not.
NextJs is slow. It doesn't have to be. For their swc, turborepo and everything else going on they could have made the actual production usage fast just as well.
reply