Referring to "25 centimeters per year" as "so fast it can be seen from space" is so funny to me.
We have very good satellites! Lots of things can be seen from space. "can be seen from space" is an awful way to provide readers intuition for the scale of something.
In which case doing this in the dark is clearly bad for the community -- if that location is what's scarce then they should be demanding a better deal.
The only way this makes sense for communities is as a kind of "finder's fee", i.e. you might argue that if BigTechOne™ knew that they'd have to bid against BigTechTwo™ they'd never even bother to scope out the location.
Still, if the prospecting is the bottleneck there could be 3rd parties (or even the tech companies themselves) entering into agreements with towns which allow both a finders fee and open bidding for the lot.
The described pattern is standard in Meta. This, along with the infrastructure and tooling to support it, was the single largest "devx quality of life improvement" in my experience moving to big tech.
Microservices should have clear owners reflected in the org chart, but the topology of dependencies should definitely not be isomorphic to your org chart.
TL;DR: assuming you've squashed all regular non-determinism (itself a tall ask), you either need to ensure you always batch requests deterministically, or ensure all kernels are "batch invariant" (which is absolutely not common practice to do).
It says docker compose is at the "wrong-level of abstraction", but I kept feeling the author was instead expecting docker compose to solve different problems that it was ever meant to solve.
In fact, they seem to be expecting a highly-opinionated, high-level interface which solves problems that I don't think anyone using docker compose in prod should even be worried about.
A lot of concerns seem to be around avoiding spinning up duplicate instances of reverse proxies, databases, and caches. First of all, why is this a concern? Idle threads have basically no impact on a system, so this generally isn't a concern. This is a nuanced and context-dependent issue, but generally it won't even make the list of top-100 performance bottlenecks for most applications. Yet the article takes for granted that the benefits of solving this problem outweigh the many cons of coupling them together.
Even if you wanted to enforce your applications sharing a postgres instance under the hood, why would you want that to be black-magic performed by the container orchestrator?
Other stuff like DB backups just don't seem like issues docker compose users have. If you need to orchestrate across multiple nodes in order to meet your SLOs, then don't use docker compose.
Finally, it seems like the actual solution is significantly under-discussed. I both have tons of questions about how it's supposed to work, and I see lots of shortcomings with the parts that I do understand.
Beyond the specific issues I see, the fundamental attitude seems to be "force everyone to architect their applications in a very specific way, and don't even try to support any use cases which fall outside of it". You need a damn good reason to be that opinionated about these sorts of things, and it by definition will only work well in specific contexts. I'd be interested to read an article which tried to articulate why such an opinionated API would improve SDLC-considerations over docker-compose, but I don't think that's the article I just read.
These are great points, and probably worth their own blog post to answer.
> First of all, why is this a concern? Idle threads have basically no impact on a system, so this generally isn't a concern
Idle threads have very low impact on CPU utilization, probably, if the application is well-behaved (and I expect most databases and caching layers to be well-behaved in this way). The application itself, however, will need memory and the way containers are built prevents the usual de-deplication of system libraries.
> but generally it won't even make the list of top-100 performance bottlenecks for most applications
True, but it makes the short list of "how much stuff can I run on a $100 computer", and it's one of the relatively few concerns an application operator has when they are not the application developer.
> Even if you wanted to enforce your applications sharing a postgres instance under the hood, why would you want that to be black-magic performed by the container orchestrator?
To make self-hosting much simpler. If the container orchestrator doesn't do it, what do you think should do it?
> Other stuff like DB backups just don't seem like issues docker compose users have. If you need to orchestrate across multiple nodes in order to meet your SLOs, then don't use docker compose.
The DB backups are meant for disaster recovery rather than supporting multiple nodes. I guess that's multiple nodes through time... But, yeah, I agree, docker-compose is not a good fit.
> Finally, it seems like the actual solution is significantly under-discussed. I both have tons of questions about how it's supposed to work, and I see lots of shortcomings with the parts that I do understand.
Yeah, agreed, I'll be writing other things to discuss what I think the correct solution should be. I'm curious to find out if other people have existing solutions to the problems I outlined. If it's a solved problem and I just don't know about it, that'd be better.
> I'd be interested to read an article which tried to articulate why such an opinionated API would improve SDLC-considerations over docker-compose, but I don't think that's the article I just read.
I actually like that analogy. It's somewhere in between. Enough that LLMs can help in many ways, but the current models are still far away from doing everything.
We have very good satellites! Lots of things can be seen from space. "can be seen from space" is an awful way to provide readers intuition for the scale of something.