Generally interesting reflections here, yet I see the same kind of myopia and fatalism that is rampant in our (fashion) industry:
> yet I have no doubts that this looping future is going to be our future despite the fact that I presently resent it
Why would anyone concluded this? LLMs are just one kind of application of MLs to software production. There is a vast solution space for automating parts of software production. The idea that slop loops are the inevitable future because they happen to be accelerating output at the moment just seems profoundly short-sighted and lacking in vision.
Would be interested in a comparison with https://sourcehut.org/ (which has a comparable minimal aesthetic, but also has the deep benefit of being FOSS.
Oh really? A 195% cost to revenue ratio isn't bad at all? I'm not a biz expert, but I spent a few minutes looking this up (e.g., what are usual cost-to-revenue ratios for new lines of business), and this sounds like BS to me.
If "cost" is mostly capital investments, absolutely. Normally you'd use operating cost (which for capital equipment would be depreciation and interest), and here they are using the capital cost as full cost.
No one really knows how quickly AI hardware investments will become obsolete and thus how long it should be amortized, but 2-3 years would be extremely conservative, and in fact used H100 (discontinued/2 generations old) prices are higher today than they were when the equipment was new several years ago.
But if it's fully being amortized, then it means they don't buy new Nvidia GPUs anymore for a while. The situation is either "your GPU AND the datacenter infrastructure it's running on is obsolete", or "Nvidia's profits tank because people are staying with current-level infrastructure".
That would be true if everyone weren't supply constrained and buying literally everything they can find.
There are actual risks that this trend doesn't continue, but as long as the trend continues, it is pretty good for revenue. "AI shown to hit a wall/doesn't actually deliver/stops growing so fast", "massive improvement in hw efficiency or tech such that all the old stuff becomes obsolete", "bottleneck on power/regulations/etc such that no one wants anything but the most efficient cutting edge stuff" would be the ways it could end and then all these factors reverse. Right now, power is so constrained that old, inefficient power generation is actively being turned back on or set up at new sites (e.g. old aviation turbines which are very inefficient compared to combined cycle).
> It is with great joy that I have accepted the invitation to write you a personal letter. Even in your scientific writings your presence as a person is very strong, which is unusual for a mathematician. The traces of your personal life and your dramas extend beyond the limits of your own personal circumstances to concern us all ...
>
> ...
>
> Perhaps the greatest catastrophe of anti-scientific computationalism can be seen in the recent theory of "The End of Theories". In a series of widely quoted articles, informaticians or managers of very large databases explain that: "Correlation supersedes causation, and science can advance even without coherent models, unified theories". In short, networked computers, bringing to light very extended correlations in huge databases, make it possible to predict and act, without the need to "understand": scientific intelligibility is an uncertain luxury that is subjective and outdated, and theories are fallible proposals. Data, especially in large quantities – tera-terabytes, Big Data – is objective, is a new form of the absolute, is individually exact, expressed in digits. Thus, they argue, the larger that databases become, the more that statistical regularities, brought to light by computers, can govern us without the need to understand the meaning of the correlations, to interpret them, and without the need for theories about them, for interpretations.
>
> Fortunately, mathematics allows us to demonstrate the absurdity of these claims: Cristian Calude and I have written an article about this. Precisely the immensity of data involved has allowed us to apply the theorems of Ramsey and Van der Waerden. These make it possible to show that, given any "regularity", or any correlation between sets of numbers, you can find a number p large enough, such that every set with at least p elements contains a regularity (or a correlation between numbers) with the same structure. Now, since this applies to every sufficiently large set (with at least p elements), this also applies when it is generated … by a random process. Indeed, we observe, almost all sets of fairly large numbers are algorithmically random (one can give a mathematical definition of them, in terms of incompressibility), i.e., the percentage of non-random tends to 0 as p goes to infinity. So, if you observe regularities in increasingly large databases, it is increasingly likely that the inserted data are due to chance, in other words are perfectly without meaning and do not allow prediction nor action.
reply