The first benchmark is pretty much an empty request, so it measure each framework "overhead". That shows Rails+Puma at `0.25ms` overhead per request, and rage+iodine at `0.013ms`. That is significantly less yes, but it the overwhelming majority of case it absolutely don't matter, and this difference mostly come from Rails providing extra features.
As for the second one, it's entirely irrelevant. It's benchmarking a purely IO workload with Puma caped at 5 threads vs Iodine without a concurrency cap. You can increase "Rails" throughput about as much as you want in that benchmark by raising the number of Puma threads or switching to Falcon.
But also, not capping concurrency like this is risky. The second some CPU heavy work is mixed with this pure IO work, latency will suffer terribly. It's fine for a micro-service where you know it's really all IO though.
Disclaimer: I'm a member of Rails, but I welcome alternative frameworks, I'm just very annoyed at how terrible most benchmarks are.
Also this doesn't show how database access is handled which is the hard part. If you are not touching the database, you can run Rails on falcon and get fiber based concurrency.
If you run falcon on rails and access database, then you have to explicitly checkin/checkout a connection to be safe. Details here - https://github.com/rails/rails/issues/42271.
> The RealWorld Demo is a large community-backed open source project that showcases all web frameworks client or server-side through a building an example app that is much more substantial than the typical TodoMVC toy demos or the stripped out scenarios you find in benchmarks. As Conduit, a Medium clone of sorts, your application has to handle real issues like authentication, routing, and async data loading. It has a standardized specification [...]
It all depends on what you are trying to demonstrate. The current benchmarks would be fine if properly contextualized and presented a bit differently.
e.g. for the first one, instead of representing it in request/second, do a breakdown of average/p50/p90/p99 latency over X requests. That's both much more interesting data, and much less misleading.
Even the second one could be fine if used to demonstrate why an event loop can be the appropriate thing to use for IO heavy workloads.
But here they are just included with no explanations, implying a blanket "we're 25 times faster than Rails".
That is precisely my point. One has concurrency capped at 5, the other has no cap (because it uses an event loop).
The Puma process is likely 99% idle in that benchmark, you could crank up the thread count to 50 and get (not quite) 10x better results.
An event loop is definitely more efficient than a thread pool for such a work-load, I'm not denying that, but in this specific instance Rails+Puma is configured (granted it's just the defaults) with ball and chain.
I'm not saying the benchmark is dishonest, I'm saying it makes no sense.
That objection makes no sense. The fact that Puma is idle most of the time reflects a disadvantage. The event loop makes much better usage of a single thread, and that shows in the benchmark.
It's like saying comparing Node to Rhino, or Vertx vs Spring Boot is unfair. Better performance per thread / core is the whole point.
You will not achieve the same level of performance with threads except in low-volume benchmarks, and _especially_ in any kind of cloud environment where you usually only have a single core and a shared kernel.
My point is that event loop are not inherently superior to thread pools.
When you have a workload that is CPU heavy enough that you won't have more than a handful of threads of concurrency, using a thread pool will allow preemption, hence give you a better tail latency, etc.
It's all tradeoff, event loops are good for some work loads, threads for others.
So to make an analogy of my own, it's like making a race between an F1 car and a semi-truck. They are both useful and efficient motor vehicule, but they're not optimized for the same thing.
That's true. Would you then agree that an event loop is very well suited for web server workloads, with short lived requests + reasonably light CPU usage + lots of IO, which happens to be the territory both Rage and Puma are situated?
You said the benchmark is "entirely irrelevant" because it doesn't take into account a scenario that is explicitly not the goal of either project. For those workloads Ruby is already far down the list of best choices.
> Would you then agree that an event loop is very well suited for web server workloads, with short lived requests + reasonably light CPU usage + lots of IO, which happens to be the territory both Rage and Puma are situated?
No, that's where I disagree. The average Ruby web application isn't as IO heavy as people claim, and often contains big chunks of CPU work (typically HTML templating or JSON generation/parsing) that run havoc in an event loop.
My measurements on dozens of Rails app showed the IO ratio tend to be in the 30-70% range. Most closer to 50%. The apps that do more IO than that either have terribly optimized DB queries, or are mostly acting as some sort of HTTP client for an API, but from my experience that's the exception, not the rule.
> For those workloads Ruby is already far down the list of best choices.
Raw speed is far from the only criteria when one chose a stack...
Could you please stop creating accounts for every few comments you post? We ban accounts that do that. This is in the site guidelines: https://qht.co/newsguidelines.html.
You needn't use your real name, of course, but for HN to be a community, users need some identity for other users to relate to. Otherwise we may as well have no usernames and no community, and that would be a different kind of forum. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
There is certainly a lot of speculation in Techempower benchmarks and top entries can utilize questionable techniques like simply writing a byte array literal to output stream instead of constructing a response, or (in the past) DB query coalescing to work around inherent limitations of the DB in case of Fortunes or DB quries.
And yet, the fastest Ruby entry is at 274th place while Rails is at 427th.
Except, increasing the threads increases the memory usage proportionally. From what I have seen, the memory won't be released back and it keeps increasing until the process gets OOM'ed. At least this was the case with Ruby 3.0 and 3.2.
My whole thing is that ruby and performance don't mix, choose something else.
I'm not saying don't bother optimizing something written in Ruby, I'm saying Ruby has inherent limitations in terms of performance. If performance is your goal you're using the wrong language.
> API-only - the only technology we should be using to create web UI is JavaScript. Using native technologies is always the most flexible, scalable, and simple solution in the long run. Check out Vite if you don't know where to start.
Thanks. I'll pass on that one. Nevertheless, both rage and iodine seem like pretty cool projects FWIW.
>> API-only - the only technology we should be using to create web UI is JavaScript.
> Thanks. I'll pass on that one.
I don't know, there's definitely some merit in separating your front end from your back end - at least that's the impression I got after struggling on JSF/PrimeFaces Java projects for a few years, because at least with a well defined API and no magic behind the scenes everything becomes more observable and easier to reason about, with fewer bugs. Some might prefer TypeScript or something else to JavaScript, but I think the point still stands.
In addition, this lets migrate or change everything piece by piece, as long as the interface isn't broken:
- different frameworks (let's say from Spring to Spring Boot, or from the old ASP.NET to the new variety that runs on Linux, or Rails to Rage or vice versa)
- different languages (if someone decided to take their PHP Laravel solution and migrate to Ruby with Ruby on Rails, or Python, or anything else)
- deprecated technologies can also be handled gradually (e.g. supporting an EOL AngularJS solution while developing a new front end in Vue/React/Angular/whatever)
As for the reason for those migrations, sometimes code just rots due to neglect, other times you need features that aren't available in your solution of choice (for example, pre-made UI components, like PrimeVue, PrimeReact and PrimeNG have; or maybe needing to have responsive maps in the app). In addition, you can also deploy your front end and back end separately, have different rules for the reverse proxy in front of them, suddenly have a very clear view of where your static assets are, might have all of the benefits of a SPA and PWA (if needed), anyone can integrate with your API if needed/allowed because you've hopefully built it to actually be usable and so on. Plus, your front end can talk to multiple different APIs, so if you have multiple teams working on those, suddenly that is a solved issue for you as well.
That said, it's also a lot of work because SPA and adjacent approaches bring a lot of complexity into the mix and something like Vue/React/Angular and its ecosystem will bring its own pain points and things you have to learn. If all you need is a bunch of forms, then pretty much anything will do fine and SSR is a good choice. If you need an "app" on the web, then HTML/JS/CSS isn't a very good choice to begin with (historically not built for that; though I don't think there are actually good choices, same as with desktop software), but those SPA solutions will see you pretty far.
Use HTMX and you'll be surprised how much you don't need JS for (or to be more accurate, very little JS).
Rails especially is very good for this with partials. Use ERBs (or Haml, Slim, etc) and it's an absolute bliss with the added benefit of easy caching.
Stop duplicating state. Stop duplicating logic. Maybe in the future when your app needs 5 different clients, it makes sense to only have an API. But let's not pretend that should be the default!
I still cry every time I see a new application using GraphQL to have literally one client which is simply serving HTML with a bloated JS framework. The whole premise of it was to be flexible for multiple frontends. Come on folks, YAGNI.
I used HTMX since the intercooler days [0] but the stuff you can make is rather limited. Also you still need the JS to deal with a11y things like expanded state (or hyperscript, apparently).
If you have a lot of components to implement, everything requires thinking. Example: You want to add a repeat field to a form. Easy, get the input from an endpoint, and append. Done? No, how do you remove a field? Add a line of JS, simple! Drag and drop if the order needs to change? Let's add a dependency! [1]
I really love it for simple applications though. Resist implementing a complicated menu, live notifications, an editable data-table and such non-web-native things and you can create the fastest CRUD app ever.
And you will need another client (don't YAGNI me on this one, burned too many times by people not caring about endpoint design), but that's not really an issue if your view model does not contain non-public data (it shouldn't), as you can convert it to JSON at the same endpoint (respect the accepts header) and call it an API.
Rails with turbo and stimulus is amazing. Very similar to htmx and phoenix liveview. Stimulus lets you make a really minimal web framework for ui elements that are mostly defined as ruby components, and turbo lets you to use erb without whole page reloads. Whole responsive web apps with a dozen lines of js, its great.
Yeah you can define viewcomponents in ruby. Have them rendered based on stimulus code. Turbo lets you do updates without full page reloads, viewcomponents make much more reusable and flexible ruby based components than pure erb, and stimulus handles front end state. I rewrote a medium size react + graphql page with this and thr code base was literally 1/10th the size
> I don't know, there's definitely some merit in separating your front end from your back end
This is 100% true and the biggest loss in web development in my opinion.
SPAs actually end up making this worse in the long run though. The major SPA frameworks are all moving to an RPC model of all things, entirely coupling your frontend and back end together. The original SPA approach did offer the ability to separate frontend and backend, but the frontend scaled poorly and brought in too many complex considerations like routing, caching, error handling and recovery, etc. The new approach is basically abandoning the benefits of an isolated back end in favor of tight coupling simply because the existing frameworks are too far down the wrong path.
If your goal is ever to isolate the frontend and backend you'll want to consider three environments - backend, web server, and frontend. Ironically that's exactly where we were when SPAs broke out onto the scene
> The major SPA frameworks are all moving to an RPC model of all things, entirely coupling your frontend and back end together.
I'm not familiar with this development. Do you mean that for example React has grown a backend running on a server, querying the database, etc?
I know that backend frameworks like Rails and basically all the major ones added a JS layer running inside the browser and obeying to the backend. It listens to messages from the backend with changes to the UI elements and that saves full page reloads.
I'm thinking mainly about about SPA frameworks (metaframeworks?) like NextJS, SvelteKit, Remix, and I think NuxtJS as well though I'm less familiar.
They're all adding features where code is marshalled back to the server either through useServer() hooks, pragmas, or magic $ signs and imports.
They all create RPC calls as part of the build and bundle step. Depending on the specific implementation, that usually means taking code that otherwise looks like client code, moving it to the server bundle with a unique API signature, and swapping the RPC API call into the client code.
Rails has been a full stack framework since the start, with the JS component changing frequently over the decades, but with template helpers that generate JS being a fundamental component since the beginning.
> the only technology we should be using to create web UI is JavaScript
Quite an interesting idea, but stating that JS is the true way of creating Web UIs misses the mark by miles. You would be surprised by how far you can get with HTML and CSS alone. You will ofc need JS for more dynamic interactions, but ditching the server entirely and delegating everything to JS will just take us to the SPA mess that most of have been burned with.
Especially now that rails is shipping with turbo, hotwire and stimulus. Using no (additional) javascript you can make awesome performant reactive webpages.
Most recent app I worked on used Mantine for a base component library. Having such a large collection of drop-in components made the app come together very quickly. Performance matters but a well built react front-end for a solid MVP is performant enough.
I haven’t seen a vision like this for the stimulus/htmx world. How do you import components?
My team is usually view components in our application. Basicslly ruby erb renders the view component in a turbo frame. Turbo allows async updates without redrawing. Then we use stimulus to call small js component that are put into the view components.
I prefer not to use JavaScript unless forced to do so. It's burdensome and almost always unnecessarily complicates the stack. With the rise of tools such as Turbo[1] and HTMX[2] the need for JS frameworks like React and Angular are greatly reduced, to the point that they are (almost?) unnecessary.
This was unexpected but welcome, finally something new and fresh to compete with what we currently have. Since it's Rails-compitable? Does that mean I can connect Avo, Bullettrain, Rack-attack aswell?
Can't wait for this to popup on techempower benchmarks!
But I would've hoped that there was some proper benchmarks done instead of these unrealistic ones.
Is facil.io maintained? Last commit was 2 years ago, which is a bit concerning for something written in C that handles a bunch of security-sensitive things including a from-scratch JSON parser.
Great idea, wish it would be merged with Rails itself, but without the opinionated lines, like "the only technology we should be using to create web UI is JavaScript"
It's a really stupid line and whoever wrote it is embarrassing themselves.
Also, a more real/demanding benchmark would be great.
They're right and they should say it. You would think I was insane if I made a CLI tool worked by fetching and running dynamically generated pre-compiled binaries from a server for each subcommand command and yet that's what we do with erb template style apps.
The API + static versioned self-contained html/css/js asset bundle is the way for anything you would reach for erb for.
This is great. I had been long looking for something like this in Ruby eco-system. I worked on a Rails app which interacted with a bunch of micro-services and I missed the async-await so much. The problem was, when the app was waiting on API requests, the throughput suffered a lot.
We couldn't increase the number of threads a lot, since it increased the memory usage a lot which eventually led to OOMs.
I tried to put something together using async-http [0], but it was not very stable.
One question would be, how does scheduling work with existing psql and myself db gems. Should the calling code wrap the database calls with `Fiber.await { Fiber.schedule { ... } }`?
A framework like this would be very useful when the tech-world is moving towards micro-services, for the good or bad.
[0](https://github.com/socketry/async-http)
I mean 180+ commits, 3 month old, bold claims and already 8 releases on rubygems?
There ought to be a vetting/voting process for packages, before they are accepted into a public repo and pollute the global namespace. Like in arch repo/AUR.
This might be useful but the moment someone asks you to build an admin UI for your API only service you end up wondering why you didn't go with Rails in the first place. And the idea that frontends should be JS only is preposterous.
These kinds of projects are good for the ecosystem. Sometimes the underlying technology (such as using fibers here) could end up being used or merged into Rails itself, after successfully demonstrating the benefits.
Because unless you want to expose your Ruby server directly to the internet without a load balancer, there isn't much to gain in having HTTP/2 between the LB and the app server.
I'm curious though, what's your use case to and HTTP2 up to the app server?
Seems like a cool project. I really like rails and for projects where you only want to use it as an API this seems great. Feels like DHH has been so obsessed with killing JavaScript recently that rails has lost its way a bit.
I mean that’s fair enough, it’s his framework he can do what he wants. But the last two places I’ve used rails it was being used as backend only. I suspect that’s a very standard way to use rails these days and no amount of DHH telling me it’s wrong is going to change that.
I find Rails API-mode hard to work with, integrating the frontend towards Rails crud-like api is cumbersome because of all the endpoints you have to manully work with.
While using Rails own templating engine creates all the links and urls for you!
I'm not sure it's fair to say DHH wants to kill JavaScript, only that he wants to use JavaScript only where he thinks there's no better alternative. i.e. progressive enhancement.
There's still nothing stopping you from jettisoning view support and just using Rails as an API if you want to.
I would say so, I am subscribed to a mailing list which sums of what's happening within the Ruby and Rails community every week, and there are constantly new startups or projects going on.
An interesting point is that Twitter started in rails and migrated to something (java?) that was faster after they got users. Ideally it would not be necessary to fully replace the application after launch but getting it out the door quickly to start iterating has its upsides.
The biggest problem Twitter had was using MySQL for data storage.They moved to Scala, and while it was not quite a disaster, it did slow progress and also had to be largely simplified. Meanwhile places like GitHub and Shopify proved you could keep on scaling with Rails.
Exactly- using Twitter's problems on Rails as evidence it doesn't scale is only slightly more wrong than blaming on them on MySQL. Twitter started moving to Cassandra in 2010 to deal with scaling issues they were having with MySQL sharding (gizzard), but continued to release patches to MySQL to address their needs, then eventually created their own data store, Manhattan, optimized for their particular usage patterns. None of this means MySQL (or Rails) can't scale.
MySQL was the biggest problem they had though! They brought in Lucene (Earlybird) that massively improved performance- which makes a lot of sense.(in addition to adding Cassandra and eventually manhattan).
Then they started building a lot of things that weren't web applications, like RPC servers (where they created the finagle library) they ended up with a team where Scala and the JVM was more their area of expertise and interest and solution to every issue.
No one thinks you use MySQL to build every kind of data application, no one thinks you use Rails to build every kind of server.
Last two orgs (and they're large orgs) I've been in have phased it out in favour of node or python on the basis that nobody wants to hire for it. Short-sighted in my view, but that's not a battle I can win.
Crystal had a bit of a slow start. It was promising, but the deployment issues persisted longer than they should have, and by the time they were fixed, you had other toolchains that filled the same niche. Elixir did "ruby" better than crystal OR ruby did.
I really wanted to like Crystal, and used it for a few projects. But the immaturity and timing was it's kryptonite.
This thread's context is large rails userbase around 2008'ish (people were changing computers from pcs to macs with textmate to do rails, perception of devs changed from nerds to cool kids - the whole thing was quite huge) that dissolved substantially. Crystal's syntax in this context feels like huge wasted/missed opportunity.
The first benchmark is pretty much an empty request, so it measure each framework "overhead". That shows Rails+Puma at `0.25ms` overhead per request, and rage+iodine at `0.013ms`. That is significantly less yes, but it the overwhelming majority of case it absolutely don't matter, and this difference mostly come from Rails providing extra features.
As for the second one, it's entirely irrelevant. It's benchmarking a purely IO workload with Puma caped at 5 threads vs Iodine without a concurrency cap. You can increase "Rails" throughput about as much as you want in that benchmark by raising the number of Puma threads or switching to Falcon.
But also, not capping concurrency like this is risky. The second some CPU heavy work is mixed with this pure IO work, latency will suffer terribly. It's fine for a micro-service where you know it's really all IO though.
Disclaimer: I'm a member of Rails, but I welcome alternative frameworks, I'm just very annoyed at how terrible most benchmarks are.