I would not mind if it was +50% more expensive... if it was TRULY a competition to f.ex. a Macbook Air. Many more techies would not mind it. But I don't think we ware there yet.
Yea I've posted there twice as far as I remember. You will absolutely get help, whether you understand the answers is a whole different story.
Elixirs community is great. Its just hard to learn because it's not yet widely adopted, there are no (non senior) roles for it and it's a lot of work understanding all the BEAM concepts. A thing just being interesting isn't enough motivation for me to learn, I need a bigger goal but with Elixir there do not seem to be any.
My last experience with it was building something with Phoenix Liveview until I noticed how easily you can hijack the websocket and just spam random commands to your server or temper with payloads (with regular webapps ive built i never had this issue). Which made me quit that project.
Fair. If you have this friction then it's not worth pursuing.
One thing that really helped me pick it up was saying YOLO and rewriting one part of the business stack from Ruby on Rails to Elixir. It taught me quickly and well.
The official guides are also great and IMO you can get through them all without a rush in two weekends. But again, if you don't want to then don't.
You can also try asking right here in this HN thread. Maybe I or others would be willing to give you a more detailed response.
When building I couldn't get "what if I have ghost processes", "what if I spawn too many processes", "what if this architecture is bad compared to...", "when to kill processes", "whats the correct restart strategy for this" out of my head... It's so confusing to build for the BEAM that I ultimately gave up on it.
> It's so confusing to build for the BEAM that I ultimately gave up on it.
Every new paradigm is confusing if you don't put in the work to learn it. That's just how the mind works.
What's important is what you get after you don't give up on it long enough. And that, on BEAM, is a hilariously OP superpower of effortlessly[1] parallelizing and distributing workflows. Then there are Elixir macros and the OTP supervision model. The addition of gradual typing is huge, and when the annotation syntax lands, I will definitely switch to Elixir for everything on the backend.
In any case, the only thing I can tell you is that learning Elixir is worth enduring the confusion. From personal experience, it's just a matter of learning it bit by bit over time - there's a finite set of "confusing" ideas in the OTP/Elixir/BEAM mix, and learning about some of them every other day works wonders over a few months.
[1] An exaggeration - I know! But it does make it much easier to implement parallel and distributed workflows. Recently, most of the important languages finally started getting their m-n concurrency models (from Java to Python), so the BEAM is not as much ahead on SMP, but for distribution (you can send closures to execute on different machines transparently!) it is still in a league of its own.
I'm not sure what a ghost process is? I guess something that's living beyond its usefulness / isn't supervised, etc? ... I don't speak Elixir, but you can do the equivalent of this Erlang to see everything on the node:
rp([{X, erlang:process_info(X)} || X <- erlang:processes()]).
Then you'll know what's going on. Caveat: if you have a lot of processes, that's going to use a bunch of memory; for production you probably don't want to use erlang:process_info/2 with specific items instead of the default items. And you might don't want to output something for all the processes if you have a lot of "normal" processes that won't need to be listed.
> "what if I spawn too many processes",
The default limit is 1,048,576, if you want to have more, you can add +P X to the erl command line with a bigger limit? Have your monitoring alert you when you're at ~ 80% of the limit.
> "what if this architecture is bad compared to...",
This probably addresses the real question of your too many process question. If your architecture is bad or if you spawn more processes than a good architecture would, your performance will be bad. If your architecture is really bad, you'll have a hard time solving the problems you're trying to solve. Future you will look upon your system and despair; you may also despair in the present...
Eh, you're going to make bad architecture. BEAM won't solve all your problems. But, if you've got problems it can solve, IMHO, it can be a very nice way to solve them.
> "when to kill processes",
Kill processes (or let them crash) when they misbehave. Kill them (or let them exit normally) when they've done their work and they don't have anything else to do or wait for. When you spawn a process, you'll often have a pretty good idea of the conditions that would lead to its death... Ex: if you spawn a process to handle a connection, it should probably die around the time that the connection ends. If you spawn a process to handle a request, it should probably die when the request is handled. If you spawn a process to listen for connections, it probably should die when you don't want to listen anymore. Etc.
> "whats the correct restart strategy for this"
Well... it depends. Almost never the default strategy. The default strategy is a big foot gun; at least it is for Erlang, maybe they changed it in Elixir. I need zero hands to count the number of times I actually wanted BEAM to stop because some supervised process failed 3 times in a small time frame; but it's happened to me a lot more times than that. For per connection or per request things, the appropriate strategy is not to restart at all; for other things, try to restart a few times quickly then maybe every minute or so is probably sufficient. You'll want some sort of alerting. And if the restart strategy isn't right, you can always console in and poke it.
I haven't dug into this for a while, bit you should be able to define a catch-all event to return a respond to non-compliant requests . It should be built-in to some degree IMO, but I think it's not an unsolved problem.
This will not work if a attacker guesses a function signature correctly as the catch all block usually is at the bottom of the module. If you use atoms in the function signature, attackers can just guess them, even if you never intended that function to be reachable from frontend code.
That being said, I am not forced to use liveview, its just that most ressources nowadays use it.
With respect, this topic in particular has been beaten to death.
I too liked Clojure when I tried it some years ago (agreed on the composition and data structures; both are _great_). But the real value-add is in the runtime, not the syntax. Java has a solid runtime but it's not yet as good as Erlang's, maybe even not up to the standards of Golang -- I am talking concurrency / parallelism here (for memory management I have no doubts Java is very good). And I know: green threads and stuff. Well, call me when you can do what Erlang / Golang can do. Then I'll look again, very seriously too.
Programming language syntax scarcely matters. It does to some extent but we the programmers tend to over-romanticize it. The runtime and its properties are the much better thing to optimize for.
Clojure is about its rigorous and pragmatic "immutability first" paradigm that you simply don't get from other PLs.
LISP is much more than just a runtime syntax, such as its distinct evaluation model and metalinguistic core.
The JVM was chosen for Clojure because of its reach and vast ecosystem. People have ported Clojure to other runtimes, even Beam (Clojerl), where it enjoys decent success, too.
Clojure on JVM with virtual-threads (green threads) and communicating only via core.async channels (CSP inspired) using immutable data structures is pretty neat, FWIW.
Erlang is weird in this regard. It has very strong guarantees when it comes to per-process heap that make GC much simpler: no escape hatches for mutability when most immutable languages do include them! But on the other hand inter-process communication is a form of mutability (using another process as a global mutable variable is trivial) and ETS etc. present a mutable interface.
Clojure brings more than syntax though... there's an opinionated take on making all data structures immutable (as in, structural sharing [1]) by default. That's a huge difference in how you architect the program and debug it.
I do love immutability. If a language does not have it I am very weary of using it. I only made exception of Rust because of how good is it for so many things + you can design with immutability first and only use mutability when you truly have no other choice and/or just want more performance and are willing to shoulder the extra effort of verification (potentially fuzzy testing even).
When it comes to concurrency, what can golang's runtime do that is so special? When I tried it, it seemed like a worse version of Erlang's for people that prefer C style syntax. Depending upon your design space pervasive immutability is a huge boon too and golang doesn't have that but Clojure does - Erlang obviously having that and more.
I agree Golang is a worse version of OTP, no question about it, but if you are not allowed to code in Erlang/Elixir/Gleam (which sadly is 99.9% of the projects on the planet) then Golang is the next best thing.
It has footguns, sure, but with library support and discipline it can get you very far.
To me it's embarrassing that PLs still tout syntax and various other goodies, completely glossing over runtime. I might be missing something. But faux humble statements aside, I feel many others are the ones who miss something -- and that's the fact that doing stuff in parallel is a fact of life for 20+ years now and it's time all popular PL runtimes finally wake up to that fact.
If not, I am simply not considering them. And I am not saying that arrogantly though it sounds that way; there are some PLs that I _really_ liked and was almost heart-broken that I had to abandon them and not work professionally with them. But I have enough experience to know that runtime choice matters, a lot.
For the record, Racket was one of those PLs I abandoned. I know they started working on parallelism some years ago but I had to make a decision next week back then so, Elixir + Golang + Rust it is for me.
Have you tried Lisp Flavored Erlang [0]? I never got around to trying it out. I used Elixir for a couple of years, building web backends, and I truly loved the experience. I remember wanting to try out LFE but never got around to it before moving on to a different employer/stack.
I have and I did kind of like it but ultimately admitted to myself that I no longer want to use too niche or too new PLs. Elixir has a fairly solid ecosystem at this point and I am only going to switch to something even bigger (I already use Goland and Rust as well).
Love the idea of LFE but it needs a bigger ecosystem.
> the real value-add is in the runtime, not the syntax. Java has a solid runtime but it's not yet as good as Erlang's, maybe even not up to the standards of Golang
won't lie, this is hilarious. you got me from nodding along to being the spitting out food meme guy in a span of couple seconds.
JVM runtime is undeniably the most well researched and optimized runtime in history of runtimes, specifically in realm of concurrency and parallelism, it literally carries like half the world on it's back.
not to throw any shade on erlang vm - i've been a fan for well more than a decade, but other than making some interesting, but limited in practice, tradeoffs with regard to concurrency architecture, it doesn't really offer much more.
go's runtime is just a different beast altogether designed with different goals in mind and with no baggage of backward compatibility with legacy.
one particular detail i'm very grateful to Clojure for, is exactly the ability to use JVM runtime without having to touch any Java.
> Programming language syntax scarcely matters
on the contrary, it matters quite a lot.
you might be drinking some of that AI koolaid, conflating our suddenly hypertrophied abilities to produce code regardless of our familiarity with the syntax or the APIs with ability to produce and deliver good quality products, but this delusion is getting reality check as we speak.
a realization is propagating through the industry that being able to produce more code than you're able to review, comprehend and internalize is actually not a great thing.
and that's where syntax matters - it has to be high signal/noise, it has to expose you to right abstractions and it has to be pliable to allow the codebase reflect the problem in a way that minimizes cognitive load both during production and during consumption.
LLMs are language models and syntax is a crucial part of any language.
LLM bashing aside (although I tend to agree), I agree with midnight_eclair. The claim that Erlang or Go are outright superior to the JVM doesn't really stand up. They're better at some things, and worse than others.
Regarding language syntax, it definitely matters. In the same way the vocabulary we use shapes our thoughts, the expression of a programming language shapes the implementation. Of course, as Clojurists know all too well, it's entirely possible to write Java in any language!
I don't think you, I, him and others necessarily disagree at all here, it's just that living language has defects and I can't spend 30 minutes clarifying beforehand like I am doing a math proof.
To me the strengths of Erlang BEAM VM and Golang's runtime nullify their weaknesses (and of course they do have weaknesses, some are pretty hard to swallow too). To me they sit on the positive side of "right tool for the job".
I just can't work with global mutability anymore. It's an endless hopeless pit of determinism bugs. I picked my battles. Respect to whoever wants to make a career out of chasing them but that's no longer me. I want to make measurable business progress when I work and not babysit defects that should have stopped existing two decades ago.
I can agree with other posters that the JVM has come a long way. I might reassess if I get the time. And I am not bashing on anything here. I am saying what my experience showed me. To me it's tiring to pretend that all languages and runtimes are equal and I'll keep claiming they are not.
As mentioned above: I don't think we necessarily disagree at all.
Well, you are kind of using my comment to vent your frustrations about AI while it has barely anything to do with it -- but you tried to link the two, unsuccessfully. Which is not fair as you have no clue of my stance on AI and are extrapolating a bit too much.
Syntax does not matter simply because it's an extremely leaky abstraction of the runtime below, is my point.
Of course syntax must be high signal/noise ratio, I believe every reasonable programmer will agree. But many are making entire careers in PLs where that's not the case. Hence, in practice it does not seem to matter much, for the better or the worse.
RE: runtime, try and pay attention to the parameters given in my comments. I specifically acknowledged that the JVM is a great and mature runtime but it's lagging behind on STM / actor capabilities. Tearing down a straw man is not impressive and it comes across as you trying to gain visibility by deliberately misrepresenting your discussion opponent's arguments.
> you have no clue of my stance on AI and are extrapolating a bit too much
apologies, but maybe next time try to elaborate more on sweeping statements like "syntax doesn't matter", because in current context my assumption for why you would say that is not all that outrageous.
> Syntax does not matter simply because it's an extremely leaky abstraction of the runtime below, is my point.
that would be the reason why syntax does matter, wouldn't it? nobody wants leaky abstractions!
ironically, Clojure is a great example of a hosted language that does not leak much in terms of underlying runtime, as evidenced by the fact that it has been implemented on top of a variety of runtimes with decent control over cross-runtime code reuse.
> acknowledged that the JVM is a great and mature runtime but it's lagging behind on STM / actor capabilities
you're stating this as if it's a fact, but what is your evidence? afaik jvm has a very extensive actor model library (Akka) and clojure does include a solid STM implementation (https://clojure.org/reference/refs).
the reality is that both of these approaches to concurrency are simply not popular enough, so your grievances with JVM for (allegedly!) lacking some important features relevant to them are not in sync with the demand.
> Tearing down a straw man is not impressive and it comes across as you trying to gain visibility by deliberately misrepresenting your discussion opponent's arguments.
don't debate-bro me bro, there are no straw men and no misrepresentations of your messages. if there are invalid assumptions - it's because instead of turning this into a dozen-messages-deep interrogation of what you really meant, i'm taking shortcuts and assuming what i believe is most plausible interpretation.
> that would be the reason why syntax does matter, wouldn't it? nobody wants leaky abstractions!
Well I thought we were describing our current reality, not our _desired_ one? Yes nobody wants leaky abstractions and yes they are everywhere.
Syntax matters insofar as to discourage bad habits, is what I'd refine from my previous statements. Most programmers go for the default so defaults and syntax that steers you the right way to think and write matter a lot.
That being said, people write FP Rust (myself included) and have plethora of JS libraries where immutability and FP patterns are the default. Which is a sad state of affairs but much better than nothing -- as it's introducing programmers to immutability and FP and they otherwise would never know.
> as evidenced by the fact that it has been implemented on top of a variety of runtimes with decent control over cross-runtime code reuse.
That was my top 1 reason to try it btw; I was intrigued by the fact that people are interested in making it work universally in at least two very different runtimes. To me that signals good language design and good architecture. Which I already knew; Clojure and Racket are amazing on their own.
> you're stating this as if it's a fact, but what is your evidence? afaik jvm has a very extensive actor model library (Akka) and clojure does include a solid STM implementation (https://clojure.org/reference/refs).
As already said multiple times in the thread -- my info is stale (as claimed by multiple posters).
That being said, has Akka started making full use of JVM's new green threads? Has Java itself started introducing immutability and STM / share-nothing as first-class citizens? If not, then by the "programmers reach for the defaults first" rule above I'd think Java is not yet ready. OK Clojure has these amazing libraries, kudos. Has anybody rolled up their sleeves and said "Alright, BEAM VM's reign is over, I am making the same or better runtime as them in Java / Clojure!"? If not, I'll not yet revisit.
I just don't want to deal with the endless pit of determinism bugs that global mutability nets us. The gift that keeps giving.
If Akka / Golang's runtime / Rust's various actor-emulating libraries catch up to the OTP, I'll very likely drop Erlang/Elixir because it's a struggle to have a good stable employment (or even contracting lately) with them.
Even if the BEAM VM is slower and has a few annoying sharp edges, its strengths nullify its weaknesses due to the nature of my work (HA web / API servers and also API gateways and orchestrators).
> That being said, has Akka started making full use of JVM's new green threads? Has Java itself started introducing immutability and STM / share-nothing as first-class citizens? If not, then by the "programmers reach for the defaults first" rule above I'd think Java is not yet ready.
> OK Clojure has these amazing libraries, kudos. Has anybody rolled up their sleeves and said "Alright, BEAM VM's reign is over, I am making the same or better runtime as them in Java / Clojure!"?
will akka use green threads? i'm sure it will when the developers behind it deem them useful.
will jvm add immutability, stm and share-nothing primitives that (i assume you allege) are missing? sure, i guess, when it becomes frequent enough ask.
you make it seem as if the world of software development is in some constant battle for supremacy, but it just isn't.
there is no "BEAM VM's reign". and if there was - it's unlikely that anybody would care to topple it. people just want to get their job done and they use whatever tools are available, familiar and convenient for them.
> I just don't want to deal with the endless pit of determinism bugs that global mutability nets us
and a lot of people agree with you. and there's a lot of tools that address that. and i can assure you, when working with clojure, even on the blasphemous mutable jvm runtime, that class of bugs is non-existent.
Well, I don't think you and I disagree on the premises, with the exception of you believing that I make some outrageous general claims -- which I did not.
> you make it seem as if the world of software development is in some constant battle for supremacy
I could have misinterpreted other people in the past -- very possible. But I also stopped caring about "battles for supremacy" a long time ago and at this point in life and career I simply use my experience and brain to go where my work is more productive, deterministic and fulfilling. To me immutability, share-nothing actors, strong vertical scalability (where a lot of PLs and runtimes do well, not only my favorites) and DX (like live prod REPLs and generally trivial observability) are those axii along which I thrive.
You and a few others seem to have emotional reactions to this which, I assure you, are very unnecessary. I ain't threatening neither your livelihood nor preferences.
> That being said, has Akka started making full use of JVM's new green threads? Has Java itself started introducing immutability and STM / share-nothing as first-class citizens?
Amazing how it doesn't even cross your mind that there are trade-offs to those choices. Green threads are awesome, but guess what, they come at a cost. Same for share-nothing semantics.
> Has anybody rolled up their sleeves and said "Alright, BEAM VM's reign is over, I am making the same or better runtime as them in Java / Clojure!"?
You are again presupposing the BEAM has an absolute superiority over the JVM. "Better runtime" makes no sense on its own. Better is always relative to something. Better for whom? For what?
I'd bet that you work on a traditional CRUD enterprise software, and that IO(the database) is the real bottleneck of your app. In that case, sure, the BEAM is a solid choice(so is Python, Ruby and PHP nowadays). But let's please not pretend that is all there is to software engineering.
Any good reason for your rude tone? If we are going to invoke the what crossed somebody's mind trope, I'd lead with that when talking to you -- did it cross your mind to speak calmly and not assume something "did not cross" somebody's mind?
RE: your other similarly rude comment, I have not "appealed to authority" anywhere. I said that I have used multiple PLs / runtimes and made an informed choice... for me. I don't intend to add "...for me" after each sentence. It's redundant and obviously implied when it comes to tech because obviously people have made well-working prod systems with combinations of bash and Perl ages ago. So obviously people can make nearly everything work.
If you don't intend to discuss out of position of curiosity but want to jump on people then I am not interested.
Your first comment assumed I was "speed-running to a conclusion and squinting too hard", and this was "similar to the weird childish name-calling". I think that is in the same area(or worse) than saying "Amazing how X didn't even cross your mind". And sorry, but invoking that you have experience with X, Y, Z and thus your opinion is informed after criticizing some technology IS an appeal to authority.
> If you don't intend to discuss out of position of curiosity
I'm not the one making sweeping statements on the superiority of one piece of technology. Reading your other response, I think you are the one who have little to no curiosity in understanding how you might be wrong.
> With respect, this topic in particular has been beaten to death.
Yes and no. From the discussion here I've learned about the existence of jank, which wouldn't have come up a year or so ago and might be an interesting solution to a problem for me as it evolves (that problem mainly being me not wanting to use C++ or any of the other directly supported languages in a plugin ecosystem). So these things are worth bubbling up every now and again just for the discussion to have a chance to play out.
The JVM is perfectly capable of Golang-style green threads now. As for Erlang, the creator of Clojure have commented in the past on why he dislikes the Actor model, and I think it is a fair criticism. Sometimes I see people praising Erlang VM as some panacea in which all the VMs should strive to be like. This is overly simplistic in my opinion, and ignores the huge trade-offs that the Erlang VM has.
You might be speed-running to a conclusion and squinting too hard if you use the word "panacea". Similar to the weird childish name-calling people do in Rust threads (somebody met one brainless zealot and now of course they'll judge a community of hundreds of thousands of devs by that one loony).
I used Java, Golang, Rust, Elixir (so Erlang).
My opinion is informed. STM / share-nothing-actors lend themselves amazingly well to online services for many reasons, better explained by other people and documented elsewhere (and I did not come here to advocate but to express preference and offer the take of somebody who has been around).
I am not denying that the JVM might have almost caught up in the meantime. More than a decade ago it did not.
And yes the BEAM VM is absolutely and markedly _not_ a panacea. It has a few weird sharp edges. It's just that in my work I have found having to avoid them still worth it compared to the alternatives (global mutability and more primitive parallelism which was the case for the JVM for decades).
I have used Clojure(JVM), Elixir(so Erlang) and a bit of Golang professionally too. So my opinion "is informed" too for that matter, but this kind of appeal to authority adds nothing to the discussion.
> I am not denying that the JVM might have almost caught up in the meantime. More than a decade ago it did not.
This presupposes that the JVM had something to catch up in the meantime. Again, this lacks nuance and brings nothing to the table. The JVM makes different trade-offs than the Erlang/Golang VM does, and has different strengths and weaknesses. Both of your comments completely ignores that.
> It's just that in my work I have found having to avoid them still worth it compared to the alternatives (global mutability and more primitive parallelism which was the case for the JVM for decades).
Clojure runs on the JVM and avoids mutability pretty well. It is amazing for writing concurrent software, and has been for many years(i.e more than a decade ago).
> Similar to the weird childish name-calling people do in Rust threads
If they are, I have not heard about it (which does not mean much, I check Java once a year). And if they really are then I'd give Java a serious look again because it's a mature ecosystem that was gimped by ancient runtime decisions for literal decades.
As of Java 24 (Java 25 being an LTS) I'd say they are equivalent. You can use a virtual thread just like you use a regular thread and there's basically no handicaps or gotchas. In Java 21, when they were released, there is a gotcha that the pretty normal use of the `synchronized` keyword would pin a "carrier thread" which ends up blocking all virtual threads from running on that carrier thread.
Pinning can still happen in some much more rare cases, same with go. For example, FFI.
The memory usage, performance, etc are all go like. You can spawn millions of virtual threads with hardly and memory requirements and without overburdening the OS with context switches. The JVM also enjoys faster GC performance with virtual threads.
Thousands of share-nothing actors (fibers / green-threads) with first-class support for communication between them, for a start. Erlang/Elixir -- immutability as well.
BEAM threads are kinda magicsauce tho, instructions have a cost and after a certain cost total (quantums) the scheduler can divert to another virt thread to guarantee forward progress. Also the immutability rules etc make it easier to optimize this switching.
Eh, reduction counting isn't magic. Golang manages similar preemption semantics without counting that many operations (some tight loops do have barriers inserted every so often, but that's the exception and not the rule). And reduction counting has some serious costs! It slows the runtime down a shitload (and the BEAM is already in the bottom half of interpreted language runtimes by speed) and makes lots of JIT-flavored runtime optimizations slower or harder to implement.
I like immutability too; I wish Java and Golang did more of it. It costs a lot in terms of unexpected copies in the BEAM though, there's less copy-elision optimization than you'd think. That especially bites if you're doing a ton of message passing, because of how process heaps are implemented and how garbage collection (traditional or ETS/ThreadProgress-based) works.
I think what I want is something like Golang but with goroutine-based ownership semantics (or Rust with the Go runtime and goroutines): en excellent scheduler for extremely light-weight green threads, no refcounting or reduction counting, and all the clever optimizations around channel sending and copy elision--but no ability to use a value after it's sent to a channel, and only channel-based access to shared global state. That'd get most of the benefits of process-local heaps but without the (copying, cache/memory fragmentation) drawbacks.
These are all true and I have recognized those as innate limitations of the BEAM VM. For now I am OK with those but I am already skirting at the limit and I am starting to want to jump to Golang and Rust again.
Obviously. But it's really nice to have the option, and none of us knows the future. I've been bitten by those "0.1% chance" things much more times than I would be not-embarrassed to admit, and I know I a not alone.
Good, thanks for the grounding. I'll have to reevaluate at one point then.
As I just posted in another comment (https://qht.co/item?id=48384622), I'd probably drop Erlang/Elixir due to difficulties of employment and contracting -- if the more popular languages get those STM / share-nothing runtimes.
"requires" is of course subjective, there are always multiple ways to do something. But sometimes it is convenient to model a system as concurrent execution streams, for example: multiple sessions (servers), multiple entities (games, robotics), multiple in-flight transactions (any kind of i/o or concurrent compute). Agreed these are often C++ use-cases but there are obvious benefits to using Erlang or other virtual machines: memory safety, isolation, fault tolerance.
from experience, during bursts it's never actual web/api server that is bogged down, it's the downstream io bottlenecks.
if your accepting layer is abstracted away and implemented correctly, there is very little performance difference between different concurrency approaches and all you're exposed to as developer is implementation of your handler functions.
Not the case; good abstractions are valuable, but the performance differences between runtimes are very real.
Take the example of some simple HTTP<->blob store service gets slammed with millions of requests when someone using the API does a backfill via some framework on their end that aggressively scales request volume up and out.
Something like, say, async Python/starlette with a coroutine per request is gonna perform slightly worse than Erlang, which in turn is gonna perform much worse than Go.
You're right that those differences are sometimes marginal when the latency of whatever IO the backend's doing dominates the equation. However, in my experience huge volume surges show issues with the runtime (the thing managing/launching multiplexed request handler routines) or the ecosystem (the backend IO libraries' ability to work with the runtime's IO multiplexing and make things like request coalescing easy or automatic) more often than you'd think.
It really takes surprisingly little volume to cripple a return-hello-world Phoenix app that indirects the "hello world" behind way too much middleware and message passing; it takes even less to kick over, say, a Gunicorn instance returning "hello world" at the bottom of the Django middleware stack. Golang with Gin, on the other hand, is surprisingly hard to cripple in the same way. And I say that as someone who likes Elixir and Python a lot more than I like Go!
> You're right that those differences are sometimes marginal when the latency of whatever IO the backend's doing dominates the equation. However, in my experience huge volume surges show issues with the runtime (the thing managing/launching multiplexed request handler routines) or the ecosystem (the backend IO libraries' ability to work with the runtime's IO multiplexing and make things like request coalescing easy or automatic) more often than you'd think.
fair enough, although at this point we start talking about LB in front of the thing, consumption mechanics, autoscaling signals
i will still maintain that my simple advice for a dev worrying about scale, is that they should focus their efforts on ensuring downstream IO doesn't get overwhelmed (db read replicas, caching, etc) before optimizing runtime performance or autoscaling out unnecessarily.
> focus their efforts on ensuring downstream IO doesn't get overwhelmed (db read replicas, caching, etc) before optimizing runtime performance or autoscaling out unnecessarily.
All good advice, but the choice of runtime can affect the point at which autoscaling and load balancing even need to enter the conversation at all. Optimizing, say, a mostly in-memory cache service and writing it in Golang may yield results like "we can run a single instance of this and serve three orders of magnitude of business growth; slap it behind a DNSRR or a k8s NodePort for update/replacement/fast failover if it crashes, no complex load balancer needed", where writing the same thing in, say, PHP might require discussing orchestration/load balancing/memory/worker process recycling/autoscaling early on in the service's lifetime. Being able to skip those conversations (entirely or for a long time) is a very significant business benefit.
Thank you. As a guy who made a career out of Elixir (and begins to regret it recently but oh well) I agree that Elixir's throughput is not amazing. However, it can get very far and we should always optimize for the most common usages.
I've personally rewritten one hobby and one professional projects from Elixir to Golang and loved the result; as you said, extremely difficult to bring down a Golang service to its knees.
One clarification: Phoenix server behind Caddy/nginx fairs better btw. But, details. Your point stands.
I am yet to see a Rust web/API service I wrote to _ever_ buckle under pressure and just crash. It was either an application bug (like the famous Cloudflare's `.unwrap()` error from the last weeks/months) or the Linux OOM killer. Literally never crashed. But I did witness it brutally murder a MySQL cluster because it couldn't serve it fast enough. That was both fun and terrifying to watch on the dashboards.
> I did witness it brutally murder a MySQL cluster because it couldn't serve it fast enough. That was both fun and terrifying to watch on the dashboards.
Haha yep. In my experience, everyone running CGI/process-per-request application servers is bullish on switching to a concurrent or cooperative runtime...until they realize they just removed the primary ratelimiter on downstream DB/service accesses.
The converse war stories are also amusing: people rewrite their whole app in a concurrent/asynchronous framework and nothing changes, because the DB driver is still farming out all queries to a tiny fixed-size threadpool of connections that was the bottleneck all along.
Oh yeah, definitely. If your DB server (or any storage backend) cannot have like 200+ connections alive at all times then it's absolutely pointless rewriting your app in Elixir or Golang. You'll just serve DB timeouts in your responses.
(Upvoted for a really relevant and valuable refinement to the thread.)
Admittedly that layer is almost always abstracted i.e. AWS / GCP and various other smaller hosted solutions that handle a good chunk of load balancing for us. In that landscape BEAM VM's strengths shine even brighter. I've seen firsthand that you can in fact bring a BEAM VM to its knees if you expose it just like that to the net. It's not pretty. Golang fares a touch better and Rust seems almost immune (provided one does not screw up their caching layer and don't do elementary N+1 query mistakes).
although this is a deliberate choice rather than some accidental defect. Clojure went with STM as its concurrency model, if you're not buying into that and you want an Actor-centric language it's not the right choice to begin with.
STM is seldom used in modern Clojure projects, it is certainly not the dominant model. Most projects I am aware of use a few or even exactly one atoms with immutable data structures.
I think one of Clojure's main strengths for concurrency / parallelism is structural sharing of immutable data across threads, which you simply cannot do in Erlang.
I kind of agree. Erlang took the "share nothing" thing to extremes by copying everything which, while improving memory safety by a huge margin, introduced (a) various GC footguns (like the shared big binaries gated behind refcounting which is a famous footgun for very long-running apps) and (b) reduced its raw speed.
> Programming language syntax scarcely matters. It does to some extent but we the programmers tend to over-romanticize it. The runtime and its properties are the much better thing to optimize for.
I'm not sure I understand this argument. Java and Clojure share a runtime, but an idiomatic Java codebase is going to have a very different architecture and design to an idiomatic Clojure codebase. Conversely, a codebase written in Go may end up looking very similar to a codebase written in Java, despite using different runtimes.
To be clear, I'm not questioning your choice of runtime or language. I'm just curious why you think that "Programming language syntax scarcely matters", as to me that seems the same as saying "How a codebase is architectured and designed scarcely matters".
I don't see how the latter follows from the former? The former is much bigger and more abstract; syntax is just one of the vehicles to try and codify it.
F.ex. if you have an universal construct of green threads / fibers then 7 PLs could express it 7 different ways, yet underneath they'd all be the same.
The programming language informs the design of the system. As I said in my earlier comment, an idiomatic Java codebase is going to be designed very differently to an idiomatic Clojure codebase, even if they both intend to solve the same problem.
Scala has no immutability encoded in its runtime either (as it's the same as Java), but yet syntactically it's immutable in practice. Will the JRE technically allow a val to be edited through some third party thread inspecting your code and messing with memory? Sure. But it's not a reasonable fear in any real world environment, where I cannot remember, in 15+ years of professional scala, a case where anything I expected to be immutable (everything) to be mutated under me. Nowadays people using in in an FP style don't even think of the physical threads, as green thread libraries are taking care of all the scheduling.
So focusing on the runtime's guarantees doesn't seem like a practicality focused argument to me.
You are citing a commendable exception (Scala) to tear down a bigger argument which is not exactly a fair discussion.
Furthermore, if you trace my comments, you'll see that I had to choose PLs years ago (12+ to be precise). Things were quite different at the time. Java might have almost caught up today; back then we couldn't even be certain `synchronized` is stable all the time. Just saying.
Scala did very well then, judging by your words. I could probably offer a loose analogy to Typescript as well; while it does compile to JS underneath, they added a stricter layer that makes programming in it more deterministic and stable. (Not the same thing because my main point was "runtime" but hey, show me a perfect analogy.)
You are free to say your last sentence. I am free to disagree. My practice has shown me that runtimes bleed into syntax almost always. Exceptions exist, sure.
But syntax must necessarily include what it's representing, no? For instance, `{:a 1}` represents an immutable map in Clojure, in the same way that `42` represents an immutable integer in Java.
I'm not sure I agree. Certainly there are differences other than syntax, but that doesn't mean syntax is irrelevant. For instance, would Clojure programmers use maps as much if there was no syntax for map literals?
Syntax determines what parts of a language are within easy reach, and therefore affects how programmers use the language. Tools that a syntax make easy are used often; tools that syntax makes hard are used infrequently. This indirectly impacts how a piece of software is designed.
This is very much what I meant in the post (hi, I'm the author :P)! CL has maps, but they're a pain to use - not just because of the syntax, but because of the relative dearth of standard library functions to work with them compared to say, lists or even vectors.
Good thing you have a variety of those nowadays.
Clojure runs in the browser, Node.js, cross-compiles to Dart, works stand-alone via babashka and has a brand new C++ interfacing implementation in Jank.
The ergonomics of using a proper REPL and interactive programming is hard to beat.
> Programming language syntax scarcely matters. It does to some extent but we the programmers tend to over-romanticize it. The runtime and its properties are the much better thing to optimize for.
But that really depends on what you're doing. For example if I'm not mistaken Amazon was run for a very long time on a Java backend. And so was GMail's backend (and back then GMail's frontend was, IIRC, Java converted to JavaScript using GWT).
And by "early Amazon" and "early GMail", we're already talking about massive scale. It's not as if the JVM got worse since then (as someone commented: a recent addition is that Clojure now use Java's virtual threads) and it's not as if it didn't scale.
So I'd say having Clojure on top of Java (for those using that Clojure: there's also ClojureScript, babashka, etc.) ain't really a problem, as long as you're fine with the occasional Java stacktrace and Java ecosystem (GP mentions that btw: that he's not familiar with Java and that, I think, can be a bit of an issue).
I'm not sure Clojure is about it's syntax: I like the focus on immutability / pure functions and I do really dig the REPL a huge lot. In addition to that something has to be said as to the incredible stability of the language and many of its libraries.
The big value add to me is that I can have a REPL and inspect, in dev (or in prod but that'd be wild), the app I'm working on. And manipulate it: redefining variables and functions etc. And it's not some hacky hot-reloading bolted on as an afterthought kludge: it's a real Lisp REPL. There's value in that IMO.
Elixir has all that _and_ Erlang OTP's amazing guarantees. Hence I landed on it.
Elixir also offers LiveBooks i.e. you can create pre-made recipes with which you directly remote into your staging / prod and do stuff.
All that with immutability and potentially 6 digits of actors / green threads with a share-nothing architecture.
---
RE: early Amazon / Google, sure. They made do with what they had and it was and still is a heroic effort. But can we agree that they succeeded _despite_ the numerous warts and defects of the PLs and their runtimes at the time? Not _because_ of them?
I feel that people latch onto the misleading "they succeeded with language X and are big, hence the language X is great" thing way too often. No. It's not true. The only thing that follows from "big company A made it big with language X" is: "company A has an amazing engineering team". Nothing else.
However, certain fundamental decisions of a language can be dealbreakers.
Requiring declarations on your functions and giving those declarations sigils so that they can be parsed quickly is an important syntax decision. Almost every modern programming language has converged to this idea.
Or take, for example, Lua. For me, personally, the 1-based-ness of Lua is simply a dealbreaker no matter how good anything else about it is.
For the "Lisps", I LOATHE the fact that you traverse lists and vectors in completely different ways--you can't just drop any container-ish thing into something that iterates/collects it. This is something that both Clojure and Racket seem to agree on--you have something that acts like a "collection" and you can walk across it the same way regardless of the specific type of collection it is. Of course, that is why a bunch of Lisp purists loathe Clojure and Racket while I like those languages. Shrug.
I find RAII (Resource acquisition is initialization) to be the source of all things evil if it infests a programming language. The popularity of C++ and Rust speaks to the quite large number of people who think my opinion is bullshit.
So, yeah, base syntax matters far less than it used to. But the engineering decisions that went into making that syntax correspondingly are far more important.
I agree with most of what you said, but I am puzzled about your claim about RAII.
Whether it is good for any kind of resource acquisition to look the same like an initialization is debatable.
On the other hand, I cannot see any counterargument to the principle that releasing any resources should not be done explicitly, but only implicitly, upon leaving a block (using reference counts for shared resources).
I doubt that you advocate for the use of explicit release commands for resources, which are a notorious source of bugs, so what is that you consider as not being the same as RAII?
RAII was a not very useful acronym that was just another form to say that the C or C++ programmers should never use the PL/I style of explicit free commands, despite the availability of functions like "free()" or "close()" in the standard library, but both memory and files and any other kinds of resources should be managed with automatic releasing.
I do not see how this sound principle can infest any language.
Obviously, I have seen examples of bad RAII implementations, like I have seen examples of misuse for any other programming principle.
> I doubt that you advocate for the use of explicit release commands for resources, which are a notorious source of bugs, so what is that you consider as not being the same as RAII?
Those are not the only choices.
Garbage collection is at one end of the spectrum--fully manually managed is at the other end of the spectrum. There is also another axis of acquiring/releasing allocation by object or acquiring/releasing object by pool. And, if you have it, there is the axis of allocate only at startup and never free until end of thread/application vs. allocate only at a frame of of time and then destroy them all at the next frame vs. allocate whenever and wherever.
RAII encourages the usage of lots of tiny individual objects allocated whenever and wherever all with their own lifetime cycles and makes understanding the memory usage of your application very difficult (this was the whole reason Rust was made--C and C++ made managing memory in Firefox ridiculously diffused and impossible to corral).
And, I'll be blunt, I think that Rust/Zig/C3 etc. are not the right direction in spite of the fact that I use Zig a lot. I think that the garbage collected languages cede far too much in terms of performance to the compiled ones and GC languages (like say OCaml) should be being used for systems programming more often.
For example, I think we would all be in much better shape against AI vulnerability scanners if more systems programs were in GC-type languages.
I think we can easily agree they are two entities (syntax / runtime) that feed off of each other. And I do agree that previously the syntax mattered more.
(I very much agree on Lua btw.)
Personally I am very disheartened. Surely algebraic data types should be universally a good thing and all PLs should gradually adopt them? But no, endless HN / Reddit threads bike-shedding.
That's completely fair and that would be my process exactly, if not for one thing -- fan noise.
I really _hate_ laptop fan noises. That's why a Macbook Neo will win for me when I get to travel more. Hopefully the world will produce a good competition for it in the meantime.
That's fair, we all have our preferences. Each vendor has "their feature" that drives sales, for sure. Neo has the "I hate fan noise" crowd. Lenovo has the "I like the pointer nub" crowd. Dell has the... uh... well they have that secure boot thing.
> Once you realise that SQLite is just an aggregated view over this log it calls into question if you needed a third party query engine at all, suddenly the whole NoSQL vs SQL debate becomes a meaningless implementation detail.
You are correct on the premise, however my observation has been that teams who try to be lean and mean and minimize dependencies always start off believing they don't need a query engine... and they always end up needing one, often making things much worse by refusing to just migrate to any database due to sunk cost fallacy.
I fully agree with your argument that it's all just file operations in the end. To me SQLite wins for everything beyond persistence however: the SQL commands, extensions, backups, changesets / patchsets.
Can you, I, and many others write something that has 10% of SQLite feature set that serves us perfectly? Absolutely! But I don't want to, and in business settings nobody will allow you to hand-roll crucial infrastructure software (like databases) unless they're in full crisis mode and you're the mega-expensive consultant brought in to save the day.
I would love to read your analysis somewhere. It's a current interest of mine, both hobby and professional -- and limited to Elixir, Golang and Rust -- and I'm still slowly scoping the landscape.
Recommendations for good engines? Or just thoughts and analyses?
MacBooks are still unbeaten hardware-wise. Yes 8GB of RAM is embarrassing, no question about it.
I'll buy a Neo just for travel. I will remote to my machines with it.
Though I'm flabbergasted why has nobody else made as thin and lightweight laptop like Apple... for decades. And that includes no fan, or just a very quiet one.
I have no love for Apple at all. But the Neo is a game-changer and a well-deserved kick in the nuts of, well, every other hardware company really.
An interesting perspective the designers of the Tesla Model S had was "battery bucks". The oft-cited example was some fancy wheel bearing which had a expensive per-unit cost which no normal OEM would even consider, because being half a percent more efficient didn't matter. Whereas for Tesla, half a percent efficiency reduced the need for hundreds of dollars of batteries.
I wonder if there's some opportunity for Apple to think about "RAM bucks" and turn memory efficiency into an economic opportunity. I find it difficult to believe that modern operating systems actually need to consume so much.
Point me at a laptop as small and quiet. Hell, let's forget weight. I'll gladly lug around 2.5kg machine if it almost never turns its fan on and is as compact.
Actually let's not care about compactness either. Can go up to 17". Just show me a truly quiet laptop.
You are misjudging me here. I'm moving back to Linux after using Macs for 7 years. My desktop Mac is soon going to be history. macOS makes too many choices for me that I dislike. We're not just talking stylistically or philosophically; those I can mostly get over if needed. We're talking objectively and technologically.
I want a good travel machine however. People look at you funny on meetups when your laptop starts blasting jet engines. Nobody is saying it out loud but you're becoming the odd one out and that has a social and career cost. (That's not even touching the fact that noisy machines stress me out and break my focus.)
I'll get a Neo for travel even if I hate giving Apple money. Because everyone else is two decades behind.
Point me at a laptop with a touch and pen support and detachable keyboard, within the 13" range and 260+ppi resolution, that can virtually run modern software and mid-tier games at workable speeds.
That's the bar my personal laptop clears, and no mac does. Should I be claiming that it has unbeatable hardware ?
PS: We've been in this for a very long time. Even during the PowerPC days, it was a meme to take a mac, adjust for exactly the same hardware specs and tout that the mac was unbeatable for the price. Thing is, having different hardware specs and tradeoffs is exactly the point of the PC market. I still feel great for the people that exactly fit the "one size fits all" offering, but getting a machine that perfectly fits one's needs is extremely valuable IMHO.
Sure but that's only a theoretical, as in, a non-existing argument. You don't get to tell me what I want in a laptop; I already stated it and stand by it.
"Unbeatable" comes from "satisfies all my hardware requirements". If you want to be difficult and try to grill me to append "...according to what I deem an important criteria" at the end of each sentence then that's a separate conversation, and one I'll refuse to have.
"the Neo is competitive for basic office and media stuff" can be a better way to put it. I'd say the Surface Pro is a more competent but pricier jack of all trade.
There's so many ways to express the same points with not much more words. Especially when the article is about a machine that is a completely different league from the Neo.
Not really, I tend to avoid computer appliances, other than phones and tablets.
I certainly am not paying 800 euros for a travel laptop, when a 350 euros Asus 1215B, that could be upgraded, was able to survive about a decade, with 8GB in 2011!
> x86 is on the way out in for most consumer devices.
Define "consumer devices"? I am holding on to my AMD Ryzen machines until they literally fall dead. I have no complaints from them. Maybe some modern or even next-gen ARM CPUs will be even better on Linux but I don't think we are quite there yet.
x86_64 is here to stay for a long time still.
But maybe you literally meant x86 as in the 32-bit CPU arch? If so, I'd mostly agree but not quite; they could be used in low-power micro-PCs for a long time still as well.
From my research on Macbook alternatives only the Zenbooks looked like almost-an-even-match to me. Curious what's your experience with day-to-day fan noise and heat.
I am rooting for Framework myself.
reply