Mh, I thought about this a little and came actually to exactly the opposite conclusion: Implement as much as possible in Rust to get the fastest code possible. Do you have any more insights why this should not be possible / unsustainable?
You have two distinct products 1) An interpreter 2) a math language.
Don't write your math in some funny imperative computer language.
Keep the interpreters surface area as small as possible. Do some work to make sure you can accelerate numeric, and JIT/compile functions down to something as close to native as you can.
Wolfram, and Taliesin Beynon have both said Wolfram were working internally to get a JIT working in the interpreter loop. Keep the core small, and do that now while it's easy.
Also, it's just easier to write in Mathematica. It's probably 10x smaller than the rust code:
f[x_Integer]:=13*x;
f::help:="Multiplies x by 13, in case you needed an easy function for that."
EDIT: Another important thing to note is the people who really deeply know specific subjects in math won't be the best, or even good rust programmers. So letting them program in woxilang will give the an opportunity to contribute which they wouldn't have had otherwise.
I'm not a PL expert but isn't building a decent JIT a massive undertaking? I guess you're saying that the JIT itself would be what makes a project like this worth using in the first place?
It's like most things in software, if you constrain the problem enough, focus on the problems you actually have and make some smart choices early on, it can be a very modest lift on the order of a week or two for a 90% solution, but on the other end of the spectrum, it's a lifetime of work for a team of hundreds...
Most humans like external validation. Having other people use what they've built is a common metric of validation in open source software. Economically, these metrics can be used as credentials for things like job applications or consideration in joining groups.
Such a massive undertaking would be almost impossible without AI agents, so yeah, they help me. But with around 5000 tests, they are actually helping to improve the software quality!
Reviewing the correctness of code is a lot harder than writing correct code, in my experience. Especially when the code given looks correct on an initial glance, and leads you into faulty assumptions you would not have made otherwise.
I'm not claiming AI-written and human-reviewed code is necessarily bad, just that the claim that reviewing code is equivalent to writing it yourself does not match my experience at all.
Plus if you look at the commit cadence there is a lot of commits like 5-10 minutes a part in places that add new functionality (which I realize doesn't mean they were "written" in that time)
I find people do argue a lot about "if it is reviewed it is the same" which might be easy when you start but I think the allure of just glancing going "it makes sense" and hammering on is super high and hard to resist.
We are still early into the use of these tools so perhaps best practices will need to be adjusted with these tools in mind. At the moment it seems to be a bit of a crap shoot to me.
You wrote: “If they hand wrote the code we know they at least looked at it once.”
But that’s not true, because even if they did indeed hand-write the code, we as third parties wouldn’t have any reliable proof of that, and therefore still couldn’t draw the conclusion that they looked at the code.
Them claiming to have hand-written the code isn’t any better than them claiming to have thoroughly reviewed the code. We can’t know in either case.
> Them claiming to have hand-written the code isn’t any better than them claiming to have thoroughly reviewed the code. We can’t know in either case.
It is better, because most people aren't out-right liars. If he said "I hand-wrote the code", sure it doesn't prove it, but I would believe him. When he says "I thoroughly reviewed the code"... yeah maybe. That's the sort of thing people do lie about (even to themselves).
Sure, but you've got to start somewhere! And with the amount of progress I was able to make in just a few weeks, I'm very optimistic that the polish will come sooner rather than later.
Based on the list of contributors to your project, I am not sure this starting location is optimally suited to the task of building a foundation for polished, reliable, expandable software.
I have no dog in this fight, but simply claiming a count of tests get you anything is like saying your code coverage is 100% - it sounds really good until you think about what 5000 unreviewed tests actually... do.
If I go by the contributor numbers on Github, I see Claude has committed something on the order of 300,000 lines of code. I don't think it's reasonable to review that much code, even in weeks worth of time.
I haven’t needed to do such a thing in a while, but my “rule” for explaining how unreasonable is to say “if I only glanced at each line of code for 1 second, without bothering to understand the details, it would take me 3 and a half full 24hr days non stop to simply look at”. So it’s definitely more than 1 work week because presumably other stuff is going to need to get done in that time too. Actually understanding it is going to be at least a multiple of that, and probably the multiple is ~30x.
It's a defense mechanism. I was guilty as charge as well initially. Suddenly most of your l33t skillz are trivialized and surpassed by an inhumane actor. It's a tough pill to swallow.
i'm curious if you intend to reimplement highly optimized numerical algorithms, symbolic algorithms, and so on, accumulated and tuned in mathematica since its 1988 release?
it's a huuuuuuuuge amount of technology in the standard library of mathematica, beyond the surface syntax and rewrite system, i mean.
Hi, I'm the main developer. We're steadily getting closer to the next release which will support most features of Mathematica 1.0 plus some of the most popular newer functions (> 900 overall!). AMA!
It's a worthwhile effort. If successful, Woxi can enable a large mass of scientists and engineers who don't have access to Mathematica to run legacy code written for it. Also, Woxi would give those scientists and engineers who regularly use Mathematica a non-proprietary, less restrictive alternative, which many of them would welcome.
How does Woxi compare to other "clean-room implementations"[a] of the same language?
--
[a] Please check with a lawyer to make sure you won't run into legal or copyright issues.
There's a mystique around Mathematica's math engine. Is this groundless, or will you eventually run into problems getting correct, identical answers -- especially for answers that Mathematic derives symbolically? The capabilities and results of the computer algebra systems that I've used varied widely.
Hard to tell honestly. So far there was always some surprisingly straight forward solution If had any problems with the math engine. There is actually a lot of public research how equations can be solved/simplified with computer algorithms. So I'm optimistic.
I also stumbled upon a few cases where Mathematica itself didn't quite do things correctly itself (rounding errors, missing simplifications, etc.). So maybe it's actually a little overhyped …
Scmutils from MIT does a very good -- arguably better -- job for correctness. No symbolic integration by ideology and not identical. Sussman and Terman. Amazing attention to detailand correctness. Claude could probably bridge Scheme to Wolfram.
I'm not sure how important but- for-bug identical output really is.
This really doesn't bode well... I'm no expert in CASes, but everything I've looked at seems very naive. Vibe coding a Mathematica replacement makes zero sense to me.
Interesting, thanks for sharing. Naive question as I'm not familiar with Mathematica much (but aware of it and Wolfram Alpha and related tools), how does it compare to e.g. Jupyter or Julia or maybe another language (with its framework) that might be even closer?
I think Wolfram Language is just so much more ergonomic. No need to import dependencies - everything's included and consistent, very readable - yet compact - syntax, less gotchas than Python, R, etc., sensible default, …
Ymmv, but I've found that you sure do need to import things eventually, and it's not so ergonomic because most projects just end up as mega-notebooks.
Just like Python or any other language that looks easy for the learning examples, there are still hairy bits, they're just better hidden. The difference is that the debuggers for Python are far better.
Mathematica is great for quick stuff, but once you hit a particular level complexity it goes crazy. In this regard I find it similar to Bash.
License is a big deal, and not just for cost and openness, but also for practical use in pages like docker, ci/cd pipeline, cloud deployments, or other places licenses need to be dynamic.
Yeah, I've already looked into it, but decided to keep developing it "example driven" for now. Aka I'm playing around with it, and whenever I find something that's broken I keep a note of it and then I pick those notes one by one and implement them. Once the most common things are implemented I will start writing property tests to catch all the edge cases of each feature.
I'm saying you can go even further and automate the entire thing using LLMs/agents, it is pretty much the ideal use case: you have a black-box reference implementation to test against; descriptive documentation for what the functions should do; some explicitly supplied examples in the documentation; and the ability to automatically create an arbitrary number of tests.
So not only do you have a closed loop system that has objective/automatic pass-fail criteria you also don't even have to supply the instructions about what the function is supposed to do or the test cases!
Obviously this isn't going to be 100% reliable (especially for edge cases) but you should be able to get an enormous speed up. And in many cases you should be able to supply the edge case tests and have the LLM fix it.
(Codex is still free for the next few days if you want to try their "High"/"Extra high" thinking models)
You accidentally raise an interesting point: good, thorough public documentation, once considered a great selling point for your system, now invites automated reimplementation by competition. It would be a shame to see public docs vanish because it turns out they are literally machine readable specs.
I also think that sandboxing is crucial. That’s why I’m working on a Wolfram Language interpreter that can be run fully sandboxed via WebAssembly: https://github.com/ad-si/Woxi
Awesome. I'm pretty unfamiliar with the Wolfram Language, but my understanding that the power of it came from the fact it was very batteries-included in terms of standard library and even data connections (like historical weather or stock market data).
What exactly does Woxi implement? Is it an open source implementation of the core language? Do you have to bring your own standard library or can you use the proprietary one? How do data connections fit into the sandboxing?
I realise I may be uninformed enough here that some of these might not make sense though, interested to learn.
Yes, we agree that a lot of the value comes from the huge standard library. That's why we try to implement as much of it as possible. Right now we support more than 900 functions. All the Data functions will be a little more complicated of course, but they could e.g make a request to online data archives (ourworldindata.org, wikidata.org, …). So I think it's definitely doable.
We also want to provide an option for users to add their own functions to the standard library. So if they e.g. need `FinancialData[]` they could implement it themselves and provide it as a standard library function.
