I have many GPL projects (e.g. https://github.com/rochus-keller/Oberon, https://github.com/rochus-keller/Luon, https://github.com/rochus-keller/Micron) and spend a significant amount of time in them. GPL has always explicitly permitted commercial use; that's a feature, not a bug, dating back to Stallman's original vision. Any person or company can use my code (or Kefir code) under the terms of the GPL, as I use code given away by companies under GPL or even more liberal licences for free. That's the deal. GPL is a license explicitly designed to maximize use, so it doesn't make sense to object to a specific form of use. The claim that AI companies are somehow violating GPL by training on GPL code is legally baseless (I studied law here in Switzerland and had lectures about international IP law); also the FSF itself has not claimed otherwise; even if it were prohibited, it would be a copyright enforcement problem, and not a reason to stop publishing. I don't know Kefir, but it looks like a great (even optimizing) compiler. So it's really a pitty that its development is no longer open source.
The GPL, unlike the BSD and such, intends to prevent the closing of distributed derivative works. LLMs trained on GPL code can produce derivative works without any enforcement mechanism.
You may be fine with that, but the GPL is not a public domain license, and LLM training treats all things as if they were public domain.
> LLMs trained on GPL code can produce derivative works
This confuses two completely separate things. GPL governs distribution of derivative works. An LLM trained on GPL code does not distribute that code. The model weights are not a copy, a derivative, or a distribution of the training data in any legally recognizable sense; "influenced by" is not "derived from". The enforcement argument is a non sequitur; the GPL has never had a technical enforcement mechanism; it's always been legally enforced after the fact by copyright holders who discover violations. So if the LLM would indeed produce output sufficiently similar to my code and someone would publish it in violation of GPL, I have the same legal means to enforce my rights as if the code was copied by a human.
> An LLM trained on GPL code does not distribute that code.
You can't simply make that assertion. You'll have to prove that LLMs do not actually contain encoded copies of copyrighted code and that they are incapable of reproducing such code verbatim.
There is no evidence for such a claim, and so your entire argument is completely baseless.
> You'll have to prove that LLMs do not actually contain encoded copies
In law, the presumption is that an act is lawful unless proven otherwise. The burden lies on whoever claims a violation occurred. I already went into the case of sufficiently similar reproduction in my previous response.
Same here. The 2b, 3b and Zero 2 have the best cost-benefit ratio. I recently migrated the Oberon System 3 to those three boards and particulary the Zero is a great host for this bare-bone system: https://github.com/rochus-keller/oberonsystem3native/. I hope the Raspi company postpones the end of life date of the 3b and Zero 2.
Thanks! And that's awesome! You know, it's strange when I've been using RNBO to do much less heavy stuff than many of these patches do I kind of assumed that the reason it ended up with a fairly high WebAudio resource consumption was because... I'm an idiot. :D
But now I kind of wonder if it could be connected to the fact that signal-rate processing (when using RNBO) is per sample - at least according to my understanding, I could be wrong. So if you have a vector size of, say, 128 samples then whatever you do would execute 128 more times than if you "just" did signal-processing PER vector. So maybe it could be related to that... I think I'm just might send a mail to cycling74 and ask hehe.
But nonetheless, it's pretty crazy how insanely low the resource consumption is for the pd patches I've profiled so far. Sometimes it's not even above 1%.
Interesting results. Particularly they barely found a speed-up of newer compared to older LuaJIT versions (rather the contrary). Maybe they should have used the Are-we-fast-yet suite instead of the (random looking) set of microbenchmarks. I did measurements of Lua and LuaJIT some time ago based on Are-we-fast-yet and saw significant differences in LuaJIT performance (see http://software.rochus-keller.ch/are-we-fast-yet_LuaJIT_2017...).
I am currently implementing a personal programming language on top of luajit. Have you found that generating bytecode is actually faster (compile time and more importantly runtime) than just generating Lua (like Moonscript and Fennel does)?
Cool. It's still a great backend, even for a statically typed language, and the FFI and debugger features are very useful. Mono is yet another great backend which in addition supports multiple threads and has the higher performance, but with a (slightly) higher footprint. I have used both in different projects (see e.g. https://github.com/rochus-keller/Oberon or https://github.com/rochus-keller/Luon), and currently I'm even implementing a LuaJIT bytecode generator for my Micron intermediate language which uses LuaJIT essentially as a kind of C interpreter, even with my own native stack and GC (https://github.com/rochus-keller/Micron).
The reason why I switched from generating Lua source code to LuaJIT bytecode in 2019 was mostly the higher flexibility. I could better avoid operations not yet supported by the tracer, and some language features of Oberon+ didn't match well to Lua. And adding line information for debugging was much easier. And it was a fun project and the bytecode was well documented. It's certainly also a bit faster, but I haven't done specific measurements.
Deep copy value passing semantics by default (with outs for hotpath and some optimization).
Persistence by default using SQLite. (Again, opt out for hot paths).
Lua tables, but callable. Also every object is a continuation. Optimizations for js are float64 arrays with the 0 index holding some meta stuff - but semantically just objects and messages.
No variables, just fields.
Fields can be thought of as canned message responses. Fallthrough is “call”.
When you consider SQLite, you might want to get around SQL for impedance and performance reasons. There is a robust and decently fast key/value store in the backend. I used it in some of my projects (e.g. https://github.com/rochus-keller/udb used e.g. in https://github.com/rochus-keller/crossline/). Depending on read/write frequency, lmdb or tidesdb might be a better fit.
V8 was significantly faster than LuaJIT when I did measurements in 2021 (see https://github.com/rochus-keller/Oberon/blob/master/testcase..., didn't measure it since). Sure there are microbenchmarks where LuaJIT performs equally well or even faster, but overall (geomean of factors) V8 was 2.5 times faster. This comes with a price though. If you're interested in performance, Mono was equally fast as V8 in my measurements but much leaner, and more flexible with a well-documented bytecode, or use a LuaJIT or Mono backend for the development environment and offer a C transpiler for maximum performance at runtime.
Thank you. I am planning to argent Lua and Js(browser), and my (very much LLM assisted) demo already is with x2 range of js.
I guess in my head, I really liked the idea of the simplicity and size of Lua/luajit - but maybe for people like me, who have no hope of designing competitive JITs, the future is just to spit out JavaScript.
Your work has been an inspiration to me for quite a long time but the way.
Cool. Generation of symbolic music using transformers is indeed a pretty neglected field. I assume you have to encode more "musical knowledge" into the embeddings than when you "just" compress waveforms. Can you provide information on your embeddings?
Hi, It is actually not using transformers, those would be too slow. It is using a combination of CNN's and linear layers. Correct, it uses embedings, not waveforms or spectrograms. The inputs are midis, some of which I made myself in FL Studio. The model creates a "latent representation" from each midi, I can then sample randomly from this latent space to get an original piece. The most important part is the preprocessing in my opinion.
That's fascinating. This sounds like a variational autoencoder. The embeddings, which from my humble point of view (as a trained musician) are a largely unexplored field not really supported by existing theory, are at the same time game-deciding. Have you found a good solution for this?
reply