Yes, that is definitely a limitation. If all models become worse at the same pace, we won't see any degradation either. I couldn't find any historical dataset of model benchmarks (I'd really have loved that, to see how performance holds over time vs. the initial announcement), so the Elo data from Arena AI was the least imperfect proxy I could find.
It'd be amazing if you could open an issue with a screenshot so I can take a look, I haven't been able to find issues when clicking on a group of models: https://github.com/mayerwin/AI-Arena-History/issues. Note: the model change points label being hidden when more than one curve is active is by design (to avoid cluttering), if this is what you were referring to.
Yes, Claude was very helpful to make this project work too (it would have taken me months otherwise to dig into how BLE works, and I'd probably have missed a lot of edge cases)!
Yes! I was surprised myself it was so complicated, especially as BLE MIDI is not something particularly new (Apple has nailed the implementation much better, luckily Pete at Microsoft is now doing his best to provide a comparable experience). When I played with USB MIDI 25 years ago it felt so much simpler.
I haven't measured it as my use cases were not sensitive to latency, but it felt pretty instant. Results will probably vary depending on the Bluetooth adapter, so best is to just test and see!
I doubt anything will come from this, granted how much he hyped Comma and where its at now.
Namely, the fact that this is still on the website:
> How is tinygrad faster than PyTorch?
For most use cases it isn't yet, but it will be
Despite being in development for so long is kinda telling. Like at least prove that you can make it work on CUDA as well as Pytorch.
Otherwise its just an personal project based on your ideology, and then when other companies don't play along with what you think is right, you get to blame their culture because you can never be wrong.
