I'd like to suggest trying integrating the syntax highlighter from JOE for speed, but JOE is GPL...
Also, I'm not sure how large the Lua/LPeg state is, but for JOE we found the highlighter is more accurate if you retain the state of each line (or every n lines), and start parsing from a real known state. It looks like Vis starts parsing 16K before the start of the window.
Thanks for the hacking file, it is a good read for people like myself who are interested in text editor implementations. If time permits I will try delve a bit into the joe code base.
As for the syntax highlighting in vis, yes it currently is stateless and only considers a fixed range of text prior to the start of the window. Having no state at all simplifies some things and allows vis to highlight even extremely large files at the expense of the occasional glitch. In practice it seems to work reasonably well.
This approach also requires that the highlighting is fast, because nothing is cached, it is completely redone after every cursor motion. I'm quite happy with LPeg so far, it is convenient to express grammars in a high level language like Lua. It also supports nested grammars nicely, but due to the completely stateless approach taken by vis this is currently not fully exploited. As an added bonus there were already ~100 lexers (of admittedly varying quality) available for use.
Thanks for writing the JOE editor. I've been using it for simple text editing tasks (commit messages, etc.) now that I've moved over to a Linux system. I appreciate its simplicity.
I shall add to your thanks. Joe's the first thing I install on any new box or server and has been my default "quickly edit this thing" editor for a decade and a half now.
Also, I'm not sure how large the Lua/LPeg state is, but for JOE we found the highlighter is more accurate if you retain the state of each line (or every n lines), and start parsing from a real known state. It looks like Vis starts parsing 16K before the start of the window.
https://sourceforge.net/p/joe-editor/mercurial/ci/default/tr...