This was an intentional move even if no one else wants to admit it.
You can't hide implementation from the User in such an environment; at least not to the degree you can with precompiled programs with debug symbols stripped. One requires you to infer function from raw assembly, the other, lets you step through areas of code that vendors want to keep out of the reach of user understanding.
90% of established security practices around code delivery are around keeping your product from being too easily reversed or "owned" by those who use it.
Thus the popularity of -as-a-Service business models. You can't invoice for something a user just uses.
Operating Systems that are actually in use show that the real difficulty of Computer Science isn't deeply complex algorithmic complexity (although OSs have algorithms and data structures in spades)
The real problem is just mundane management / organization / interface / documentation that plain old enterprisey software fails on.
A lot of what is lost was handling a much smaller breadth of hardware, applications, and as you pointed out, a much simpler security exposure profile.
A model with unrestricted introspectability wouldn't have worked as a web-replacement anyway, it is not secure enough!
There are even servers like Servant for Haskell that make REST a first-class citizen of the server programming language. So we have the best of both worlds now: security and convenience.
You'd think I was an expert, given that I gave that segment of the talk, but I haven't yet dived that deep into Genera's internals.
As I understand it though, you don't have the concept of page faults at the image level; IIRC, Genera doesn't have any form of virtual memory, or if it does, it's all in microcode. Task switches are, iirc, handled in the image, but in the same way that it's done in Smalltalk: there's a function in the "kernel" to decide what task to switch to next, but the actual context switch is handled in microcode.
Also, there's no such thing as privilege separation; everything's running in a single address space and namespacing is handled using Common Lisp's package system.
None of those actions were visible to the virtual machine that contained the debugger. The CPU didn't even have the concept of an interrupt.
The MIT Lisp Machine ran a simple kernel written in assembler that also included an interpreter for the user-visible instruction set, it would check periodically whether devices had raised an interrupt then return back into Lisp to handle the event.
The linked video seems mostly about UI usage, and one rarely sees nice live action UI demos for machines that (mostly) no longer exist. They should do a sequel that shows these other very powerful system features you mention.
One practical downside is that the live editing / code & continue / source code-free operation (mentioned in the Interlisp-D segment of the video) could lead to needing "just the right image" with all the installed definitions you wanted. Sometimes "source code" might not even exist which could lead to coordination issues.
Sadly the deep dive got cut due to time limitations (both in terms of time during the talk and finding out that all of my alphas were in various states of non-working as I started preparing that segment)
Had I had time, I would have patched the NFS code to run in "permissionless" mode, where it asks what creds are necessary to access a file, then just sends those creds. Hopefully somebody will be inspired to explore and blog about it though :-D
I'd be curious to know how you quantified this. I was curious myself, so I dug into where Smalltalk landed in Stack Overflow's developer survey (which, of course, is not a complete overview of the entire programming industry!) - it seems that Smalltalk was the second most Loved language in 2017 [0], but it completely disappears off the chart in 2018 [1]. I would imagine that we've lost quite a few Smalltalk programmers...
Well, did we? You can still do that! Those environments were extremely primitive and it doesn't take much work to match or vastly exceed their capabilities. This is so much the case we often don't even recognize such environments staring us in the face. Consider another story on the HN front page right now: IntelliJ 2021 is out. Load IntelliJ and you have something that which can be arbitrarily scripted, altered and extended as it runs, and which appears to match/exceed Genera and the other demo'd systems.
Here are some features of IntelliJ that are reminiscent of a single-address-space fully reflective operating system:
• A multi-language scripting console that lets you access arbitrary parts of the IDE and give it interactive commands, either via its real API or (using reflection) anything else:
• An interactive debugger that can debug any app, including the IDE itself. Of course it can be risky to use breakpoints as you might suspend the debugger you're using and get stuck, but that's a conceptual issue, not a technical one.
• An interactive auto-completing command line like thing for commands, files, symbols, menu items etc. Double-shift opens it for me.
• Real-time CPU, memory and task monitors. Try opening the command palette, type CPU and then pick "CPU and memory live charts", or search for "activity monitor" and open it.
• A fully virtualized filesystem layer with sophisticated file explorer. You can explore 'into' files at the structural/symbolic level.
• Obviously it's the JVM so everything is fully garbage collected.
• A whole UI framework that supports both tiling and floating windows.
• The ability to load, unload and upgrade apps (plugins) on the fly without restarts. They can alter the behaviour of the environment and other plugins, either via real APIs or by the big hammer of loading a JVM agent on the fly. JVM agents can rewrite code in any arbitrary way and swap it into the app as it's running.
And you can program it with Lisp if you really want to, via Clojure. I watched the talk, it was a nice talk, but I couldn't help feeling that the answer to "What have we lost?" is actually - cynically - not much. Those environments had some interesting ideas and the good ones seem to have made it into production, just not via custom hardware and operating systems.
Edit: interesting how quick this hit -1. I think a lot of people like the idea of lost treasures in computing. I used to like it too. But I can't identify something these machines or operating systems do that is strictly superior to modern technology or large-scale 'operating environments' like IntelliJ.
IntelliJ idea is, in many ways, more advanced than Genera, yes. I use it every day. But I stand behind my assertion that we've lost something there: Genera had most of the features you listed (even though I didn't have time to show them), and in the case of debugging the debugger, AFAIK, you can't get IntelliJ into a state where the debugger is debugging itself, because in order to attach a debugger to an instance of IDEA, you need to run a second instance.
Adding plugins without restarts in IDEA is a feature that's only a couple of months old now, and even then, making your own plugin is a very involved process. You can't just copy a single function into your own code, modify it, load it, and be done. In genera, that would take 1-5 minutes; quick enough that if something bothers you, you don't really need to think about whether it's worth taking the time to fix it. By the time you've done a cost/benefit analysis, you'd have been done with the fix.
I think you're right that the UI will block you from debugging itself, but, this is just a safety check that could be disabled. It might be a fun project to try and turn IntelliJ into a fully Genera-like environment.
I think as part of that you'd need to teach the debugger how to attach to itself without risk, probably by defining some notion of a protected system thread and then preventing breakpoints from affecting them. The JVM is certainly capable of doing this, although the debugger UI would have to be extended. I guess Genera either didn't have breakpoints or had some equivalent notion of un-breakable operations.
It's true that there's no way to just edit a single function and replace it, at least not out of the box. This is partly conceptual - Symbolics Lisp was pretty much a dynamically typed language if I remember correctly, as are most Lisps, and didn't have any real notion of version control or a compiler checking things for you ahead of time. You wouldn't want to edit only a single function in isolation if editing the internals of IntelliJ because a lot of the value of the tooling is putting the code in context via type checking, static analysis, folding and so on, which requires the rest of the code to be available and ideally visible.
Still, there's nothing missing from the core technology. To make things editable like that you could have a plugin that grabs the matching source zip for a selected symbol or bit of UI and opens it in a new project window, configured in such a way that compiling it causes it to be reloaded into the IDE. Either via the plugin reload mechanism or the bigger hammer of something like DCEVM, which lets you arbitrarily redefine classes at runtime even if their field layout changes. It's more of a workflow issue than anything else. The additional overhead is mostly buying you things you'd want anyway, like a notion of sharing and version control for your changes.
It'd be easier to do it with the Chrome debugger though. I sometimes 'fix' websites that have annoying popups or similar with the dev tools.
With complex apps you have the problem that the owners don't want you modifying them, so they're minified and have other techniques applied that debuggers don't like. But that's not something a Lisp machine would have fixed. You can obfuscate any program.
Also, debuggers aren't the same thing as code editors so of course, you'll lose your changes when the page reloads. Making them persistent means throwing together a browser extension.
Chrome is not like IntelliJ though, because Chrome is a compiled C++ app. It's not written in JavaScript and it isn't a reflective environment like IntelliJ is. With IntelliJ you actually can do things like just open up an editor tab and invoke arbitrary internal functionality. With Chrome, not so much.
Ah, by "crash" you mean if the renderer process crashes, not website JS code.
You can't easily do that with a C++ process indeed, which is why my example was about IntelliJ, not Chrome or any other C++ app. It's much easier to debug and change code on the fly when it's running on a VM like the JVM. I'm not arguing that something like Genera can be matched by any modern app, just that a few operating-system-like apps with similar properties do exist. We just don't think of them as operating systems because they pose as something else. Sort of like how emacs is a Symbolics Lisp-like environment posing as a text editor.
If you're going to argue that it's not the same unless it goes all the way down to the metal, well, people have built Java/JVM based operating systems before like JNode. It's technologically possible. Just not worth it. Being able to edit a function in the middle of your filesystem driver without reloading it is neat in theory but I wouldn't want to actually develop a filesystem that way.
