I mean, it definitely wasn't a virtual machine in the strictest sense, because it does not run on any virtual hardware. I think I may need to read up on this some more, though, because that both kind of reminds of chrooting and kind of not. Got any resources at hand?
It's essentially a bit of terminology drift, as the core concept of a
virtual machine is about virtualization of resources, exposing a "virtual machine" to the end-user that it can treat as if it was the one and only. One of the early examples of that was process separation, where each individual process has its own
virtual environment and doesn't see or interact with other code except through explicit APIs. With the introduction of IBM VM (with its companion, IBM CP), an idea of virtualizing enough of the environment that you could run unmodified software that expected
raw hardware was introduced, and that's what most people recognize as
virtual machine these days. But the concept spans everything that abstracts the
physical reality with a virtual one, including even components of runtimes of natively compiled programming languages (as they provide common services that abstract/virtualize the external world).
That was a big change for the majority of Windows users with Windows 95, as previously all windows applications were essentially loaded into memory without any separation, linked with few DLLs (KERNEL.DLL most importantly), and had to cooperate by calling specific routines in KERNEL.DLL that switched to the next application. With Windows 95 the preemptive VM model of processes that already was present in other Microsoft OSes (Xenix, WinNT) was introduced to the wider user base, though a stripped-down variant was provided as an extension for Windows 3.11 in the form of
Win32s.
Generally, Operating System textbooks are a good place to start looking, as well as
older computer science literature that focused on areas of operating systems. It used to be more explained in older books, these days it often feels that the actual concepts behind
processes, threads etc are bypassed unless one takes an operating-system focused course