kvm: native LDT updates

[stsp]

As pointed in #982, this can give some speed-up
to win31. Since dosemu2 is all about speed, we
probably should not miss such possibility.
I think using PML for tracking updates would be
optimal. And #198 have most of the needed code
in.

There is a suggestion to make MAPPING_LOG_DIRTY
generic, but that would probably mean too much work.
Instead we can make it to return failure if not supported,
so that the subsystem can apply some fall-back.

[jschwartzenberg]

I just went over the changelog the other day and noticed the LDT emulation you wrote to support win31. Maybe I should look for more notes that like that :)
This also triggers my question again if there are more differences between QEMU and DOSMEU2 when they are ran on KVM. Are there more bits emulated by DOSEMU2 that QEMU might let KVM handle natively?

[stsp]

Sure thing!
Entire DPMI server is native under qemu.

[jschwartzenberg]

So what about native DPMI through KVM? ;)

[stefand]

Why does Win 3.11 modify the LDT so much? Is that how it implements task switching?

[stsp]

So what about native DPMI through KVM? ;)

hdpmi.exe (or maybe cwsdpmi.exe + https://github.com/stsp/pmdapi).

Why does Win 3.11 modify the LDT so much? Is that how it implements task switching?

I think so, but this is only a guess.
At least dpmi-1.0 requires per-client LDT/IDT.
I think they initially wanted to make every
(then only planned) win32 app a separate
DPMI client and off-load the task-switching
(in win95) to the DPMI server, but ended up
with dpmi-0.9 + PM DOS API + vwin32.vxd.

[stsp]

There is a winos2 kernel that has the
only difference, which is, it doesn't write
to LDT directly. Could you please try it
to compare the performance?
There are 2 possibilities:

• Either is does the same thing, just with
  the casual DPMI API. In that case it will
  likely be slightly faster.
• It completely avoids the excessive LDT
  updates. In that case we'll be able to see
  the speed we can aim for with the native
  LDT access.

[stsp]

OS2WIN31.ZIP
winos2 package.

[bartoldeman]

You can certainly give Windows direct access to the LDT with KVM. I am just not sure then where the dirty log comes in. Do you want it to use it to know that a specific page of the LDT has been written to and then sanitize it from garbage sometime after?

As for native DPMI under KVM, this should be possible if dosemu implements VCPI. I actually wrote a draft patch for that some time ago, but to do it properly I first need to finish my IDT work, namely handling IRQs properly by injecting the actual hardware interrupts into the VM, instead of the workaround we have in place now.

[stsp]

Do you want it to use it to know that a specific page of the LDT has been
written to and then sanitize it from garbage sometime after?

Windows not only writes garbage to LDT to
"unuse" entries. It also writes valid descriptors
there, so you need to sync it up on any KVM
exit. Of course you can do it differently, for
example you can remove one of 2 LDT buffers
and turn Segments[] array into a function for
dynamic access, but there are 2 points to consider:

1. The 2 LDT buffers are not always equal
2. There may be pre-existing LDT entries (from
   old glibc for instance)

Which is why I thought PML is the simplest solution.

As for native DPMI under KVM, this should be possible if dosemu implements VCPI.

... and someone explicitly disables DPMI in config?
Or just steal int0x31 from it?
Well, that all sounds crazy, and the speed-up will
likely not worth the troubles, but I suppose VCPI is
good to have by itself.

[stsp]

Mm, point 2 does not stand, as with KVM we don't
need to touch the real LDT ever. In which case only
syncing up Segments[] is needed, and the accessor
function may quite do the trick.

[jschwartzenberg]

With VCPI mentioned I will also ask the next question. Could that not be handled natively? If I understand correctly this would entail being able to support ring 0 execution via KVM...

[stsp]

Could that not be handled natively?

No, only natively.

[jschwartzenberg]

But ring 0 does work in QEMU? Is there some documentation on the difference with DOSEMU2 there?

[bartoldeman]

DOSEMU2 could in principle support ring0 but it's presently not set up for that, it would be a fair amount of work at least. There are many places that assume ring3 or vm86, I just don't know how bad it is.

When I first added KVM it was booting in real mode which confused some applications (with the CPU EFLAGS register) and made it impossible to trap software INTs. Some DPMI applications when run in real mode assume that you can enter protected mode manually (using neither dosemu's DPMI nor trying to use VCPI) and that then crashes.

[stsp]

But ring 0 does work in QEMU? Is there some documentation on the difference
with DOSEMU2 there?

dosemu2 does not have tcg to emulate
ring-0, and neither does simx86.
So VCPI can only work with most things
(like paging) done natively. And yet, as
Bart pointed, some things should still be
emulated at the same time, so that would
be a nuclear mix.
Given that only a couple of apps need that
(and they can work in dosbox), I remain
sceptical. There are only 2 problematic
software pieces: jemm and run286.
jemm is fully decompiled (myJemm), and
from its sources it is very clear what to do.

[stsp]

And dosbox have reverse-engineered run286,
wow. So we can have it w/o VCPI.

[jschwartzenberg]

Given that only a couple of apps need that
(and they can work in dosbox), I remain
sceptical. There are only 2 problematic
software pieces: jemm and run286.

And Windows 9x I guess.

[stsp]

Oh, I wouldn't bet on that.
And even if it would be possible to
run win95 that way, there may be
problems in providing it with the host
FS, network, mouse and all that.
But let's see, maybe there is more
magic to expect than I can imagine. :)

[jschwartzenberg]

Oh, I wouldn't bet on that.
And even if it would be possible to
run win95 that way, there may be
problems in providing it with the host
FS, network, mouse and all that.

Windows 9x can use any filesystem that works in DOS. There will be a warning somewhere "Drive C: does not use 32-bit access" or something like that, but it works fine. This was used by SCO Merge/Netraverse Win4Lin.

As for hardware, I would say the existing hardware emulation takes care of it. (Side note: even many Windows 3.1 drivers can be installed in Windows 9x. See for instance the second screenshot here: http://toastytech.com/guis/misc2.html)

But let's see, maybe there is more
magic to expect than I can imagine. :)

I think the ring 0 usage of some bits is really the biggest problem. Win4Lin includes a set of patches that it applies on some Windows system files, probably to take care of (some of) this.

[stsp]

Windows 9x can use any filesystem that works in DOS.

Does local fs work in win95 under dosbox?

[stsp]

This was used by SCO Merge/Netraverse Win4Lin.

Who told you exactly that win4lin allowed a ring-0 access?

[jschwartzenberg]

Windows 9x can use any filesystem that works in DOS.

Does local fs work in win95 under dosbox?

The limitation that dosbox has is that it doesn't have any boot loader code to load an external DOS. If you want to use any DOS other than the built-in DOS, you have to use an hdimage with a proper boot sector that can load the DOS that's on it. Using a hdimage in dosbox disables its local fs support even in plain DOS.

This was used by SCO Merge/Netraverse Win4Lin.

Who told you exactly that win4lin allowed a ring-0 access?

I have no idea whether win4lin allowed ring-0 access and I don't think it did. I know they patched out some calls which I guess were ring-0 and they also used some memory relocation tricks. If you want I can upload the patch files (they're hex patches I think). Also the changes they made to the Linux kernel and their kernel module contain a lot of comments on how everything worked. I can upload this as well.

(One of the original authors of Merge/Win4Lin was Gerald J. Popek. He was one of the authors of a paper that laid out the requirements that CPUs needed to fulfill to enable efficient virtualization. These were implemented by Intel/AMD in VT-x/AMD-V.)

[stsp]

I have no idea whether win4lin allowed ring-0 access and I don't think it did.

And we are talking about VCPI, that does.

[jschwartzenberg]

I have no idea whether win4lin allowed ring-0 access and I don't think it did.

And we are talking about VCPI, that does.

It seems you are correct. I just searched through the Win4Lin patches and they do not refer to ring-0 (or any ring) at all!

[stsp]

What scares me in that whole idea is
the desire of making dosemu2 a kvm-only
emulator. And if win95 is to work reasonably
well with VCPI, then it will became kvm-only.
Maybe its a destiny of all emulators, but at
least its an unportable destiny. Doing kvm-specific
speed-ups is one thing, but turning entire emulator
into kvm-only is another. And that scares me
quite a lot. :) Because the real kvm-only emulator
should look entirely differently - simpler to
port mfs.c+fdpp to qemu and call it dosemu3,
rather than starting that out of dosemu2.
Just a thought.

[jschwartzenberg]

Your thoughts make sense. My proposals were also intended to trigger a thinking process.

One point I think that's relevant is that win95 is not running stably in QEMU with KVM. Maybe dosemu2 with VCPI would let win95 be more stable than any other way of virtualizing it. Win4Lin was also known to be more stable than pure win95.

I think a better first goal would be an alternative for the vxd emulation and possibly win32s support.

[stefand]

I gave the os2win files a try but could not find a performance difference in winbench compared to the originals.

[stsp]

There is something even more
interesting. The doc mentions winme,
and in the code I see they return an
entry to vwin32.vxd. So at least there
was some attempt to run win9x.
Can you check under qemu how it
works? Can you just run
system32\krnl386.exe
and get win95 running?

[jschwartzenberg]

I could try, but I'll have to find a Windows 95 installed in QEMU :)

Btw. you have time to check KVM on my notebook?

[jschwartzenberg]

No that doesn't seem to work:

[stsp]

Thanks. :(

[jschwartzenberg]

It seems Windows 9x has bugs that are triggered on newer CPUs:
https://www.virtualbox.org/ticket/19275#comment:8

Maybe Windows 3.x also has issues like this?

[stsp]

That ticket refers to the vmware site, but
the URL is broken. So not much to gain.
Anyway, whatever the bugs are, we are
not affected, as we entirely skip the windows
ring0 kernel.

[jschwartzenberg]

Which URL doesn't work for you? These both work for me and seem to detail the problem:
https://communities.vmware.com/thread/579537
http://blog.stuffedcow.net/2015/08/win9x-tlb-invalidation-bug/

If skipping the Windows ring0 kernel means that these bugs are avoided, I would say that's quite a big feature to boast about!!!

Furthermore could this be a reason why Windows 9x appears to run more stable in Merge/Win4Lin? Except for VMware and maybe Parallels, it seems other hypervisors gave up on virtualizing non-NT Windows.

[stsp]

Thanks, the second link is good.
From vmware site I was getting
"site is down for maintenance"
yesterday, not sure why, but there
is nothing interesting anyway.
What surprises me, is that it seems
your second url is the only url in
the whole www that details the problem.
If the problem is that simple, why it
is not analized on many sites, why no
one released a binary patch for win98?
Quite strange.

[jschwartzenberg]

The group of people with interest seems relatively small and not so knowledgeable, otherwise Windows 98 would also be in a better state on KVM+QEMU. I'm also surprised about the interest in DOSEMU2 compared to DOSBox. I think it's a matter of power user versus simple user.

Maybe you could create a binary patch for this ;)

[stsp]

otherwise Windows 98 would also be in a better state on KVM+QEMU.

If its not possible w/o patching windows, then not.
KVM will never apply for the run-time patching of
known patterns, as some commercial projects do.

I'm also surprised about the interest in DOSEMU2 compared to DOSBox.

Could you please share your observations?
I think compared to dosbox, its negligible.

I think it's a matter of power user versus simple user.

I want dosemu2 to be friendly to any user.

Maybe you could create a binary patch for this ;)

No, I can't even test it on dosemu2. :)
And if we manage to "skip the kernel",
it will never be needed.

[jschwartzenberg]

otherwise Windows 98 would also be in a better state on KVM+QEMU.

If its not possible w/o patching windows, then not.
KVM will never apply for the run-time patching of
known patterns, as some commercial projects do.

The sources of older VirtualBox versions should have the ring 0->1 patching for win9x.
I guess running win9x inside Merge/Win4Lin inside KVM+QEMU would really be more stable.

I'm also surprised about the interest in DOSEMU2 compared to DOSBox.

Could you please share your observations?
I think compared to dosbox, its negligible.

DOSBox actively discourages any non-gaming use/features and caters even to people that have never used DOS as their main OS but who are interested in older games.
Users that are aware of DOSEMU2 and the technical differences between it and DOSBox, tend to prefer DOSEMU2.

I think it's a matter of power user versus simple user.

I want dosemu2 to be friendly to any user.

I know, but today this is the divider.

Maybe you could create a binary patch for this ;)

No, I can't even test it on dosemu2. :)

:D

And if we manage to "skip the kernel",
it will never be needed.

That would be amazing yeah! It seems nobody else is thinking of this.

[stsp]

The sources of older VirtualBox versions should have the ring 0->1 patching for win9x.

Yes, but KVM is not.

DOSBox actively discourages any non-gaming use/features

... and has lots of forks that do not do this
(like dosbox-X). Competing with dosbox w/o
windows and android ports is pretty much
impossible.

Users that are aware of DOSEMU2 and the technical differences
between it and DOSBox, tend to prefer DOSEMU2.

If you have particular URLs in mind, please share.

I know, but today this is the divider.

You mean its still user-unfriendly?
What's missing?

That would be amazing yeah! It seems nobody else is thinking of this.

Me too, don't hold your breathe.
It would require getting IDA and doing
some serious reverse-engineering.

[jschwartzenberg]

Users that are aware of DOSEMU2 and the technical differences
between it and DOSBox, tend to prefer DOSEMU2.

If you have particular URLs in mind, please share.

You can search on Reddit for instance and see the comments on the project.

I know, but today this is the divider.

You mean its still user-unfriendly?

Not at all!

What's missing?

Fame! Once there is a release showing up on Slashdot and similar sites, that will change.

That would be amazing yeah! It seems nobody else is thinking of this.

Me too, don't hold your breathe.
It would require getting IDA and doing
some serious reverse-engineering.

I think someone who did it already for win31 knows :D

[stsp]

Fame! Once there is a release showing up on Slashdot and similar sites, that will change.

Not sure.
But we'll see, as soon as there is something to show.
So far - no even rpm build deployed. :)

I think someone who did it already for win31 knows :D

That was basically me and Japheth, and I am
not sure if either of us knows how to run win95
the same way. :)

[jschwartzenberg]

And DPMIONE, but its author doesn't seem interested anymore.

Possibly Merge/Win4Lin as well... it seemed to have some debugging hooks. I also thought it might be possible to port the Merge Kernel Interface adapter to KVM. That would allow using their binaries again.

[stsp]

And DPMIONE, but its author doesn't seem interested anymore.

Even qdpmi can run win31, but yes,
DPMIONE seems to be the only DPMI
server that really tried to run win95
(likely unsuccessfully).

[jschwartzenberg]

It seems it became very hard to find the Win4Lin Linux patches and binaries online, so I'll provide them here:
kernel-feat-core-win4lin-2004.12.11-alt2.M30.1.src.rpm.gz
Kernel-Win4Lin3-2.6.12.6.patch.gz
Kernel-Win4Lin3-2.6.14.3.patch.gz
mki-adapter26_1_3_13.patch.gz
Kernel-Win4Lin3-2.6.18.patch.gz
mki-adapter26.patch.gz
Win4Lin-5.5.20e-s.i386.rpm.gz

A working license can be found here:
https://www.suares.com/index.php?page_id=25&news_id=215

[jschwartzenberg]

A book which has some info on vwin32: http://tkiel.free.fr/doc/Windows%2095%20Programming%20Secrets.pdf

[jschwartzenberg]

Photocopied versions of Unauthorized Windows 95 to be viewed with an account:
https://archive.org/details/unauthorizedwind00schu
https://archive.org/details/unauthorizedwind00schu_1

[stsp]

While Schulman's book is not a
secret, and have a minor use here
(doesn't include too many low-level
details of their reverse-engineering
work), the other book you've found,
is indeed, quite interesting.

Likewise, many places in the VxD code keep pointers to KERNEL32 variables or, even worse, KRNL386 variables.
The worst offender here maybe VWIN32.VXD,

Not a very encouraging description
for starting to implement that. :)

[stsp]

VWIN32.450 As you can tell from looking at this table, VWIN32 provides numerous Win32 VxD services -- so
many, in fact, that I could probably write an entire book on them. Another book, though; not this one. In this book, I'd like to
focus on just three of the services listed in the previous table

Wow!
This book is full of "promises". :)

[stsp]

Please find the "another book" mentioned
above. :)

[jschwartzenberg]

A short article by the same Matt Pietrek: https://web.archive.org/web/20070312044227/http://www.microsoft.com/msj/archive/S2CE.aspx
His site lists the articles and columns he wrote, they probably could be similarly retrieved through The Internet Archive: http://www.wheaty.net/
Maybe there are here too: https://blogs.msdn.com/matt_pietrek/

A book on Windows 95 file system stuff (non-trivial to access): https://www.scribd.com/document/383387667/Inside-the-Windows95-File-System
A better source for probably the same book (I could get a copy here): https://pdfcoffee.com/inside-the-windows95-file-system-3-pdf-free.html

Some older books from Pietrek and Schulman:
https://archive.org/details/windowsinternals00piet
https://archive.org/details/undocumentedwind00schu

Wine also has a reference to Pietrek's book:
https://github.com/wine-mirror/wine/blob/master/dlls/vwin32.vxd/vwin32.c

I don't see any other related books listed for PIetrek, so I don't think he worked out the other book :)

[stsp]

Well, then we should @ Matt Pietrek
here on github, and convince him to
work on dosemu2. :)

[jschwartzenberg]

Another book, probably more general knowledge: https://archive.org/details/systemsprogrammi00oney

[jschwartzenberg]

Pietrek's book in The Internet Archive: https://archive.org/details/windows95systemp00matt
It has about 70 more pages than the PDF version above for some reason.

It's also possible to search inside multiple books at the same time: https://archive.org/search.php?query=vwin32&sin=TXT

[jschwartzenberg]

Two more more on the system:
https://archive.org/details/insidewindows9500king
http://index-of.es/Magazines/hakin9/Windows%20System%20Programming%20(4th%20Edition).pdf

[dionorgua]

@jschwartzenberg Do you have working win4lin setup? it's pretty hard to find it nowadays. It looks like you've shared 'server' version, but publically available keys are for 'desktop'.

EDIT: I've found a way to register it. A few notes if somebody will try to setup it:

• patch for kernel 2.6.18 doesn't work (applies cleanly to Redhat 194.17.4.el5 kernel). But anyway Win4lin VM crashes randomly
• 2.6.14 is probably latest working kernel
• It's pretty hard to boot 2.6.14 kernel. It's much easier to use old distributions like Debian Sarge (good luck for setting up screen resolution in VM)

[jschwartzenberg]

You can run Win4Lin inside a QEMU/KVM VM. The easiest way is to take a distribution that has its default kernel already patched for Win4Lin such as LindowsOS/Linspire or Xandros.