Is -DWIN31SUPPORT ready yet?

[lss4]

I've been struggling to get Win3.x working on latest FreeDOS CI kernel but no avail.

I thought my configurations were wrong, only to realize that the CI artifacts were not actually built with -DWIN31SUPPORT.

I tried building one myself but still not successful, even using a Ubuntu 20.04 VM with just the necessary stuffs for vanilla.

With -DWIN31SUPPORT (XCPU=86 or XCPU=386 doesn't matter), the build process complained about things like undefined reference to _winInstanced", _winStartupInfo and _winPatchTable.

Does building such variants require a Windows machine? As for now they were only present in build.bat...

EDIT: It seems ALLCFLAGS only affects the C compiler and all nasm jobs are still being done without that flag. Will figure out a way to add it to see if it makes any difference...

[lss4]

After some fiddling I got it working... OpenWatcom only for now, as building with GCC currently fails with "ELF format does not support segment base references" which I suspect is this one.

Need to set -DWIN31SUPPORT for both ALLCFLAGS and NASMFLAGS (the latter is undocumented in config.mak).

[PerditionC]

sorry I've been a bit busy and getting over a cold; yes this is currently only tested with OW (ia16-gcc will be supported if possible), and I am glad you found that you were missing the flags from NASM portions. I will try to update the makefile based build to include the win31 option and time permitting add the missing response for win30. Currently I primarily build under DOS/Windows, but do have a Linux box or two I also test building under. My goal is to have a new release out in time for Halloween (end of October).

[PerditionC]

borrowing my daughter's Chromebook, didn't mean to close

[HnkGitHub]

Also having problems getting Windows 3.1 to work. Applied the defines to an Open Watcom v 1.9 build (Open-Watcom-v2 fails, but it builds FDOS-freecom). Tried running Windows himem.sys instead of FREEDOS' himemx.exe and was able to get windows 3.1 running in standard mode, but not 386 mode. The startup window just freezes after blanking out and never shows the startup logo. At this point, I just gave up. Wonder if there is a particular version of himem.sys that some are using to get their's working? Tried the himem.sys from Windows 3.1.

[lss4]

Also having problems getting Windows 3.1 to work. Applied the defines to an Open Watcom v 1.9 build (Open-Watcom-v2 fails, but it builds FDOS-freecom).

What errors did you encounter when compiling?

I did notice that with most recent source tree, using the most recent Open Watcom 2.0 snapshot would emit some type conversion related warnings similar to this one. Suppressing that warning via -wcd=106 allowed the build to complete and produce the kernel binary, though I haven't tested it yet.

I suspect that was the reason behind 4e23a63. Maybe if you use that tagged version of Open Watcom (or earlier) you will not have that issue while building. I just tested building the most recent source tree using Aug 26 2022 Open Watcom snapshot with -DWIN31SUPPORT and it finished fine.

Tried running Windows himem.sys instead of FREEDOS' himemx.exe and was able to get windows 3.1 running in standard mode, but not 386 mode. The startup window just freezes after blanking out and never shows the startup logo. At this point, I just gave up. Wonder if there is a particular version of himem.sys that some are using to get their's working? Tried the himem.sys from Windows 3.1.

I think Win3.x's HIMEM.SYS is still a good option. It's only XMS 2.0 complaint so you only have 64MB of XMS available which kind of avoids startup issues related to having too much RAM. This at least allowed me to complete the installation to the very last step.

Looks like you've hit the same wall as I did. I recall I pretty much reached as far as you did even with a kernel that had -DWIN31SUPPORT defined... so it's probably not 100% there, not to the point that 386 enhanced mode could be used.

PS: Japheth's HX has a DOSX replacement meant to be used with HDPMI16. I haven't tested, maybe it could improve the experience with Standard mode to some extent.

PPS: I tried both 86 and 386 kernels. I'm not sure about the core differences, but it seems for my actual PC, only the 86 kernel could boot okay. 386 kernel would crash/hang with certain TSRs. With UMBPCI it'll simply hang there, while if using JEMM386, the crash in question would be caught by JEMM so it becomes apparent before the system actually hangs.

[PerditionC]

the freeze during windows logo display sounds like missing adding InDOSPolling=true to system.ini

I am working on the critical section handling so it's not needed but currently without it windows will fail to start in enhanced mode. Also currently only tested without fat32 support and with windows provided memory managers. I recommend 4dos over freecom as well. Each piece needs to work with windows and doing above limits effects to kernel incompatibility not other programs.

[PerditionC]

I need to download a more recent compiler build so that I can fix the type error. My older versions don't have it. Currently I'm tracking down a memory corruption issue with some recent changes (not pushed) that rearrange some stuff to be more compatible and add optional initial GPT partition support.

[lss4]

An update on my building of most recent commit (edd0732).

I'm testing this with a VirtualBox Win3.1 VM configured at 256MB. I'm considering getting the functionality validated first in a VM before actually trying it in an actual system.

Unfortunately the experiments were not successful. I still need MS HIMEM.SYS because Win3.x standard mode (after text mode install) do not run when using HIMEMX.EXE, and I'm having even more issues trying to run MS HIMEM.SYS on FreeDOS.

Using 86 kernel (XCPU=86), Win9x HIMEM.SYS freezes at this line while startup.

Kernel: allocated 40 Diskbuffers = 21280 Bytes in HMA

But Win3.x HIMEM.SYS (which limits to 64MB memory) can be loaded fine. However, the VM would crash/freeze upon finishing initial text mode install of Windows 3.1.

Using 386 kernel (XCPU=386), Win9x HIMEM.SYS freezes without any output, and Win3.x HIMEM.SYS freezes right after displaying information about installing A20 handler and such.

Note that regardless of kernel CPU variant, a cold boot is needed if booted into default FreeDOS configurations (HIMEMX/JEMM386) to ensure HIMEM.SYS would install, due to #90.

I'm not sure if the compiler I'm using might be related, as I'm using latest OpenWatcom snapshot. From one of the commits it seems issues with -DWIN31SUPPORT when building using GCC may have been addressed but I currently don't have a proper build environment for that yet.

[boeckmann]

Thanks for sharing your results on this!

Regarding HIMEMX, I do not think that this by itself imposes a problem with Win 3.1. I have Win 3.1 running fine with HIMEMX and the EDR-DOS kernel in enhanced mode.

There are pre-built GCC based binaries, for example here. You may give them a try if you think it is a compiler related issue.

[lss4]

Thanks for sharing your results on this!

Regarding HIMEMX, I do not think that this by itself imposes a problem with Win 3.1. I have Win 3.1 running fine with HIMEMX and the EDR-DOS kernel in enhanced mode.

I'm not sure about the EDR-DOS kernel. Can it be booted directly via GRUB4DOS just like FDOS kernel? On my actual system I'm already booting FreeDOS this way, loading the FDOS kernel file directly via GRUB4DOS without relying on a boot sector.

Though EDR-DOS kernel being fine with Win3.1 is not necessarily within the scope of this issue... as that's a kernel of its own after all.

There are pre-built GCC based binaries, for example here. You may give them a try if you think it is a compiler related issue.

I know about the CI. It doesn't have -DWIN31SUPPORT enabled.

[ecm-pushbx]

I'm not sure about the EDR-DOS kernel. Can it be booted directly via GRUB4DOS just like FDOS kernel? On my actual system I'm already booting FreeDOS this way, loading the FDOS kernel file directly via GRUB4DOS without relying on a boot sector.

GRUB4DOS isn't very well-behaved, for instance it checks for the short jump and "CONFIG" block signature at the beginning of kernel.sys (and a file size > 16 KiB) to detect a FreeDOS load. My single-file EDR-DOS drload or iniload stages won't be detected by that, and likewise MetaKern will not either. And even when found it seems it doesn't pass the load partition's BPB anywhere to the kernel, as opposed to the original FreeDOS loaders. In fact, BP isn't initialised at all.

It also skips the initial loader of the MS-DOS v7 IO.SYS, which it detects as a file > 64 KiB with an "MZ" signature word at seek 0 and a zero word at seek 510.

The relevant files are grub4dos-0.4.4/stage2/builtins.c and grub4dos-0.4.4/stage2/asm.S

[lss4]

GRUB4DOS isn't very well-behaved, for instance it checks for the short jump and "CONFIG" block signature at the beginning of kernel.sys (and a file size > 16 KiB) to detect a FreeDOS load. My single-file EDR-DOS drload or iniload stages won't be detected by that, and likewise MetaKern will not either. And even when found it seems it doesn't pass the load partition's BPB anywhere to the kernel, as opposed to the original FreeDOS loaders. In fact, BP isn't initialised at all.

It also skips the initial loader of the MS-DOS v7 IO.SYS, which it detects as a file > 64 KiB with an "MZ" signature word at seek 0 and a zero word at seek 510.

The relevant files are grub4dos-0.4.4/stage2/builtins.c and grub4dos-0.4.4/stage2/asm.S

I'm currently using the most recent GRUB4DOS 0.4.6 versions. AFAIK FreeDOS kernel is the only DOS kernel that can be booted directly by it. I did try booting other kernels (MS-DOS, PC-DOS, etc.) directly before and unfortunately none worked, with or without the special arguments.

I don't know if there's a capable bootloader which could correctly load and boot any DOS kernel of choice, that can be chained by GRUB4DOS or perhaps NTLDR/BOOTMGR. GRUB4DOS in its current state is capable enough to reduce a good amount of hassle I'm having with multibooting.

[ecm-pushbx]

I'm currently using the most recent GRUB4DOS 0.4.6 versions.

Where can I find the sources for that version?

AFAIK FreeDOS kernel is the only DOS kernel that can be booted directly by it. I did try booting other kernels (MS-DOS, PC-DOS, etc.) directly before and unfortunately none worked, with or without the special arguments.

Did you try MS-DOS v7 or only earlier versions? According to what I read in the sources v7 and (compressed) v8 kernels should work. What arguments do you mean?

I don't know if there's a capable bootloader which could correctly load and boot any DOS kernel of choice, that can be chained by GRUB4DOS or perhaps NTLDR/BOOTMGR. GRUB4DOS in its current state is capable enough to reduce a good amount of hassle I'm having with multibooting.

You can boot into lDebug (usable as a number of load files, its instsect program can install a loader to a DOS drive or into a chainloadable binary file). Then, load any supported kernel with a boot protocol=PROTO ldp/NAME /NAME command where the partition specification and the filenames are optional. After loading a kernel / a kernel's initial loader, run q to uninstall the debugger and run the kernel. You can build a "menu" using the Script for lDebug file which is loaded by default when you do not pass a kernel command line to the debugger. If you are interested, I can show an example.

The boot commands are described some in https://pushbx.org/ecm/doc/ldebug.htm#cmdboot and https://pushbx.org/ecm/doc/ldebug.htm#helpboot

To load Enhanced DR-DOS (my single-file revisions) you can use boot protocol=edrdos edrdos.com // or boot protocol=freedos edrdos.com (these two commands also work with edrpack.com, edrdos.sys, or edrpack.sys) or boot protocol=ldos edrdos.com (this command only works with edrdos.com or edrpack.com). To load @boeckmann 's Enhanced DR-DOS builds so far, just use boot protocol=edrdos (using the default filenames drbio.sys and drdos.sys).

[lss4]

Where can I find the sources for that version?

Here.

Did you try MS-DOS v7 or only earlier versions? According to what I read in the sources v7 and (compressed) v8 kernels should work. What arguments do you mean?

I did, mainly with MS-DOS 7.1. I recall there's some kind of --msdos or --pcdos argument that can be used for the chainloader command but when I tried back then it did not work (hangs).

You can boot into lDebug (usable as a number of load files, its instsect program can install a loader to a DOS drive or into a chainloadable binary file). Then, load any supported kernel with a boot protocol=PROTO ldp/NAME /NAME command where the partition specification and the filenames are optional. After loading a kernel / a kernel's initial loader, run q to uninstall the debugger and run the kernel. You can build a "menu" using the Script for lDebug file which is loaded by default when you do not pass a kernel command line to the debugger. If you are interested, I can show an example.

The boot commands are described some in https://pushbx.org/ecm/doc/ldebug.htm#cmdboot and https://pushbx.org/ecm/doc/ldebug.htm#helpboot

To load Enhanced DR-DOS (my single-file revisions) you can use boot protocol=edrdos edrdos.com // or boot protocol=freedos edrdos.com (these two commands also work with edrpack.com, edrdos.sys, or edrpack.sys) or boot protocol=ldos edrdos.com (this command only works with edrdos.com or edrpack.com). To load @boeckmann 's Enhanced DR-DOS builds so far, just use boot protocol=edrdos (using the default filenames drbio.sys and drdos.sys).

Thanks for the advice. Will try getting lDebug chainloaded and see how it works.

[ecm-pushbx]

Where can I find the sources for that version?

Here.

I studied the sources some. Again builtins.c and asm.S are the files to check out.

Did you try MS-DOS v7 or only earlier versions? According to what I read in the sources v7 and (compressed) v8 kernels should work. What arguments do you mean?

I did, mainly with MS-DOS 7.1. I recall there's some kind of --msdos or --pcdos argument that can be used for the chainloader command but when I tried back then it did not work (hangs).

From my study of this revision's sources, not 100% sure and didn't verify these claims with testing:

MS-DOS v7 should be autodetected without any switch. --msdos switch is to force the MS-DOS v6 load protocol. MS-DOS v7 kernels cannot be loaded using the MS-DOS v6 load.

In this version of GRUB4DOS's MS-DOS v6 load you have to place the BIO kernel module in the first file that matches the pattern IO.??? in the root directory and the DOS kernel module in the first file that matches MSDOS.???. --pcdos is for a similar protocol, but the filenames to load must match IBMBIO.??? and IBMDOS.???. (Volume labels are not specifically skipped, so don't name your volume "MSDOS" or it could hang the system.) The directory entries need not be the first two entries of the root directory. For both of these load protocols, the data start including hidden sectors (LBA sector number within the unit of the drive's first data cluster's first sector) must be below 10000h because this revision of GRUB4DOS never explicitly sets AX as passed to the initial loader; it happens to be zeroed in asm.S before the control flow branch. The MS-DOS v6 load expects AX:BX to hold this sector number but GRUB4DOS only sets BX to the low word.

Loading lDebug should succeed if you name it IBMBIO.COM and make sure to have it be the first file that matches IBMBIO.??? and pass the --pcdos switch and make sure that the data start value fits as described above. The EDR-DOS load detection in lDebug's iniload should not activate because the stack is passed as SS:SP = 0000h:0400h by GRUB4DOS, well below the cutoff for EDR-DOS load. It will trash the interrupt 1Eh vector though, because GRUB4DOS never sets up the original vector content that is expected by the kernel. To boot lDebug from its own lDOS boot sector loader instead, run instsect.com C: /B=ldebboot.bin /G auto /P none then chainload the created binary file. (May require more switches like /L none to force CHS access if LBA access doesn't work, or /Q none if the query geometry doesn't succeed during boot time.) Note that the lDebug boot sector loader will only emit a single letter and a beep to indicate its failure to load the next stage.

What I also found during my study is that, absent the switches, all files are detected using magic numbers. The source doesn't make it easy to read them as text, but some of them are text. FreeDOS is detected from its short jump followed by CONFIG block signature, and a file size > 16 KiB. All other DOS kernels, except MS-DOS v7/v8, are detected using a bunch of magic numbers. These mostly include the very first instruction in every file. MS-DOS v7/v8 are detected by a file size > 64 KiB and the "MZ" signature at offset 0 as well as a zero word at offset 510. (Which is somewhat lucky, as this word is never zero in my iniload stage.) Unlike MS-DOS v6, the MS-DOS v7/v8 load doesn't depend on the filename and should allow loading from any kernel file. Also unlike the MS-DOS v6 load, this one skips the initial loader of the kernel and directly loads the file starting from seek offset 800h to linear 700h. If an MS-DOS v8 "CM" signature is found, then GRUB4DOS will also act as the depacker of the compressed kernel image.

Finally, a lot of the code like this directory scanning uses a bunch of hardcoded numeric pointers as "variables" of sorts. This is baffling to me because it makes things very difficult to read for no upside I can imagine whatsoever.

[lss4]

MS-DOS v7 should be autodetected without any switch. --msdos switch is to force the MS-DOS v6 load protocol. MS-DOS v7 kernels cannot be loaded using the MS-DOS v6 load.

Okay... just tested booting MS-DOS IO.SYS with GRUB4DOS again and it worked this time. Don't know why it didn't back then... This is on my test VM, and I think I should try booting it from an actual system and see if it works there also...

To boot lDebug from its own lDOS boot sector loader instead, run instsect.com C: /B=ldebboot.bin /G auto /P none then chainload the created binary file. (May require more switches like /L none to force CHS access if LBA access doesn't work, or /Q none if the query geometry doesn't succeed during boot time.) Note that the lDebug boot sector loader will only emit a single letter and a beep to indicate its failure to load the next stage.

Many thanks for the hint. I've been trying to get lDebug booted and so far have been unsuccessful (a single letter and a beep just like you said). Looks like I've indeed created the boot sector file wrong.

I don't know which switch was recently added. I previously used lDebug6 and its INSTSECT complained about a switch not supported. With the just-released lDebug7 the command you mentioned works, and with the created bootsector file I succeeded in getting lDebug booted from GRUB4DOS. Perhaps I'll try loading it from NTLDR also.

Anyway, I've succeeded in booting your single-file edrdos.com through lDebug on my test VM. If Win3.x can be run from EDR-DOS maybe I can test it now that I've got it working. On the other hand, I also managed to get PC-DOS 7.1 (from IBM ServerGuide Toolkit) successfully booted using ibmdos protocol.

[ecm-pushbx]

To boot lDebug from its own lDOS boot sector loader instead, run instsect.com C: /B=ldebboot.bin /G auto /P none then chainload the created binary file. (May require more switches like /L none to force CHS access if LBA access doesn't work, or /Q none if the query geometry doesn't succeed during boot time.) Note that the lDebug boot sector loader will only emit a single letter and a beep to indicate its failure to load the next stage.

Many thanks for the hint. I've been trying to get lDebug booted and so far have been unsuccessful (a single letter and a beep just like you said). Looks like I've indeed created the boot sector file wrong.

Do you recall which letter was displayed? R (sector read error) or F (file not found) perhaps?

I don't know which switch was recently added. I previously used lDebug6 and its INSTSECT complained about a switch not supported. With the just-released lDebug7 the command you mentioned works, and with the created bootsector file I succeeded in getting lDebug booted from GRUB4DOS. Perhaps I'll try loading it from NTLDR also.

The /G switch was added in 2024 February. It isn't necessarily needed (the default /G keep or old instsect behaviour will work) if your boot sector already contains the correct CHS Heads, CHS Sectors, and Hidden sectors. The CHS Heads and CHS Sectors can also be detected using /Q auto (the default for /Q) which should usually work, especially on a HDD. The hidden sectors can be overridden by what the MBR passes with /P auto but GRUB4DOS is unlikely to pass the partition table entry to the chainloaded file like lDOS boot expects. This is why I advised you to use /P none to avoid false positives. (There are two checks for a valid partition table entry used for /P auto but they aren't perfect.)

Anyway, I've succeeded in booting your single-file edrdos.com through lDebug on my test VM. If Win3.x can be run from EDR-DOS maybe I can test it now that I've got it working. On the other hand, I also managed to get PC-DOS 7.1 (from IBM ServerGuide Toolkit) successfully booted using ibmdos protocol.

That's good to know. If you run into any problems using lDebug, let me know. I also have a boot menu of sorts that I prepared a while ago, so do tell if you want to try it.

[lss4]

Do you recall which letter was displayed? R (sector read error) or F (file not found) perhaps?

Originally with lDebug6 on the test VM the letter was I, and later on my actual system with lDebug7 it was R.

It seems the FSIBOOT wasn't needed. Back then I additionally included FSIBOOT (/IB). Not sure if that was the reason of the issues I'm having... The commands back then was like instsect C: /B=boot.bin /IB. I did not use /P nor the newly added /G.

With the following command you mentioned on lDebug7, everything is working on my test VM now. Will try this on my actual system later on.

instsect.com C: /B=ldebboot.bin /G auto /P none

EDIT: Yep. The generated binary works for both GRUB4DOS and NTLDR.

[ecm-pushbx]

Do you recall which letter was displayed? R (sector read error) or F (file not found) perhaps?

Originally with lDebug6 on the test VM the letter was I, and later on my actual system with lDebug7 it was R.

The I indicates FSIBOOT not found, so I assume your VM uses a FAT32 file system as only lDOS boot32 can emit an I. If it is emitted then either the FSIBOOT wasn't installed to the file system's FSINFO sector, or the wrong sector was read. With /P auto (the default for /P) if you get a false positive this could explain the I.

Does your non-VM system not use a FAT32 drive?

It seems the FSIBOOT wasn't needed. Back then I additionally included FSIBOOT (/IB). Not sure if that was the reason of the issues I'm having... The commands back then was like instsect C: /B=boot.bin /IB. I did not use /P nor the newly added /G.

You do need FSIBOOT to run the boot32 loader. However, the loader's first stage expects to find its FSIBOOT in the actual file system FSINFO, not chainloaded behind it. The defaults for /I (/IS /IR /IV) will install to the FSINFO sector, even if you use /B=.

Albeit, if GRUB4DOS chainloads the entire 1024 Bytes file as created with /IB then you can patch the first stage in a small way. This will make it so the first stage finds FSIBOOT only when the entire file is chainloaded and never tries to load it from the FS. Save this in the file patchfsi.sld:

@r ysf |= 4000
a ss:sp - 10
 cmp ax, 20
 jae (sp)
 .
s cs:ip l 200 range ss:sp - 10 aao - 2
if (src == 1) then goto :found
if (src == 0) then goto :notfound
r ysf clr= 4000
; Error: Multiple patch sites found
@goto :eof

:notfound
a ss:sp - 10
 cmp ax, 20
 jmp short (sp)
 .
s cs:ip l 200 range ss:sp - 10 aao - 2
if (src == 1) then goto :alreadypatched
if (src == 0) then goto :notfound2
r ysf clr= 4000
; Error: Multiple already patched patch sites found
@goto :eof

:notfound2
r ysf clr= 4000
; Patch site not found
@goto :eof

:alreadypatched
r ysf clr= 4000
; Patch site found, but is already patched
@goto :eof

:found
a srs:sro
 cmp ax, 20
 db EB
 .
r ysf clr= 4000
; File patched, don't forget to run W command

Then run:

ldebug bootinfo.bin
y patchfsi.sld
w
q

With the following command you mentioned on lDebug7, everything is working on my test VM now. Will try this on my actual system later on.

instsect.com C: /B=ldebboot.bin /G auto /P none

EDIT: Yep. The generated binary works for both GRUB4DOS and NTLDR.

Good to know! If you want, you can try the patchfsi Script for lDebug with /IB too. I believe GRUB4DOS will load the entire file (albeit it can be forced with --load-length=1024). Not sure about NTLDR.

[ecm-pushbx]

Hmm, seems NTLDR will (usually?) load only 512 Bytes: https://forum.osdev.org/viewtopic.php?f=1&t=25883