Cannot boot from compiled FAT12 floppy image.

[xrayer]

Hi, I successfully compiled ELKS from current sources. When I select in kernel config floppy image format MINIX then image boots OK in VMware but when I use FAT format then image doesn't boot - it prints the image is not bootable and I cannot find kernel on FAT FS. Doesn it mean that bootable FAT is not supported yet? Then should be added some info in menuconfig help. It would be possible to adopt e.g. FreeDOS opensource bootsector and modify it to load ELKS kernel instead of FreeDOS kernel. Or is there some ELKS loader that works from DOS and run on 8086?

[mfld-fr]

Hello @xrayer

ELKS boot from FAT is not supported yet, only from MINIX v1.

FAT support was introduced in 2017 and is still in progress (see https://github.com/jbruchon/elks/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aclosed+fat).

One problem is that the root file system requires symbolic links and device files that FAT does not support, so even if booting & loading the kernel from FAT, one needs a POSIX compliant files system to be mounted as root.

[xrayer]

OK, well it's experimental.
I also found a script build_fat32_hd_image.sh in the package that seems it should build FAT32 bootable disk image that is run via QEMU - what bootsector and loader it use? So as it can boot from FAT32 is should not be so complicated to boot from FAT12 floppy.
I try to understand how ELKS boots, it seems to use 2-stage loading, am I right? 1st is boot sector, then setup and then kernel. But on minix floppy image I found only "linux" file in root. Has it joinded with setup together or is the setup placed in some hidden sectors?

And the problem with symlinks - I can see there are only 2 - sh and ftpget - can be solved by duplicating this files on FAT - no serious problem.
Maybe could I get a direct contact to Zouys user who developed the FAT support (as noted in fat_fs.txt) to ask him directly?

[mfld-fr]

I don't think the build_fat32_hd_image.sh script builds a bootable image. I remember it was used by @georgp24 to test FAT in a secondary & data-only disk. It is kept as a reminder to complete the FAT image generation in the configuration scripts.

When booting from MINIX, the first 1K block contains 2 sectors : first is the actual boot sector, second is the minimum code to locate a 'linux' file in the file system and to load it into memory (see files in https://github.com/jbruchon/elks/tree/master/elkscmd/bootblocks).

The 'linux' file is the generated 'Image' in /elks/arch/i86, that concatenates 3 things : the raw boot sector, the 'setup' program and the 'system' (= the kernel) program. The raw boot sector is useless for MINIX except some flags that should be moved elsewhere to allow removing it and make the things cleaner.

After loading 'Image' into memory, the MINIX boot loader then branches to the 'setup' program, that in turn branches to the 'system' one, after performing some hardware detection and relocation.

You are right about the symbolic links that are few and could be workarounded, but what about the devices files in /dev ? ELKS is still on the old scheme where this folder is manually populated with files with special attributes that FAT cannot handle, and we don't have yet something similar to the 'devfs' as in modern Linux.

Anyway, I agree that we should be able to boot from FAT, and I think we should do the same as in the live medium of many Linux distributions : a FAT boot loader (as the MINIX one), and the kernel + the root filesystem in files on that FAT volume (need to develop a file loop driver).

Edit : not @ghaerr, sorry, but @georgp24

[georgp24]

Yes, I did not write a script to boot from FAT. I added FAT support to support the exchange of data with foreign systems.
You could configure syslinux to boot the elks kernel I guess.

[xrayer]

How can I exactly use syslinux to boot ELKS? I tried version 3.61 (as the latest installer ver 6.04 told me not support FAT12-wtf?) put elks linux on that floppy image but what mode use in syslinux.cfg? I tried linux, kernel, bss but in best case I got "ELKS Boot" message and it stop.
BTW does syslinux run on 8086 CPU?
Then I tried GRUB4DOS but not much better, I got some 2 boot lines and then chinese tea and stop.
Then I tried freedos bootable floppy and simply replace kernel.sys with elks linux - no boot message just stop.
Last I tried to directly put linux to floppy image on sector 0 (use embedded raw bootsector) and it booted (I was surprised, not so useless) and asked for root system floppy. So I replaced floppy image in BOCHS with my FAT12 elks non-bootable image and hit enter but it didn't mount it and ended in some endless loop printoing some sector number...

What all is needed to be prepared for loading elks linux image?
from minix_first.S I can see that
load segment is 0x0100, offset to jump is 512
and AX=DS=ES=SS=0x0100, SP=0x4000-12

And abut the /dev - on FAT12 image I can see some /dev/makedev script so I expected it will create dev tree in RAM at runtime - no need to have files permanently on FS.
But it would be possible to ressurect the old UMSDOS filesystem used on ancient linux distros. It used FAT to store linux files and simply every linux directory contained some special file with extended attributes of stored files that was beyond FAT and also probalby handled symlinks.

[mfld-fr]

I am not aware that any third-party boot loader like 'syslinux' has been successfully tested with ELKS, so I think you are exploring a fresh & unknown area here...

I never said the raw boot sector at the beginning of the 'Image' file is useless at all. I said it is useless in the case of booting from a MINIX volume. Placing the 'Image' file starting at sector 0 is the 'RAW' option for the image format in the configuration, and it works well until the moment where a root filesystem is needed.

The solution to store the special attributes into a special file in the same directory looks interesting... any contribution to implement it is welcome.

On my roadmap, I would prefer to complete the ROMFS with a loopback file driver. So that I can put a boot loader on the FAT volume, the 'setup + system' as a first file, and the root filesystem embedded in a second file.

[xrayer]

I had a closer look at FreeDOS bootsector and found that it works different way than ELKS needs so it had to fail to load. FreeDOS bootsector (I investigated OEM version) doesn't relocate its own code and it load only 29kB of kernel file at 0070:0000 so it cannot load whole ELKS linux image into memory without overwriting itselsf.
So I have another idea - if there is enough free space in RAW bootsector (i86/boot/bootsect.S) it should be possible to add FAT12 data structures to make it a regulat FAT12 bootsector. There is data field that specifies reserved sectors - between boot sector and start of FAT structure and rootdir. This reserved space is usually 0B on normal DOS floppies but it can be used to hide whole ELKS linux image. So it could be bootable same as if linux image is put by dd on a floppy but the rest of it should be readable under DOS and used for root FS. If there is not enough free space in RAW sector then it would need to be simplified and reduced .

I'm not familiar with ELKS codebase I'm just curious to try it out on one old PC XT MB. I don't know how much it can share with regular linux kernel sources. I think that UMSDOS FS is still supported in current linux kernel no idea what effort it would take to port it in ELKS.

[georgp24]

Here is a boot sector which can load DOS com or exe programs. Maybe that can be adapted to load the ELKS kernel.
http://alexfru.chat.ru/epm.html#bootprog

[xrayer]

I had a closer look to ELKS boot sector (raw and minix) and see that it works differently than all other bootsectors usually do. It doesn't load whole linux image at one place but it split it and load to different memory locations. RAW bootsector goes to 0100:0000, setup goes to 0100:0200 and the kernel (system) goes to 1000:0000 so I cannot wonder that all loaders failed because they loaded image at one place. I tried to do a quick workaroud - I splitted linux image to bootsect+setup and kernel and inserted a stuffing block that shifted the kernel to right location 1000:0000 when such modified image will be loaded at 0100:0000. But it still didn't booted. I'm not sure if I have registers setup correctly before JMP to setup code (AX=DS=ES=SS=0x0100, SP=0x4000-12)...

[ghaerr]

You are right about the symbolic links that are few and could be workarounded, but what about the devices files in /dev ? ELKS is still on the old scheme where this folder is manually populated with files with special attributes that FAT cannot handle, and we don't have yet something similar to the 'devfs' as in modern Linux.

@mfld-fr: Your 'devfs' idea above could be a good idea for solving the "mfs" and "mkromfs" issues I brought up in my last post. One wonders whether it is worth having to create a full /dev directory at image-building time especially if we're thinking of supporting FAT. However, given that programs might create /dev entries at runtime, a full implementation would likely be necessary, and just use up more RAM, when this problem might be more easily solved using another mechanism.

[ghaerr]

Anyway, I agree that we should be able to boot from FAT, and I think we should do the same as in the live CD / DVD of many Linux distributions : a FAT boot loader (as the MINIX one), and the kernel + the root filesystem in files on that FAT volume (need to develop a file loop driver).

Another thought on supporting FAT boot: Since ELKS boot doesn't require symlinks but does require block/character special files, the FAT image could be built using the exact same mechanism that my (unmerged) "mfs" utility uses, but using the MSDOS "mtools" mwrite etc. utilities instead, with one modification. That is, use the mtools user-mode programs to format and write an MSDOS image without requiring kernel standard MSDOS filesystem support. The mwrite utility could be modified, along with the ELKS FAT fs driver, to use an unused FAT directory or file bit that would indicate the file is a character or block special file, and act accordingly.

Admittedly this isn't "MSDOS" portable, but neither is the ELKS boot disk really, and only those files in the C:\DEV directory (or the directory itself) would be handled specially.

Should there be support for adding the "mfs" image-building mechanism into ELKS finally, I would be willing to write the modifications to mwrite as well as the ELKS FAT filesystem driver to support this special interpretation and enable ELKS FAT bootable filesystems with a minimum of extra RAM requirements and development effort.

[mfld-fr]

@ghaerr : well... that could work, but I don't feel at ease having any special handling in FAT filesystem and I would prefer keeping the interoperability with it, as it is the de facto standard for data exchange in small systems.

Plus the thinking that implementing a loopback driver would not cost as much as you stated and keeps the things simple & consistent with what is done on many distros to boot from a FAT volume.

[mfld-fr]

@ghaerr : we also have the more simple option to add the 'initial filesystem / disk' (= 'initrd') at the end of the kernel image, mount it as the root, and mount the FAT as a secondary volume...

[xrayer]

And does ELKS support initrd? But I think that whole initrd will take much more ram than only a /dev in RAM...

[xrayer]

@georgp24 : I had a look on bootprog (Alex has there also some other interesting programs) but it doesn't give me any advantege over FreeDOS bootsector. as I already checked, there are 2 versions of FD bootsect, one that studied previously was OEM limited to 29kB loded binary and the normal FD boot can load about 140kB kernel and relocate itself so I aimed to this version but still cannot boot even I'm quite sure that I'm loading 2 parts of ELKS image to right address now...

[ghaerr]

I would like to comment further on getting ELKS booting on MSDOS volumes and ask a question:

Aside from the lack of symlinks and special file support on FAT, what else currently needs to be written? Do we require a modified boot loader that reads "linux" from a FAT filesystem rather than MINIX v1? What else? Has anyone got Linux loaded from FAT?

Comments:

1. I have looked further in to quickly moving forward on device file support on FAT12/16/32. In reading the FAT design, there are a two unused shorts in every directory entry, as well as two unused attribute bits (see Section 4 in http://www.disc.ua.es/~gil/FAT12Description.pdf). Unused attribute bit 0x40 (already used as DEVICE in FAT32) could be used to specify a special file, and directory entry offsets 12 and 20 could be used to specify the device type and major and minor unit number. This could be done in ELKS and maintain perfect compatibility with any MSDOS system or disk reader, as these attribute and directory bytes are not modified and preserved. A C:\DEV directory could be built using the mwrite utility at image generation time (see below).

2. Has anyone looked at the CONFIG_UMSDOS_FS support already built in to the ELKS msdos filesystem driver?? See elks/elks/fs/msdos/inode.c. Setting this option (not currently in our ELKS menu setup system) builds in kernel support for /dev/bda, bda1, fd0, fd1, mem, kmem, tty, etc. While not allowing the ability to write /dev at runtime, this option already supports what should be needed to boot ELKS from FAT images with regards to providing hard-coded kernel support. Why not use it in the interim?

3. Now that the ability to build ELKS MINIX images using userland utilities (mfs) rather than Linux filesystem support is imminent, I was planning on adding the ability to create (other than the special files, discussed above), MSDOS images using the mformat and mwrite utilities whose source is already part of the ELKS tree. All that needs to be done is to compile them up for the build host as well, and then use them in a new image/Makefile::dosfs target to read the ELKS applications from TEMPLATE_DIR and copy them to DESTDIR. Some small mods to write a FAT bootblock (and possibly set special device file bits as described above) and we will have the ability to create arbitrary bootable FAT images on any OS. This will eliminate the need for the build_fat32_hd_image.sh scripts discussed earlier and associated loopback driver. Also, any modifications to the mformat and mwrite commands will also run on ELKS, which means ELKS would be able to create its own disk images from itself :)

4. We might start considering the need for specifying the applications that will be put on a disk separately from the MINIX/FAT etc disk format. We could then develop a scheme to more precisely specify application sets which could then be placed on varied formats easily.

Thoughts? I am willing to write code for the filesystem generation, and others might modify the boot loader, what else is required?

[georgp24]

In the bootprog the loader reads the first sector of the file it shall load, looks at the header and then loads the file according to the file type it determined. This way it can handle different types of executable files.

[mfld-fr]

Do we require a modified boot loader that reads "linux" from a FAT filesystem rather than MINIX v1?

Yes.

Has anyone got Linux loaded from FAT?

Linux or ELKS ?

Has anyone looked at the CONFIG_UMSDOS_FS support already built in to the ELKS msdos filesystem driver??

No, UMSDOS was suggested for the first time in this discussion.

While not allowing the ability to write /dev at runtime, this option already supports what should be needed to boot ELKS from FAT images with regards to providing hard-coded kernel support. Why not use it in the interim?

Because the assumption that the root filesystem is RW is not always true.

We might start considering the need for specifying the applications that will be put on a disk separately from the MINIX/FAT etc disk format.

See #290

[xrayer]

@georgp24 : Yes but it's not needed to handle multiple file types, only need to load 2 different part of image to 2 different memory location. With dummy stuffing between 2 sections also freedos bootsector can do it. But now I need to know, how the registers have to be exactly set before jump to setup part. I'm not sure if I was right (posted above) and it maybe a cause why it's not booting. Is there still active someone who wrote the ELKS bootsector or have this knowledges?

[mfld-fr]

@xrayer : I wrote the last version of the ELKS boot sector for MINIX. The 'setup' part requires no register setting on entry (see https://github.com/Embeddable-Linux-Kernel-Subset/elks/blob/master/elks/arch/i86/boot/setup.S).

[ghaerr]

Has anyone got ELKS loaded from FAT?

Has anyone looked at the CONFIG_UMSDOS_FS support already built in to the ELKS msdos filesystem driver??

No, UMSDOS was suggested for the first time in this discussion.

While not allowing the ability to write /dev at runtime, this option already supports what should be needed to boot ELKS from FAT images with regards to providing hard-coded kernel support. Why not use it in the interim?

Because the assumption that the root filesystem is RW is not always true.

You may have misunderstood my comment. At this point, I believe that the CONFIG_UMSDOS_FS option (currently not implemented in .config, but implemented in the source code) is the best option for proceeding with getting ELKS booting from FAT volumes (if there is in fact a desire to make this happen). The code already implements special /dev file handling of a preset list of /dev files (see elks/fs/msdos/inode.c for details) and will work on root filesystems mounted RO or RW. My earlier suggestion, made before finding this implementation already in the elks codebase, requires more work and uses unused fields in the FAT directory entry (even though many operating systems use these fields for their own data as well).

@mfld-fr: Are you interested in looking into boot loader modifications to find /linux on FAT, or should I, provided there is interest in this topic?

[xrayer]

@mfld-fr
Hm, but then why in minix_first.S are that movs setting DS, ES, SS, SP before jmp to setup?
At least the stack should be set correctly?

// Ok, everything should be where it belongs call it
mov $ELKS_INITSEG,%ax
mov %ax,%ds
mov %ax,%es
mov %ax,%ss
mov $0x4000-12,%sp
ljmp $ELKS_INITSEG+0x20,$0

[mfld-fr]

@xrayer : Yes, you are right, no registry setting for any parameter, but at least a valid stack, so SS:SP.

[mfld-fr]

@ghaerr : good spot of UMSDOS option in the existing code, but that option has never been tested.

I am interested in any proposal to make ELKS bootable from a FAT volume, as already stated above and in multiple discussion threads 😄

[mfld-fr]

The discussion was going interestingly beyond the initial question from @xrayer, but I am closing this issue as that initial question was answered : no complete & experimental support for now to boot from a FAT volume, and no way to reuse a third-party boot sector 'as is'.

Any proposal to implement that support is welcome in the form of one or several PR.

[mfld-fr]

Discussion to make ELKS bootable from FAT is continuing in #296.