Multiboot2: Support for the EFI Boot Service tag

[digitalrane]

I've been working on getting Xen supported as a bootable payload via Limine for NixOS in this PR.

The short version is that Limine currently (and may not want to) support the EFI Boot Services Multiboot2 tag, and the EFI Boot Service is exited and memory remapped for all payloads. This prevents Xen from booting in a UEFI environment via multiboot2. Booting via the efi protocol works fine, however requires a bunch of extra configuration explained in the above PR. It'd be great to be able to consistently use multiboot2 regardless of BIOS or UEFI to boot Xen under Limine.

With that said, here's the long version, with some more background of what I've already looked into:

As part of initial testing, I found that multiboot2 would not boot Xen under BIOS or UEFI, however I was able to use multiboot1 on BIOS systems successfully. I raised #482 which was resolved by c8cdc9e and 80912a3. I've tested and under both BIOS and UEFI the original error is resolved. However, I have only been able to successfully boot Xen under BIOS via multiboot2. The UEFI version of Limine gets further, however after several days of snooping around source code and serial consoles, I managed to catch the message ERR: Bootloader shutdown EFI x64 boot services! (via Serial console) from the Xen kernel.

This message occurs because Xen requires, when booted under UEFI, via multiboot2, that the EFI Boot Service is left running (i.e. in Limine this happens via the efi_exit_boot_services() call in multiboot2.c. I've done a bunch of reading into why Xen needs this, and how it's handled in GRUB, and the short version is: It's a horrible, cursed workaround to accomodate Xen needing access to the EFI Boot Service & original memory map so it can work out the usage for certain allocated ranges in memory.

In GRUB, a special Multiboot2 tag was added to the spec to indicate that the Multiboot2 kernel requests that the EFI Boot Service is retained and not exited prior to passing control to the Kernel. There's some history here and the GRUB source code in grub-core/loader/multiboot_mbi2.c shows a good chunk of the logic of how GRUB handles this (the keep_bs variable is indicative that the Boot Service should be kept running in this file).

I took a very crude attempt at testing whether I could get this working by adding a config flag keep_boot_service to the multiboot2 provider, which prevented the call to efi_exit_boot_services. Additionally, I also (as a total hack to see how far I could get, I know this isn't the correct approach) converted the panic() for get_raw_memmap called whilst in boot services into a simple warning using print. So whilst this does seem like it prevented the call to efi_exit_boot_services when set, I think because the memory was still being remapped in multiboot2.c Xen still couldn't see the Boot Service and failed with the same error message. So I think the unmodified memory map would need to be passed to Xen, and the EFI Boot Service kept running to support this. Ideally, it should only happen if the prior mentioned boot tag was present.

I'd previously provided these binaries as reference, so I'll link them again. These binaries should exhibit this behaviour: https://junkyard.systems/limine-nixos-test-binaries.tar.xz (I had to configure Xen to log to serial with console=com1 com1=115200,8n1 so I could catch the log message I mentioned earlier.

[mintsuki]

Thanks for the detailed issue. I'd argue that Xen depending on the EFI boot service tag like this despite it clearly not indicating that (as in, the tag not being marked as being required to support) is an upstream Xen bug. Xen should either clearly indicate that it needs the tag (which it seemingly does not), or it should not attempt to use EFI boot services if booted via the non-EFI entry point.

That aside, this issue is still good as a feature request for the EFI entry point for multiboot2.

What about multiboot1 though? You said you could not boot Xen with multiboot1? It would be nice if you could open an issue about that separately.

Also I saw mentioned by @phip1611, in the nixOS issue you linked, about the memory map being potentially broken, and I'd like to repeat once again that there is no indication that that's the case at all, which is why I had closed #356 already (with explanation that it was likely just a printing mistake).

[digitalrane]

What about multiboot1 though? You said you could not boot Xen with multiboot1? It would be nice if you could open an issue about that separately.

Oh yep I did! That was the multiboot2 issue. Sorry, I must have made a typo somewhere, multiboot1 has always worked. multiboot1 however doesn't support EFI so that's not an option for EFI booting, which this issue is focused on.

I'd argue that Xen depending on the EFI boot service tag like this despite it clearly not indicating that (as in, the tag not being marked as being required to support) is an upstream Xen bug

I recall reading someone making this point in one of the mailing list threads too. The tag is meant to "request" that the full memory map be passed and the Boot Service is not terminated, however Xen "requesting" this does not represent the fact it won't boot at all if this doesn't happen, and there's no way for it to pass the fact that it's required, and there's not alternate path in the Xen kernel rather than a panic(). It's definitely not an ideal implementation.

about the memory map being potentially broken

I also have not reason to suspect the memory map is broken. I think for Xen the memory map is not workable, but not because it's broken in some way, just because this behaviour with the tag & passing the full original memory map directly to Xen is what's required by Xen and that's not currently supported by Limine. So this is very much a feature request to support this behaviour.

[mintsuki]

Oh yep I did! That was the multiboot2 issue. Sorry, I must have made a typo somewhere, multiboot1 has always worked. multiboot1 however doesn't support EFI so that's not an option for EFI booting, which this issue is focused on.

Alright, I probably misunderstood myself, too. That said, Limine itself can boot multiboot1 kernels on EFI just fine, though usually that's not very useful due to limitations of the protocol itself, and I assume Xen just panics or something.

I recall reading someone making this point in one of the mailing list threads too. The tag is meant to "request" that the full memory map be passed and the Boot Service is not terminated

In all fairness what this tag means is really, really vague, at least as per multiboot2 spec. But we both know that the multiboot2 spec is broken and incomplete in several places and the true multiboot2 spec is GRUB2.

however Xen "requesting" this does not represent the fact it won't boot at all if this doesn't happen, and there's no way for it to pass the fact that it's required

Well, technically there is. It can mark the tag as required, and Limine itself will panic on the fact that it doesn't know how to interpret the tag. That said, this will probably cause it to panic on BIOS too... Maybe this is the reason this isn't done? This is another quirk of multiboot2's design, but it could be easily worked around by the bootloader by only panicking if this tag is requested on UEFI, but this is also something that would be, at least in part, specific to accomodate Xen...