support for hooks

[dkwo]

Is there any room to support hooks, as e.g. in mkinitcpio?
The example I'm looking at is Asahi Linux, which recently introduced this mkinitcpio hook
https://github.com/AsahiLinux/asahi-scripts/blob/main/initcpio/hooks/asahi
It runs an update-vendor-firmware script during initramfs loading on Apple Silicon.
I wonder if functions like run_earlyhook and run_latehook can be used with booster.

Thanks.

[anatol]

Having hooks for booster is a useful idea. Though it is still unclear what the architecture for it is going to look like. I still need use-cases that justify adding hooks to booster.

Could you please add more information about asahi hooks? What exactly that tool does in initramfs? Where are the sources for /usr/bin/update-vendor-firmware tool?

[dkwo]

The asahi hook's purpose is to run the script https://github.com/AsahiLinux/asahi-scripts/blob/main/update-vendor-firmware
on Apple silicon (M1 and later), thorugh which a booted OS mounts the ESP and inspects the manifest.txt file. If this is found to be different from the existing vendor firmware installed in the OS partition (if any), it updates it by extracting firmware.tar into /lib/firmware, and then comparing both manifest files and removing any files that are no longer present in the ESP manifest. (This should be done before any drivers requiring firmware are loaded.)

[anatol]

I was trying to read the code and it looks like all the script tries to do is to unpack $EFI/firmware.tar into initramfs' /lib/firmware/. Are you aware of why do they need to do it? What is the use of the firmware files in /lib/firmware? How does initramfs use it?

[dkwo]

I think this is a way to load a small subset of firmware blobs, see also https://github.com/AsahiLinux/docs/wiki/Open-OS-Ecosystem-on-Apple-Silicon-Macs Doing it everytime in the initramfs is not vital, but it seems like a good way to do it.

[dkwo]

The linked document has been updated, with a new proposal for linux; the last paragraph now says:

Distros should then ship an initramfs with a /lib/firmware/vendor -> /vendorfw symlink, to allow the kernel to load early-loaded firmware directly. [...] The initramfs must then forward this firmware into the final root filesystem.

[anatol]

Distros should then ship an initramfs with a /lib/firmware/vendor -> /vendorfw symlink, to allow the kernel to load early-loaded firmware directly.

That is what I was thinking as well.

For me these firmware files looks similar to Intel/AMD microcode firmware that needs to be loaded early as well.

And the kernel already has a solution to handle Intel/AMD fw files. There are drivers that load fw files with a specific names (e.g. kernel/x86/microcode/GenuineIntel.bin)

https://github.com/torvalds/linux/tree/3eba620e7bd772a0c7dc91966cb107872b54a910/arch/x86/kernel/cpu/microcode

It sounds like Apple M1 firmware files should use a similar solution.

cc @marcan for more comments.

[anatol]

And if AsahiLinux moves this way then I'll be happy to add Apple firmware support to https://github.com/anatol/ucode-image-gen generator tool.

[dkwo]

Assuming kernel can load from /lib/firmware/vendor, do I need to unpack the file firmware.cpio file, located in the EFI system partition under the vendorfw folder, and symlink it, something like mount -t tmpfs vendorfw /lib/firmware/vendor and cp -r vendorfw/* vendorfw/.vendorfw.manifest /lib/firmware/vendor, within booster; or is the idea that the ucode-image-gen tool produces right away something that can be concatenated to the initramfs, and fed to m1n1?

[marcan]

The idea is that 1) the kernel has that path added to the load list, 2) the bootloader loads the cpio as an additional initrd, 3) the initramfs has a /lib/firmware/vendor -> /vendorfw symlink so firmware within the initramfs "just works", and 4) the initramfs has something to mount a tmpfs on /lib/firmware/vendor in the new root filesystem and copy the firmware into it prior to pivoting.

However, since the bootloader loading the initramfs directly is only practical in some cases (/boot as EFI, installed on internal NVMe), the initramfs should also have some fallback code to go looking for it itself on early boot and extract it, before anything that might load firmware-dependent kernel modules runs (i.e. before udev). See the asahi-scripts hook for an example:

https://github.com/AsahiLinux/asahi-scripts/blob/main/initcpio/hooks/asahi

[marcan]

As for why: most people want their keyboard to work in the initramfs (e.g. typing in unlock passphrases), and the xHCI controller used for some USB ports on some Apple Silicon machines needs firmware to work. Until then we'd gotten away without the initramfs involved in handling firmware, but that particular case forced our hand in getting firmware available from very early boot.

In particular, you can't let udev loose until the firmware is available, because it can and will get hotplug events from PCIe devices coming up late and try to load the modules before the firmware is ready, even in the initramfs.

[marcan]

Of note: these firmware files are not redistributable (so they can't go in the initramfs directly, not if you plan to ship it to users / use it in an installer), and they are not "early-load" firmware like microcode from the kernel's POV (so it doesn't make sense to do something like intel-microcode, we use the standard firmware loader stuff instead).

[scruffidog]

Adding my $0.02 for another use case:
mounting of a separate /usr partition. I understand the trend towards throwing everything into single partition but for me, it is a quick and easy approach towards immutability by just mounting /usr RO. This avoids significant amount of relearning/reinventing the wheel of structural changes to the OS, retooling of packaging/building conventions, installation and maintenance (looking at SerpentOS). If I can have a hook/facility to do that, it would make the tool ideal.

Expanding on this, it can be as simple as the inclusion of a script stub-out; sort of a initramfs rc.d facility. It is fairly easy to understand and provides flexibility to totally mess up your system for the masochists out there...

As a further thought, this facility can be sort of a external plugin repo that others may find useful.

[firasuke]

What's the current status on this? I am currently trying to get booster to work as an initramfs for a live installation ISO, and having hooks support would be very useful.

[B83C]

A good use case for hooks is adding support for systemd-hibernate-resume.
Have a look at mkinitcpio: https://github.com/archlinux/mkinitcpio/blob/f3472baf6d98e4f21763c47828dac6f074ac8048/hooks/resume#L12

Is it possible to implement the same thing in booster?