RFE: instead of invoking exactly grubx64.efi iterate through drop-in dir

[poettering]

Currently shim strictly chainloads grubx64.efi (well, at least on x86-64). This is less than ideal for two reasons:

1. not everyone uses grub (i.e. people use systemd-boot too, and currently when using that together with shim rename it grubx64.efi)
2. this is a single point of failure: if for some reason updating the grubx64.efi binary doesn't succeed properly the user will be thrown in mokmanager rather than some more suitable fallback that tries to recover from the hosed updated.

Hence I'd propose the following logic, that is dumb enough to be safe, but avoids both these issues:

enumerate EFI binaries from some to be defined directory (for example \EFI\shim-payload\*.EFI or so), sort them via strverscmp() (ideally the modified one the boot loader spec mandates, as per https://uapi-group.org/specifications/specs/boot_loader_specification/), and then invoke the newest binary. if that fails, go for the next, and so on, and so on, until one works.

This should be reasonably simple and robust and decouple shim from the boot loader implementation while increasing safety for participants in this: they can drop a new entry in the dir, and keep a couple of them around and delete the old ones when they see fit. versions never fight for the same filename anymore.

[bluca]

This is orthogonal to establishing trust and signing, so seems like a good idea in general to me. Currently the only way to change this is via a build-time makefile variable, which works fine for local testing, but not much more.

[dennis-tseng99]

This must be a good idea. Though, I'm just worried Microsoft is willing to sign those uncertain combinations of shim + (non-grub2 + some proprietary 2nd stage boot loaders) located in \EFI\shim-payload\*.EFI

[poettering]

@dennis-tseng99 msft has no problem if people sign other boot loaders. noone at msft is glued to grub.

[bluca]

The signing discussion is off-topic here, it's really orthogonal to how binaries are searched, let's focus on this

[esnowberg]

@poettering @bluca I sent the following pull request: #608 that will solve this problem with a configuration file. I would be interested in your feedback on this approach.

[poettering]

Uh, I am not sure what the benefit would be to have a config file for that. (and csv! yikes). it's much simpler to just look for the alternative files and use them, rather than making things indirect.

config files kind mean yu also want to measure them, and so on, it just makes things a ton more complex for little gain.

[julian-klode]

@poettering Directory iteration only works on local file systems, silently failing on remote HTTP and TFTP boot locations. But anyway I don't believe this is valuable that way because we already have boot$arch.csv for the fallback loader and that's supposed to be merged so we do not need to reset the system anymore but can just chainload.

[julian-klode]

Of course you could call them kernel${efi_arch}0001.efi and so on and try them in order until you run out.

[julian-klode]

See issue #482 for more details on how trying the directory bit us when we tried it for certmules.

[poettering]

@poettering Directory iteration only works on local file systems, silently failing on remote HTTP and TFTP boot locations.

For HTTP and TFTP the kind of reliability fallback logic is not really necessary, where you'd iterate through a dir to find a working version. After all (at least in my thinking) that's about dealing with bad file system drivers, incomplete writes and so on, but nothing of that sort applies to http/tftp. Hence if you ask me, it would be totally fine to do the dir thing i proposed, and if it works, then great, but also do something else, either before or after.

But anyway I don't believe this is valuable that way because we already have boot$arch.csv for the fallback loader and that's supposed to be merged so we do not need to reset the system anymore but can just chainload.

What is "boot$arch.csv"? csv files? as in comma separated vaues files? That sounds less than ideal to me, for various reasons... Is there a link about details?

Generally, is that a file that would have to rewritten each time a boot loader is updated? that's precisely what I want to avoid with my proposal: single point of failures. on vfat it's a lot better to not update files too much, but rather create new ones and leave old files as they are, until one day they are deleted.

[poettering]

for sd-boot the boot loader spec requires drop-in style stuff to define type 1 and type entries, btw, i.e. it currently requires directory enumeration to work at all. There's a TODO list item somwhere to extend that so that if directory enumeration doesn#t work, sd-boot instead looks for a file SHA256SUMS that follows the standard UNIX file format for that (that openssl also supports, and a myriad of other tool) that it uses as a list of files in the dir. Using SHA256SUMS as a "manifest" of files in a dir, is really nice, since it first of all gives you a nice, standardized way to enumerate/create enumerations of files with standard UNIX tools, and as side-effect also allows you to validate them cryptographically. And it's nicely incremental: if you can't enumerate the dir directly, use the file, if can don't. Or do, up to you, then you can validate the file. And all you have to do is to run the equivalent of sha256sum * > SHA256SUMS on each update.

[poettering]

So to summarize, here's what I'd propose:

add support for something like \EFI\shim-payload\*.EFI which is version sorted, and the newest is picked, falling back to older versions consecutively until one works. If enumeration doesn't work because backing store is not a proper fs, then load \EFI\shim-payload/SHA256SUMS instead, sort its entres by the same rules, and the load them by path.

[esnowberg]

What is "boot$arch.csv"?

It has been this way for 10 years, since the first version of fallback.

3ce517f

csv files?

Yes

as in comma separated vaues files? That sounds less than ideal to me, for various reasons... Is there a link about details?

https://github.com/rhboot/shim/blob/main/README.fallback

[septatrix]

Implementing this would be beneficial for Fedora which are working towards better systemd-boot as an alternative to GRUB depending on ones needs. https://fedoraproject.org/wiki/Changes/cleanup_systemd_install Support such a setup with secure boot is also planned down the line and would require this feature.

Automatic enumeration as sd-boot does it seems like the cleanest and most robust way forward here.

[bluca]

What we discussed a couple of weeks ago was to support all these options: first try to load the configured default second stage by name, then try to use the new csv file if present, and if not proceed to enumerate the directory content as discussed in this ticket. This should satisfy all use cases and requirements for everyone.

[TBOpen]

You know I already provided a patch (twice) that allows putting the boot file in a utf16 text file. Here's a copy (the acme stuff would be your company name).

shim-15.4.patch

[esnowberg]

@TBOpen I was not aware there was a previous attempt. Is there an old pull request that can be referenced?

[TBOpen]

I think I just sent a patch file as I did this time .. it's been a while.

[TriMoon]

Just my 2 cents worth of support for this RFE:

• As a user who ❤️LOVES to get rid of grub completely on my system, i would very much like this approach of enumerating the files in the same dir as shim 👍
  Please don't make use of csv files, it is like reaching your left earlobe using your right hand behind your head...

Lets focus on local files first instead of thinking about remote booting...