uefi:ntfs and shim

[Sporesirius]

Hello, does shim work with uefi:ntfs?

[pbatard]

No.

Shim is just an extension of Secure Boot (controlled by Red Hat instead of Microsoft), and, because UEFI:NTFS is GPLv3, it is not compatible with either Secure Boot or any extension of it.

[Sporesirius]

Is shim not just a bootloader with a signed key?

[pbatard]

It's a bootloader. But, since it is signed by Microsoft for Secure Boot, it will not boot anything that is not signed (else it would completely defeat Secure Boot), and Red Hat and Canonical (who have control over validating what the shim boots next) are only interested in signing Linux kernels.

Besides, if I were to use a shim for Secure Boot compatibility, I would rather use my own shim, since I don't care about booting EFI executables that aren't Secure Boot compliant (as opposed to the Linux shim), and it should be very easy to write one that meets all of Microsoft's Secure Boot requirements. But if I were to do that, I might as well turn UEFI:NTFS itself into a shim (since it pretty much already is and, even if the license is currently GPLv3, I made sure I gave myself the capability to relicense the boot code under GPLv2 if needed, which would then allow it to be signed for Secure Boot) instead of using somebody else's, as we really don't care about the "validating and/or revoking non Secure Boot bootloaders that are signed with some other key" part, which is the main concern of the Linux shim.

[ValdikSS]

@pbatard,

and, because UEFI:NTFS is GPLv3, it is not compatible with either Secure Boot or any extension of it.

That is not true. Grub2 is also GPLv3 but it's compatible with Secure Boot and shim.
Actually, UEFI:NTFS should work with shim, but for some reason right now it fails to load NTFS driver, although shim 15 does hook LoadImage/StartImage and it should work, but it doesn't.
But it works with PreLoader, if you enroll both hashes for the program and for the driver.

If you would be able to debug the reason why it fails to load driver with shim, you can use signed shim from Fedora with Fedora's built-in certificate. It won't launch automatically and would require enrolling certificate using MokManager interface, but it's free.
As an alternative, you can sign your own shim with Microsoft key. More information here: https://github.com/rhboot/shim-review

[ValdikSS]

You'll need to compile shim with OVERRIDE_SECURITY_POLICY define, or to write code to load driver efi file in memory and jumping to its entry point, without using LoadImage/StartImage. The former is easier, but I'm not sure why isn't it enabled by default, maybe there are drawbacks.

[pbatard]

Grub2 is also GPLv3 but it's compatible with Secure Boot and shim.

GRUB 2 being compatible with Secure Boot is in direct contradiction with point 4 of the Microsoft Secure Boot guidelines (that explicitly mentions that GRUB 2 can't be Secure Boot signed):

4. Code submitted for UEFI signing must not be subject to GPLv3 or any license that purports to give someone the right to demand authorization keys to be able to install modified forms of the code on a device. Code that is subject to such a license that has already been signed might have that signature revoked. For example, GRUB 2 is licensed under GPLv3 and won’t be signed.

So, while GRUB 2 is indeed compatible with shim, unless "compatible" means something other for you than "can be signed to work with" or Microsoft have changed their rule (in which case, can you point to the updated guidelines?), it is certainly not compatible with Secure Boot.

you can use signed shim from Fedora with Fedora's built-in certificate

I did consider that.

But that would mean add a lot of unwanted baggage, as I'd need to include a shim for x86_64, x86_32, ARM and ARM64, and, because UEFI:NTFS is designed to be used with Rufus, I am very size conscious.

Rufus is currently a 1MB executable (and that payload size does include UEFI:NTFS for all the archs I mentioned above). So I really don't want to have to double that size, or worse, just to work around an issue that would not exist in the first place if Microsoft weren't abusing their position.

As an alternative, you can sign your own shim with Microsoft key. More information here: https://github.com/rhboot/shim-review

Thanks, that is helpful information. But I'm not sure it is that applicable.

You'll need to compile shim with OVERRIDE_SECURITY_POLICY define, or to write code to load driver efi file in memory and jumping to its entry point, without using LoadImage/StartImage. The former is easier, but I'm not sure why isn't it enabled by default, maybe there are drawbacks.

I would venture to guess that OVERRIDE_SECURITY_POLICY is not enabled because if you use LoadImage/StartImage, and don't have your own mechanism for validating the signature of the driver you just loaded, you've just opened your shim to run arbitrary code (since anybody could replace the driver with their own malicious version). And I'm not seeing loading the EFI in memory and jumping straight into it as that great for security either. At least, if I was responsible for signing shims, there's no way I'd sign a binary that loads a file and runs it without first validating a signature to confirm that it's a version that was produced by the developer.

All in all, I'm seeing a lot of extra work (adding custom signature validation, since I can't rely on Secure Boot ones) and other hurdles, that will require a lot of time, which I'd rather use on something else.

I'm pretty much seeing the amount of work required to write a shim that works with the current UEFI:NTFS about the same as the amount of work required to port the GPLv2 NTFS-3g driver to UEFI, and (with a GPLv2 relicensed bootloader) get that signed for Secure Boot.

Even if I currently don't have the time for any of those, between these 2 options, I must say that I do have a strong preference for the one that doesn't require a shim.

[ValdikSS]

So, while GRUB 2 is indeed compatible with shim, unless "compatible" means something other for you than "can be signed to work with" or Microsoft have changed their rule (in which case, can you point to the updated guidelines?), it is certainly not compatible with Secure Boot.

There's the following a bit below:

a. If your submission loads a GRUB or other loaders, it must be SecureBoot enlightened.
For example, the latest GRUB 2 with SecureBoot patches.

This means that grub2 by itself can't be signed but shim can, and it should be used for grub 2 with secure boot patches (these patches prevent loading other modules and restrict other things).

I would venture to guess that OVERRIDE_SECURITY_POLICY is not enabled because if you use LoadImage/StartImage, and don't have your own mechanism for validating the signature of the driver you just loaded, you've just opened your shim to run arbitrary code (since anybody could replace the driver with their own malicious version).

No, actually it overrides security policy with it's own policy, not just remove it. I've booted UEFI:NTFS with PreLoader (which overrides security policy, the code in shim is the code from preloader) just fine. No UEFI:NTFS modification needed with OVERRIDE_SECURITY_POLICY, and this option does not allow to boot any arbitrary efi files, only the ones which are allowed by shim/preloader.

All in all, I'm seeing a lot of extra work (adding custom signature validation, since I can't rely on Secure Boot ones) and other hurdles, that will require a lot of time, which I'd rather use on something else.

shim provides validation mechanism, it's easy to use.

Also see comments, for example:

Hi Tom, GRUB 2 itself wont be signed, but SHIMs which are designed to further load boot loaders like GRUB are signed. #5 calls out that though your submission (like SHIM) can load a boot loaders like GRUB 2, only if those boot loaders honor secureboot requirements (authentication of all the code that is loaded and executed subsequently)

[pbatard]

There's the following a bit below:

a. If your submission loads a GRUB or other loaders, it must be SecureBoot enlightened.
For example, the latest GRUB 2 with SecureBoot patches.

Yes, I'm well aware of it. That paragraph is hard to miss when you are pointing to the guidelines for Secure Boot signing on regular basis...

This means that grub2 by itself can't be signed but shim can, and it should be used for grub 2 with secure boot patches (these patches prevent loading other modules and restrict other things).

Yes. The problem I have is how you seemed to express the idea that GRUB2, on its own, was compatible with Secure Boot, which is what I wanted to make clear, to avoid people coming to this thread and thinking that what I've been telling them about UEFI:NTFS not being able to be signed for Secure Boot (because it contains GRUB2 GPLv3 code) was incorrect.

So, just to clarify once more, GRUB2 (or anything GPLv3 for that matter) can not be signed for Secure Boot, so it's not compatible with Secure Boot per se. Thus, if you want to somehow use it in a Secure Boot process, then you must jump through the hoop that is the shim.

No, actually it overrides security policy with it's own policy, not just remove it.

Okay. I hadn't looked (and still haven't - will try to find some time later on) at how exactly OVERRIDE_SECURITY_POLICY works and that is indeed the right thing to do. But that is pretty much what I stated: If you want to use a shim, then you must implement a security mechanism that duplicates what Secure Boot does.

Of course, if there is existing code, that has been validated, and that can be reused so that you don't have to rewrite this from scratch, and if the resulting shim binaries are small enough, then it is something worth looking at.

I've booted UEFI:NTFS with PreLoader (which overrides security policy, the code in shim is the code from preloader) just fine.

That's indeed very nice. If you did that, then I'm very interested in any code you have to share
Can you please upload it somewhere and give a link?
Or did you not have to modify anything?

But what I'm even more interested in is how large the resulting shim was, since, again, my main concern (besides time required to craft a working shim), is how much larger adding the x86_32, x86_64, ARM and ARM64 shim binaries would make Rufus.

From what I have seen from "The Red Hat shim" source, and what I remember from distros such as Fedora, it seemed to be rather quite large, which I suspect is partly due to coming with a bit of additional stuff (e.g. httpboot) that I'm not sure would be really required for my usage in Rufus (straight USB boot). Then again, if the size addon is minimal, I don't have much of an issue with keeping this extra stuff... as long as the resulting binaries don't require me to multiply the size of Rufus by a factor 2 or 3.

shim provides validation mechanism, it's easy to use.

Okay, at the risk of sounding pedantic once again, I think we need to clarify one last thing.

There's "a shim" (which is something anybody can write, from scratch, and then ask Microsoft to sign, in order to provide their own shim mechanism) and then there's what you write as "shim", which appears to be the version of "a shim" that has been developed by Red Hat and other folks (i.e this thing) and which would probably be better qualified as "The Red Hat shim" to avoid confusion.

In the above, since you appear to be talking about existing code, I think it's worth pointing out that you mean "The Red Hat shim" rather than "a shim".

At any rate, I think your comments have provided me enough ammunition to want to look at the Red Hat Shim again, to see if, as you assert, using it for UEFI:NTFS might not be as painful as I anticipate it to be.

And once again, any data you want to provide on the work you have already carried out on testing that will help!

[ValdikSS]

That's indeed very nice. If you did that, then I'm very interested in any code you have to share
Can you please upload it somewhere and give a link?
Or did you not have to modify anything?

I did not, take a look at this archive. It contains Linux Foundation PreLoader and UEFI:NTFS x86_64. I had to copy NTFS driver twice since, for some reason, PreLoader hangs when entering Rufus folder (at least in QEMU), so NTFS driver is also in /EFI/BOOT.
To boot this image, you need to enroll file hashes of "loader.efi" (UEFI:NTFS) and "ntfs_x64.efi" on the first boot, using HashTool.efi interface (launches automatically).
uefintfs-preloader.zip

Use the following command to run this image in QEMU:
sudo qemu-system-x86_64 -machine q35,smm=on,accel=kvm -m 1024 -global driver=cfi.pflash01,property=secure,value=on -drive file=OVMF_CODE.secboot.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=vars-clean,if=pflash,format=raw,unit=1,readonly=off -vga qxl uefintfs-preloader.img
qemu-secureboot-bios.zip

From what I have seen from "The Red Hat shim" source, and what I remember from distros such as Fedora, it seemed to be rather quite large, which I suspect is partly due to coming with a bit of additional stuff (e.g. httpboot) that I'm not sure would be really required for my usage in Rufus (straight USB boot).

Red Hat Shim size is large. For example, Shim v15 x86_64 in Fedora is 1.2 MiB, and MokManager (HashTool alternative) is also 1.1 MiB. This is mostly because of the build process which does not involve unused functions stripping, that's why it contains unstripped OpenSSL with all the functions and is very big.
There's a fork with proper linking process: rhboot/shim#151

6. As the result, substantially reduced footprints of EFI files
(e.g., 425Kb vs 1Mb for shimx64.efi compiled with -Os).

But what I'm even more interested in is how large the resulting shim was, since, again, my main concern (besides time required to craft a working shim), is how much larger adding the x86_32, x86_64, ARM and ARM64 shim binaries would make Rufus.

May I wonder why? Does it matter that much if the partition is 1 MiB or 4 MiB?

[pbatard]

Thanks for your update, as well as the attachments. Let me see what I can make of them.

Red Hat Shim size is large. For example, Shim v15 x86_64 in Fedora is 1.2 MiB, and MokManager (HashTool alternative) is also 1.1 MiB.

Yeah, that's in line with what I had seen before and what I was expecting. Even if space can probably be gained with LZMA compression (which I use to compress Rufus assets), this is very very large as far as I am concerned.

May I wonder why? Does it matter that much if the partition is 1 MiB or 4 MiB?

Rufus is 1 MB. And again that includes all the UEFI:NTFS bootloaders and drivers for all archs. Even with compression, I expect that adding shims will at least double Rufus size.

Now, a lot of people might say "What does it matter if Rufus is 1 MB or 3 MB?", but, as far as I am concerned (and as opposed to the approach Etcher has taken - while I do like Etcher and think its developers have done a great job with it, I don't see having to download ~40 MB for what could be simplistcly qualified as a "glorified dd" as that great) I don't think that an application that mostly exists to create bootable USB drivers should take that much space. Especially when they are in the process of setting up something from scratch, some people may not have that great a bandwidth connection, and, if I can, I owe it to them to make the application as small as I possibly can. For instance, that's part of the reason why I am limiting the number of localizations that are included in Rufus, and why I tend to carefully consider features that require large assets in terms of "how many people will this benefit vs how much bigger will the application get".

Also, up until very recently, and possibly yet again in the future, I was paying for bandwidth.... and with circa 3 millions of downloads of Rufus a month, even doubling the size can be troublesome (but I will not hold this as a major problem, since I'm using github releases at the moment).

Finally, considering that ALL x86 platforms I know of do have the ability to temporarily disable Secure Boot, and that turning it off is only required for the initial installation, and that I do assert that if one created their bootable media after having validated the SHA of their image (which Rufus, which is also digitally signed, can compute for them), then temporaily disabling Secure Boot for the initial install is not as big a deal as the people who zoom on the "Secure" terminology of "Secure Boot" as meaning it should not be disabled ever think it is.

So, between at least doubling the size of Rufus or telling people that they can temporarily disable Secure Boot (as long as they're not trying to install on the few ARM machines that Microsoft tried to lock into Secure Boot always a few years ago), I do see it as preferable to go with the latter for now.

But of course, that doesn't mean I'm not gonna check the possibility of using the Red Hat shim and see what comes out.

[ValdikSS]

Managed to chain shim and preloader to circumvent uefi security policy, so you can load unsigned drivers.
https://github.com/ValdikSS/Super-UEFIinSecureBoot-Disk
preloader can be as small as 60 KiB.

[ValdikSS]

Microsoft deleted https://blogs.msdn.microsoft.com/windows_hardware_certification/2013/12/03/microsoft-uefi-ca-signing-policy-updates/

[ValdikSS]

https://techcommunity.microsoft.com/t5/Windows-Hardware-Certification/Microsoft-UEFI-CA-Signing-policy-updates/ba-p/364828?advanced=false&collapse_discussion=true&q=uefi&search_type=thread

[pbatard]

Thanks a lot for notifying on this as well as the updated link. Much appreciated!