Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Secure boot support #4371

Open
jasperweiss opened this issue Oct 4, 2018 · 26 comments
Open

Secure boot support #4371

jasperweiss opened this issue Oct 4, 2018 · 26 comments
Labels
C: other help wanted This issue will probably not get done without help from community contributors. P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. security This issue pertains to the security of Qubes OS. T: enhancement Type: enhancement. A new feature that does not yet exist or improvement of existing functionality.
Milestone

Comments

@jasperweiss
Copy link

jasperweiss commented Oct 4, 2018

According the secure boot specification, users can enroll their own keys for secure boot.
If the QOS bootloader were signed, users could manually enroll the signing key within the UEFI. That would be a better anti evil maid system since it doesn't require the use of potentially untrusted USB keys.

Since dom0 doesn't contain any 3rd party applications, we can enforce code signing on anything that runs within it.

This blog post mentions secure boot being problematic due to running CA's etc but providing the user with a public key they can enroll manually would be doable.

Edit: I'm aware the developers are generally not very fond of secure boot. Could anyone explain why?
Alternatively a TPM could be used to unseal the drive encryption key.

Pre-OS firmware components are hashed (measured)
Measurements are initiated by startup firmware (Static CRTM)
Measurements are stored in a secure location (TPM PCRs)
Secrets (encryption keys) are encrypted by the TPM and bounded to
PCR measurements (sealed)
Can only be decrypted (unsealed) with same PCR measurements
stored in the TPM
This chain guarantees that firmware hasn’t been tampered with

@marmarek
Copy link
Member

marmarek commented Oct 4, 2018

See this message the further down the thread. In short: just signing bootloader and/or Xen isn't enough, and would result in something trivial to bypass (for example using kernel parameters).

@jasperweiss
Copy link
Author

jasperweiss commented Oct 4, 2018

@marmarek so the bootloader should check the next piece in the boot-chain as well as as all the parameters. Wouldn't it be possible to enforce code signing throughout dom0 similar to Windows 10S?

@marmarek
Copy link
Member

marmarek commented Oct 4, 2018

Parameters are different for each installation, so you need to sign them yourself. Read that thread.

And BTW just setting up SecureBoot as a protection against someone with physical access (replacing /boot etc) isn't enough. One can simply disable SecureBoot and you'll most likely not notice. On some systems it might be non-trivial without a BIOS password (re-flashing BIOS etc), but still very much doable with physical access.
So, here SecureBoot guards only against very simple attacks. Note that SecureBoot makes much more sense on non-Qubes system, where compromising monolithic system is enough to replace boot files, without physical access.

On the other hand, AEM-like solution guard against this kind of attacks too, because it doesn't rely on BIOS to prevent the boot if something goes wrong. Instead, TPM will not release keys if you boot different binary. While there are different ways to bypass BIOS refusal to boot, you can't fake to the user data decryption if you don't have a key.

Anyway, those mechanisms are not excluding each other. Right now AEM is not compatible with UEFI, so getting either of them implemented would be an improvement.

@jasperweiss
Copy link
Author

jasperweiss commented Oct 4, 2018

Indeed a TPM would be necessary to provide meaningful protection. It measures the UEFI firmware data, settings, secure boot state and provisioned signing keys prior to releasing (part of) the decryption key.

@andrewdavidwong andrewdavidwong added T: enhancement Type: enhancement. A new feature that does not yet exist or improvement of existing functionality. help wanted This issue will probably not get done without help from community contributors. C: other security This issue pertains to the security of Qubes OS. labels Oct 5, 2018
@andrewdavidwong andrewdavidwong added this to the Far in the future milestone Oct 5, 2018
@marmarek
Copy link
Member

marmarek commented Oct 14, 2019

Notes how kernel signing is done in Fedora build infrastructure:
kernel build log:

+ '[' -f arch/x86_64/boot/zImage.stub ']'
+ _pesign_nssdir=/etc/pki/pesign
+ '[' 'Red Hat Test Certificate' = 'Red Hat Test Certificate' ']'
+ _pesign_nssdir=/etc/pki/pesign-rh-test
+ '[' -x /usr/bin/pesign ']'
+ '[' x86_64 == x86_64 -o x86_64 == aarch64 ']'
+ '[' 0 -ge 7 -a -f /usr/bin/rpm-sign ']'
++ id -un
++ uname -m
+ '[' 'Fedora Project' == 'Fedora Project' -a mockbuild == mockbuild -a x86_64 == x86_64 ']'
+ grep -q ID=fedora /etc/os-release
+ [[ bkernel04.phx2.fedoraproject.org =~ ^bkernel.* ]]
+ '[' -S /var/run/pesign/socket ']'
+ '[' -S /var/run/pesign/socket ']'
+ /usr/bin/pesign-client -t 'Fedora Signer (OpenSC Card)' -c '/CN=Fedora Secure Boot Signer' -i arch/x86/boot/bzImage -o vmlinuz.signed -s
+ '[' '!' -s -o vmlinuz.signed ']'
+ '[' '!' -s vmlinuz.signed ']'
+ mv vmlinuz.signed arch/x86/boot/bzImage

PESign tool (including pesign-client): https://github.com/vathpela/pesign
Setup exposing pesign socket to kernel builders: https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/bkernel/tasks/main.yml?id=17198dadebf59d8090b7ed621bc8ab22152d2eb6
Mock config snippet mounting that socket inside build chroot:

# mount the pesign socket into the chroot
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/var/run/pesign', '/var/run/pesign' ))
config_opts['plugin_conf']['package_state_enable'] = False

In short: kernel build environment is given access to the signing key through a bind-mounted unix socket. It may be doable to replicate similar setup in our build infrastructure (with appropriate logging what gets signed).

@osresearch
Copy link

osresearch commented Aug 4, 2020

image

As part of adding Qubes support to safeboot, I have a preliminary patch for Xen to build a unified xen.efi + xen.cfg + bzImage + initrd (and optional xsm), which can be signed with sbsigntool (including support for using a Yubikey or other PKCS#11 token to store the private key). osresearch/xen@765f0fe

With this modification there is no need for grub -- the UEFI firmware can validate the signature on the hypervisor bundle using the SecureBoot PK/KEK/db key chain and directly invoke it if-and-only-if the signature is valid. Although, as @marmarek pointed out in #4371 (comment), it is also necessary to use a TPM sealed secret that includes the platform key and secure boot DB to be able to prevent a local attacker from reflashing with Secure Boot disabled. The tpm2-attest part of safeboot can likely be ported to dom0 for remotely attesting this, although I haven't gotten that far yet in the Qubes support.

@marmarek
Copy link
Member

marmarek commented Aug 4, 2020

Thanks, this looks very promising!

I have two main issues with using xen.efi in general:

  1. Other boot files (dom0 kernel, initrd) loading by xen.efi is very unreliable - depending how xen.efi got started and/or specific UEFI implementation the required File System Protocol Interface may be missing or unable to access those files (including limitations like supporting ESP only, which on ISO9660 is limited to 32MB - pretty tight for installation initrd). Bundling everything into xen.efi may help partially here.
  2. It is quite cumbersome to modify anything about boot parameters (even things like "choose older kernel") without grub (or other bootloader with interactive menu). If you want to modify dom0 boot options, you need a file editor (grub at this point is not enough). This is especially painful if your system fails to boot - you need then external boot media to perform any kind of troubleshooting/recovery. Embedding boot files into xen.efi makes this process even harder. But you can have multiple files with different versions embedded (allows you to boot different version, but not modify boot parameters even if you temporarily disable SecureBoot).

Those two are UX and stability related issues, not necessary security related. But for anything to be considered into default installation, those are also very important. Although for non-default, for power-users only solution those are less critical.

As for security concerns mentioned earlier in this thread, I understand your approach requires each installation to use unique, user-owned keys to sign boot files and have them included into secure boot DB, right? With this assumption it should be fine. Just to be sure I haven't missed anything - there is no way to stretch this approach into something compatible with standard secure boot DB keys (MS-signed shim), right?

@osresearch
Copy link

osresearch commented Aug 4, 2020

Bundling definitely avoids having to figure out the file system APIs and also makes it possible to embed in a ROM firmware volume, if you're lucky enough to have that option.

Choosing between versions can be done with the UEFI boot manager, although this doesn't allow changing parameters.

Modifying the boot parameters is challenging, although that is somewhat intentional - since the kernel parameters and xen config are critical to security, allowing arbitrary changes by a local user/attacker is dangerous. On the (non-Qubes) safeboot systems, the configuration is to have separate EFI boot manager entries for linux, the signed+sealed+dmverity-protected system, and recovery, the signed kernel with write access, but without access to TPM sealed secrets. An administrator with access to the disk recovery key can boot into the recovery kernel, unlock with disk, install new packages, upgrade kernels, edit the linux command line, etc. Once the system has been repaired, they hash the filesystem and build a new unified Linux/initrd/etc and sign it with a hardware token. This single file goes into the ESP and replaces the existing linux target. They also pre-compute the hash of the unified image and sign a PCR policy that allows the TPM to unseal the disk encryption key when that kernel is booted.

This does require a bit more space on the ESP since the recovery and normal kernel can't share bzImage or initrd files, although that hasn't yet been a problem for the machines I've configured.

@osresearch
Copy link

osresearch commented Aug 4, 2020

For the general use case, it might be workable to have the install image signed with the distribution key and to have that key signed by Microsoft so that the installer can boot. If they don't change the defaults, they can continue to use the official kernel, initrd and config as signed by the Qubes key.

If they do want to use their own keys, perhaps during the setup the user could be prompted to setup a software or hardware key and point them towards the steps for enrolling these keys in their firmware. Then they would need to use to sign any new kernels or initrd images that they build for the dom0, as well as any xen upgrades. With safeboot it isn't seamless right now; you can boot into recovery, install updates, and then have to run a separate command to re-sign everything. This would likely only be beneficial for very

@marmarek
Copy link
Member

marmarek commented Aug 4, 2020

Modifying the boot parameters is challenging, although that is somewhat intentional - since the kernel parameters and xen config are critical to security, allowing arbitrary changes by a local user/attacker is dangerous.

Yes, definitely. I'd imagine modifying those would change some measurement and require disk recovery key (in case TPM unseal is used later) or require disabling SB in the firmware. But still possible without external tools.

Primary + recovery boot images should solve most of it, I think. As long as the user have real option to choose which one to boot - note many UEFI do not expose such menu, notably default UEFI in Thinkpads do not have it. In that case, some extra boot manager is needed.

If they don't change the defaults, they can continue to use the official kernel, initrd and config as signed by the Qubes key.

This is a bit problematic, because dom0 kernel cmdline and initrd (currently) do contain system-specific configuration. At least UUID of LUKS partitions to unlock and where to look for root filesystem. Remember we (currently) allow quite a bit of flexibility in disk layout (lvm/btrfs etc) so we can't easily hardcode defaults and in some cases even runtime detection may be challenging and lead to not finding the right partition.

the signed+sealed+dmverity-protected system, and recovery, the signed kernel with write access, but without access to TPM sealed secrets

How would that work with distribution key? The distribution-signed blob can't possibly include sealed secrets for particular TPM and PCR values... Or perhaps the sealed keys are included in the signed part? How are they loaded then?

@osresearch
Copy link

osresearch commented Aug 4, 2020

My Thinkpad X1 (gen 5 and 6) and T490 offer EFI boot manager support -- hit F12 when the Lenovo screen is displayed to bring it up, or hit Enter and then select Boot Manager from the menu. I haven't looked deeply at other systems, so I'm not sure how common it is to have it hidden.

With safeboot we found it easier to use LVM volume names instead of UUID, which also makes it possible to sign the kernel + initrd externally and have it work on different machines. For Qubes it might make acceptable to say "default installs may use the Qubes distribution key" and have more advanced users enroll their own key (or turn off SecureBoot).

Handling the TPM sealed secrets in the distribution key case is a little trickier; the signed PCR technique wouldn't work since you don't know what the PCRs are supposed to be other than the unified xen file. There is a fall back technique of sealing based on the current PCRs and storing the sealed data in the TPM NVRAM, however, it would require the disk recovery key to be available for every update since the new xen/kernel/initrd will change the measurements. That's not wonderful from a user perspective since they will need both the UEFI key (hardware token) as well as the disk recovery key in order to do an update.

If dom0 is running without lockdown, then potentially the current key can be retrieved from the running kernel, although that opens up some other security concerns...

@marmarek
Copy link
Member

marmarek commented Aug 4, 2020

easier to use LVM volume names instead of UUID

Qubes uses LVM names too. I'm referring to the LUKS partition UUID.

@andrewdavidwong andrewdavidwong added the P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. label Aug 7, 2020
@osresearch
Copy link

osresearch commented Aug 7, 2020

qubes-secureboot

As a proof of concept I have Qubes 4.1 booting with a unified Xen 4.13 on real hardware, where the hypervisor, vmlinuz, initrd, and configuration are signed with my own Yubikey. The xen.cfg file is hand-converted from the grub config (with some extra lockdown and iommu parameters)::

[global]
default=qubes-verbose

[qubes-verbose]
options=placeholder console=none dom0_mem=min:1024M dom0_mem=max:4096M ucode=scan smt=off gnttab_max_frames=2048 gnttab_max_maptrack_frames=4096 no-real-mode edd=off efi=attr=uc
noexitboot=1
mapbs=1
kernel=vmlinuz placeholder root=/dev/mapper/qubes_dom0-root ro rd.luks.uuid=luks-f03ca0fb-2c19-4f83-9ae7-88bf6c2faf19 rd.lvm.lv=qubes_dom0/root rd.lvm.lv=qubes_dom0/swap plymouth.ignore-serial-consoles rd.driver.pre=btrfs rhgb lockdown=confidentiality intel_iommu=on efi=disable_early_pci_dma
ramdisk=initrd.img

For the initial install I signed my own version of the DVD, which worked for the first phase, but the reboot to finalize the install failed since the xen it installed into the boot partition was not signed. So I turned off secure boot, booted into Qubes to finish the install, then rebooted into running Qubes, unified+signed the resulting kernel and initrd, and then turned secure boot back on.

Not the easiest workflow... I think it can be made doable for casual users if there is a way to not include the rd.luks.uuid= in the config file, and if the initrd can be pre-built. Does the resulting one after the dom0 install vary, or would it be possible to build a generic dom0 initrd that could be signed as part of the distribution?

@marmarek
Copy link
Member

marmarek commented Aug 7, 2020

(with some extra lockdown and iommu parameters)

I don't think IOMMU parameters to the dom0 kernel makes any difference - IOMMU is managed by Xen.
Also, noexitboot=1, mapbs=1 should not be needed anymore.

I think it can be made doable for casual users if there is a way to not include the rd.luks.uuid= in the config file

Theoretically we can configure installer to use constant UUID. But we risk collisions then...

Another thing that is installation-dependent (either on kernel cmdline, or built into initramfs) is the keyboard layout to use during LUKS passphrase prompt. There can be few more similar options.

Does the resulting one after the dom0 install vary

Currently yes (includes a bunch of configs from the target system), but I think it can be made generic quite easily (at least --no-hostonly, perhaps few other options too).
But the issue with few configuration parameters remains. Is there any place we could use to store some extra parameters, that do not also give too much control over the system? Some place that could be parsed by a dedicated script inside initramfs and nothing else - perhaps patch xen.efi to save efi stub parameters (but do not parse as xen options)? Or some EFI variables?

For the initial install I signed my own version of the DVD

Have you booted it from real DVD, or USB stick? I guess the latter. The ISO9660 format has limitation of 32MB for EFI boot partition. Fitting Xen+Linux+initramfs there was a major PITA. Now we can use grub2-efi to load some of the files from other place. Can you confirm the unified efi binary will also work this way (with either signed grub2-efi or shim+grub2-efi)?

@osresearch
Copy link

osresearch commented Aug 23, 2020

I've only booted from a USB stick; haven't had a physical DVD in years... The EFI executable should work from grub, although I don't have an easy way to test that on my system.

The additional configuration could go into EFI variables if they are not sensitive (in either sense), with some well tested parsers in the initrd. For instance with the change to add signed PCR policy, safeboot stores the signature in a UEFI variable (osresearch/safeboot#58). If an attacker changes it, the worst they can do is prevent the TPM from unsealing the disk key (the size is limited and the data is a raw binary signature). There are other things that are nice to have, like rollback protection, which required storing counters in the NVRAM (so that the secrets can be unsealed by newer signed images, but not by older ones) (osresearch/safeboot#62).

If the configuration values are sensitive, putting sealed blobs into NVRAM variables and using the TPM to unseal them is another option, although that does add an additional hardware dependency into the mix.

@osresearch
Copy link

osresearch commented Aug 28, 2020

The unified Xen patches are making their way through the xen-devel mailing list after several revisions and comments from the developers. https://lists.xenproject.org/archives/html/xen-devel/2020-08/msg01735.html

Backporting the fixes to the Xen used in Qubes 4.1 (and even 4.0) is quite easy since there isn't a huge amount of churn in the EFI boot path. Even if the Qubes distribution doesn't use the UEFI Secure Boot aspects, having the unified patches built into the mainline Qubes xen-vmm module would make it possible for advanced users to enable it for their own machines without having to rebuild the hypervisor.

Also, there was quite a bit of discussion about what Secure Boot compliance would mean for Xen overall. There are quite a few features that need to be turned off in order to avoid a local attacker from modifying the hypervisor. Andrew Cooper wrote in https://lists.xenproject.org/archives/html/xen-devel/2020-08/msg00290.html:

Things like LIVEPATCH and KEXEC need compiling out, until they can be taught to verify signatures.
Beyond that, things like the GDB serial stub probably need a way of being able to be compiled out, and then being compiled out. (This is definitely not an exhaustive list.)
Xen's secureboot requirements also extend to the dom0 kernel, due to the responsibility-sharing which currently exists. For a Linux dom0, Xen must ensure that lockdown mode is forced on (/dev/mem in dom0 still has a lot of system level power). At a minimum, this involves extending lockdown mode to prohibit the use of /{dev/proc}/xen/privcmd, which is still a trivial privilege escalation hole in PV Linux that noone seems to want to admit to and fix.
I think it is great that work is being started in this direction, but there is a huge quantity of work to do before a downstream could plausibly put together a Xen system which honours the intent of SecureBoot.
I know Safeboot has different goals/rules here, but whatever we put together called "Secure Boot support" will have to be compatible with Microsoft's model for it to be useful in the general case.

I think the Qubes audience doesn't need or want a strict Microsoft-definition of Secure Boot, and hope that with these patches plus some configuration updates it will be possible for users to better protect their systems.

@marmarek
Copy link
Member

marmarek commented Aug 28, 2020

Backporting the fixes to the Xen used in Qubes 4.1 (and even 4.0) is quite easy since there isn't a huge amount of churn in the EFI boot path. Even if the Qubes distribution doesn't use the UEFI Secure Boot aspects, having the unified patches built into the mainline Qubes xen-vmm module would make it possible for advanced users to enable it for their own machines without having to rebuild the hypervisor.

Yes, if there are no major incompatibilities, we can definitely do that.

I think the Qubes audience doesn't need or want a strict Microsoft-definition of Secure Boot, and hope that with these patches plus some configuration updates it will be possible for users to better protect their systems.

Yes, exactly. While having Microsoft-signed boot binary (whether with shim or some other solution) would ease installing Qubes on UEFI-enabled systems, it doesn't much affect the security aspect compared to your own keys - in fact, using own keys may be even better option (for example without grub and trusting MS keys we avoid issues like BootHole).

@osresearch
Copy link

osresearch commented Oct 9, 2020

The unified Xen hypervisor patches have finally made it through the mailing list and have almost all been pushed to staging! https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=8a71d50ed40bfa78c37722dc11995ac2563662c3

For a backport, do you prefer the individual commits (there are six patches in total) or is a single squashed patch easier?

@marmarek
Copy link
Member

marmarek commented Oct 9, 2020

Congratulation!
Preferably as landed upstream (individual commits).

@osresearch
Copy link

osresearch commented Oct 9, 2020

There were a few modifications required to back port it due to Makefile changes, as well as a few extra patches related to const* fixes in the efi code. Draft PR is QubesOS/qubes-vmm-xen#89

@tlaurion
Copy link
Contributor

tlaurion commented Jan 17, 2021

@osresearch @marmarek :
When comparing two root snapshots per

[user@dom0 ~]$ cat /lib/systemd/system-shutdown/root-autosnap 
#!/bin/sh

#This permits wyng-backup to backup root-autosnap and root-autosnap-back, taken at each system shutdowns like any other QubesOS LVMs.
#This also permits to restore to different states of dom0 from Heads and compare dom0 between reboots

#TODO: backup /boot content into a LVM and apply same logic, corresponding to each dom0 snapshots
#https://github.com/tasket/wyng-backup/issues/63

#We delete the backup of last shutdown snapshot (last last shutdown)
/usr/sbin/lvremove --noudevsync --force -An qubes_dom0/root-autosnap-back || true
#We take a snapshot of root-autosnap into root-autosnap-back
/usr/sbin/lvcreate --noudevsync --ignoremonitoring -An -pr -s qubes_dom0/root-autosnap -n root-autosnap-back
#We remove root-autosnap
/usr/sbin/lvremove --noudevsync --force -An qubes_dom0/root-autosnap || true
#We create root-autosnap from root
/usr/sbin/lvcreate --noudevsync --ignoremonitoring -An -pr -s qubes_dom0/root -n root-autosnap

And then we compare the content of the filesystems, we see that:

  • /etc/libvirt/libxl
  • /etc/lvm
  • /etc/xdg/adjtime
  • /home
  • /root
  • /var

Would need to be out of root fs to be able to have a RO QubesOS dom0 with dmverity

@tlaurion
Copy link
Contributor

tlaurion commented May 7, 2021

@marmarek @osresearch input on #4371 (comment) ?

@DemiMarie
Copy link

DemiMarie commented May 7, 2021

@marmarek @osresearch input on #4371 (comment) ?

Personally I would be fine with moving these off of the root filesystem.

@tlaurion
Copy link
Contributor

tlaurion commented May 25, 2021

@marmarek ?

@jevank
Copy link

jevank commented Oct 31, 2021

I've submitted PR with initial Qubes OS support. Tested with Lenovo X1C6/C7

@grince
Copy link

grince commented Feb 6, 2022

Question: will the installer of Qubes also support secureboot? when I try to boot from an usb-stick (made with the iso-image of qubes-4.1) grub on the usb-stick doesn't get started; when I try it with a non-secureboot-laptop it starts fine. since I want to use it an a secureboot-laptop I'd very much interested in having this feature ;)

I am willing to test it too, so if you have a modified ISO I shall test I am happy to do this. I already installed Debian on that system which kernels (and apparently installer) are secure-boot able if you choose "Microsoft and 3rd party certificates" as parameter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C: other help wanted This issue will probably not get done without help from community contributors. P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. security This issue pertains to the security of Qubes OS. T: enhancement Type: enhancement. A new feature that does not yet exist or improvement of existing functionality.
Projects
None yet
Development

No branches or pull requests

8 participants