Qubes 4.0 is not booting with boot parameters "lockdown=integrity" or "lockdown=confidentiality"

[n1m1]

Qubes OS version:

Qubes 4.0 with AEM enabled.

Affected component(s) or functionality:

Boot.

---

Steps to reproduce the behavior:

0. Install kernel 5.4.1 in Dom0
1. Set lockdown=integrity or lockdown=confidentuality in /etc/default/grub
2. grub2-mkconfig -o /boot/grub2/grub.cfg
3. Reboot the system with kernel 5.4

Expected or desired behavior:

The system boots and seals the keys for AEM.

Actual behavior:

The system hangs on. On the monitor I can see

Waiting for device XXX to be attached

Nevertheless, the USB drive with /boot partition is ALREADY attached.

I've also tried to boot the system from the USB drive without AEM, but without success.

General notes:

It is totally possible that this is not a bug, but it is actually how the lockdown feature is supposed to work (and thus, it is not compatible with AEM or booting with an external device). TBH I have no idea and I am not tech-savvy enough to give an answer to this question.

---

I have consulted the following relevant documentation:

I am aware of the following related, non-duplicate issues:

[marmarek]

We don't fully support kernel lockdown yet (that's why it's disabled by default).

Can you give more details about the problem? Try booting without quiet and rhgh kernel options to see more messages.

[n1m1]

Hi,
this is the message which I can see when I boot the system without quiet and `rhgh' parameters:

Lockdown: systemd-udev: unsigned module loading is restricted: see man kernel_lockdown.7

(which apparently can't be found with man on Dom0 : No manual entry for kernel_lockdown.7

If this is not enough, please, tell me how to save the log messages which appears during the boot process (I am using AEM with an external USB drive).

Cheers.

[marmarek]

That's interesting, indeed dom0 kernel modules in this package are not signed, but VM modules built at the same time are. Something is wrong with the packaging process.

[marmarek]

Looks like signatures are stripped together with debug symbols...
Upstream Fedora package has some code to sign modules again after handling debug info, I'll include that.

[marmarek]

While the above solves module signatures issue, lockdown=integrity still breaks Qubes, specifically PCI passthrough. Libvirt daemon is prevented from configuring PCI devices:

Lockdown: libvirtd: direct PCI access is restricted; see man kernel_lockdown.7

This makes it impossible to start sys-net and sys-usb. I'll need to look into it deeper, but I'm afraid there may be no nice way around it. The not nice way would be disabling this particular restriction.

[n1m1]

While the above solves module signatures issue, lockdown=integrity still breaks Qubes, specifically PCI passthrough. Libvirt daemon is prevented from configuring PCI devices:

Lockdown: libvirtd: direct PCI access is restricted; see man kernel_lockdown.7

This makes it impossible to start sys-net and sys-usb. I'll need to look into it deeper, but I'm afraid there may be no nice way around it. The not nice way would be disabling this particular restriction.

Hello and thanks for the time you have spent in order to investigate the issue. Apologies for my ignorance but I am curious about it: does this mean that lockdown is not compatible with Qubes OS?

Cheers

[marmarek]

Yes, this is what it means, at least right now.
You can try using lockdown in VMs (modify kernelopts property).

[n1m1]

Hi, thanks for your answer.
I have enabled lockdown=integrity in a VM. It boots as expected. According to dmesg

[ 0.000000] Kernel is locked down from command line; see man kernel_lockdown.7

[qubesos-bot]

Automated announcement from builder-github

The package kernel-latest-5.4.2-1.qubes has been pushed to the r4.1 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package kernel-latest-5.4.2-1.qubes has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[n1m1]

Hello,
I am running linux-kernel-latest v5.4.2-1 . Apparently it has solved the issue. I have been able to boot Dom0 with lockdown=integrity and lockdown=confidentiality without problems. In both cases sys-usb and sys-net were running as expected.

Now, I am using lockdown in integrity mode. According to dmesg | grep lockdown (in Dom0):

[    2.021825] Lockdown: systemd: /dev/mem,kmem,port is restricted; see man kernel_lockdown.7
[   38.082721] Lockdown: systemd: /dev/mem,kmem,port is restricted; see man kernel_lockdown.7
[   78.846228] Lockdown: Xorg: raw io port access is restricted; see man kernel_lockdown.7
[   79.421837] Lockdown: x86_energy_perf: raw MSR access is restricted; see man kernel_lockdown.7

Cheers

[marmarek]

That's interesting, maybe libvirt need to access PCI config space depends on the hardware (and options like permissive mode, strict reset). We'll need to investigate it if/when we'll want to enable it by default.
Anyway, I'm glad it solved the problem for you.

[n1m1]

@marmarek , thanks a lot for your help. I will keep testing lockdown in the next days. If something changes, I will report it here promptly.

[n1m1]

Hi,
another small bit of information related to this issue (not sure it is useful). I have enabled lockdown=integrity also in sys-usb and sys-net throught kernelopts. Both boot and work as expected. According to dmesg lockdown is working. However, if some issue emerges, I will open a new specific ticket in order to track it properly.

Cheers.

[qubesos-bot]

Automated announcement from builder-github

The package kernel-latest-5.4.10-1.qubes has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package kernel-5.4.21-1.qubes has been pushed to the r4.1 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package kernel-5.4.38-1.qubes has been pushed to the r4.1 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update