Support for Coreboot+SeaBIOS hardware

[kugg]

Qubes OS version: Qubes-R3.2-x86_64

Lenovo Thinkpad x220 i7-2620M

Coreboot 4.5

---

Expected behavior:

Booting from disk after bios stage.
The following log from coreboot shows the same system successfully booting on a (non XEN) live distribution (Tails): https://paste.debian.net/hidden/fdc9fc95/

Note: To add debug prints I had to build another coreboot flash with spkrmodem hence the differing versions and dates. The behavior trying to boot Qubes from disk was the same on both versions of coreboot.

Actual behavior:

Coreboot SeaBIOS cursor blinks OS does not start.

Output is:
Press ESC for boot menu.
(pressing ESC)
Select boot device:

1. AHCI/0: INTEL SSDSA2BW160G3L ATA-8 Hard-Disk (149 GiBytes)
2. USB MSC Drive Kingston DataTraveler 3.0 PMAP
3. iPXE (PCI 00:19.0)
   (pressing 1)

Booting from Hard Disk...
Cursor keeps blinking, nothing boots.
The following log from coreboot shows a boot failure trying to boot the successfully installed Qubes OS R3.2 from disk: https://paste.debian.net/906598/

Steps to reproduce the behavior:

1. Install coreboot 4.5 with SeaBIOS payload on Lenovo Thinkpad x220
   1.2 Turn on debuging using either spkrmodem (and record/wait for about 5 hours for bios to boot) or use a EHCI debugger (https://www.coreboot.org/EHCI_Debug_Port). Configure either: CONFIG_HAVE_USBDEBUG=y || CONFIG_SPKMODEM=y
2. Install Qubes 3.2 on disk.
3. Boot from disk in SeaBIOS.

General notes:

I have followed the discussion in: #1594
I have tried to put iommu=0 in xen.conf with no success.
I have also unsuccessfully tried to add the options suggested in: https://www.qubes-os.org/doc/uefi-troubleshooting/

---

Related issues:

#1594

[andrewdavidwong]

This is too localized for qubes-issues. Please consider posting this to the qubes-users mailing list instead. (You can read more about our mailing lists here.)

qubes-users is intended for these sorts of questions and receives much more traffic, which means that your question is more likely to receive a response there.

[andrewdavidwong]

This is too localized for qubes-issues. Please consider posting this to the qubes-users mailing list instead. (You can read more about our mailing lists here.)

qubes-users is intended for these sorts of questions and receives much more traffic, which means that your question is more likely to receive a response there.

[rustybird]

@andrewdavidwong: I think this should be reopened, because the installer bug affects all coreboot+SeaBIOS users.

Since QubesOS/qubes-installer-qubes-os@1479416, GRUB is not installed on coreboot systems. Nor are Xen/kernel parameters etc. configured in /etc/default/grub, hence e.g. the non-Plymouth boot after calling grub2-install manually.

But if SeaBIOS is running, GRUB actually needs to be configured and installed. I'm working on a patch (seabios branch, untested!) that attempts to detect SeaBIOS and also adds a skip_grub={0,1} boot parameter to override. Does someone have a coreboot-without-SeaBIOS installation around (maybe @tlaurion) to verify that biosdecode (as root) doesn't print the line BIOS32 Service Directory present.?

Meanwhile, a good workaround for SeaBIOS users is to switch to a terminal in the installer and effectively disable coreboot detection: ln -sf /bin/true /usr/sbin/dmidecode && systemctl restart anaconda

[rustybird]

@andrewdavidwong: I think this should be reopened, because the installer bug affects all coreboot+SeaBIOS users.

Since QubesOS/qubes-installer-qubes-os@1479416, GRUB is not installed on coreboot systems. Nor are Xen/kernel parameters etc. configured in /etc/default/grub, hence e.g. the non-Plymouth boot after calling grub2-install manually.

But if SeaBIOS is running, GRUB actually needs to be configured and installed. I'm working on a patch (seabios branch, untested!) that attempts to detect SeaBIOS and also adds a skip_grub={0,1} boot parameter to override. Does someone have a coreboot-without-SeaBIOS installation around (maybe @tlaurion) to verify that biosdecode (as root) doesn't print the line BIOS32 Service Directory present.?

Meanwhile, a good workaround for SeaBIOS users is to switch to a terminal in the installer and effectively disable coreboot detection: ln -sf /bin/true /usr/sbin/dmidecode && systemctl restart anaconda

[andrewdavidwong]

I think this should be reopened, because the installer bug affects all coreboot+SeaBIOS users.

Thanks, @rustybird. Reopened and updated title.

[andrewdavidwong]

I think this should be reopened, because the installer bug affects all coreboot+SeaBIOS users.

Thanks, @rustybird. Reopened and updated title.

[mfc]

FYI easiest workaround is in this HCL report:

1. boot from your Qubes R3.2 installation media
2. go to: Troubleshooting -> Rescue a Qubes system
3. after booting to anaconda installer please choose 1) Continue (and enter your Qubes partition password if you chose to encrypt it during install)
4. then enter chroot /mnt/sysimage
5. finally enter grub2-install /dev/sdX (where X is a letter for your hard drive with Qubes, for example: grub2-install /dev/sda)

now exit chroot with exit, exit bash with exit and reboot.

[mfc]

FYI easiest workaround is in this HCL report:

1. boot from your Qubes R3.2 installation media
2. go to: Troubleshooting -> Rescue a Qubes system
3. after booting to anaconda installer please choose 1) Continue (and enter your Qubes partition password if you chose to encrypt it during install)
4. then enter chroot /mnt/sysimage
5. finally enter grub2-install /dev/sdX (where X is a letter for your hard drive with Qubes, for example: grub2-install /dev/sda)

now exit chroot with exit, exit bash with exit and reboot.

[rustybird]

The workaround from the HCL report will leave the system in an undefined state:

Nor are Xen/kernel parameters etc. configured in /etc/default/grub, hence e.g. the non-Plymouth boot after calling grub2-install manually.

Note that some of those parameters, such as dom0_mem, are not just cosmetical.

Hey do you still happen to have access to a coreboot-without-SeaBIOS installation @mfc?

[rustybird]

The workaround from the HCL report will leave the system in an undefined state:

Nor are Xen/kernel parameters etc. configured in /etc/default/grub, hence e.g. the non-Plymouth boot after calling grub2-install manually.

Note that some of those parameters, such as dom0_mem, are not just cosmetical.

Hey do you still happen to have access to a coreboot-without-SeaBIOS installation @mfc?

[mfc]

this was for someone else, should i have also disabled coreboot detection like you suggested? or edited a bootfile for the necessary parameters? not interested in re-flashing SeaBIOS for them.

i run a heads machine which is coreboot without SeaBIOS. if i run biosdecode (as root) in dom0 it does not print the line BIOS32 Service Directory present, only ACPI 2.0 present and SMBIOS 2.7 present (with further details of each).

[mfc]

this was for someone else, should i have also disabled coreboot detection like you suggested? or edited a bootfile for the necessary parameters? not interested in re-flashing SeaBIOS for them.

i run a heads machine which is coreboot without SeaBIOS. if i run biosdecode (as root) in dom0 it does not print the line BIOS32 Service Directory present, only ACPI 2.0 present and SMBIOS 2.7 present (with further details of each).

[rustybird]

this was for someone else, should i have also disabled coreboot detection like you suggested? or edited a bootfile for the necessary parameters?

Just disabling the Qubes installer's coreboot detection would be the cleanest.

If you don't want to reinstall, I'd at least add the line GRUB_CMDLINE_XEN_DEFAULT="console=none dom0_mem=min:1024M dom0_mem=max:4096M" and rerun grub2-mkconfig. Not sure what to do about GRUB_CMDLINE_LINUX, because it is generated dynamically, depending on installation layout and LUKS UUIDs. But apparently it's not essential (unless they want to try AEM).

i run a heads machine which is coreboot without SeaBIOS. if i run biosdecode (as root) in dom0 it does not print the line BIOS32 Service Directory present, only ACPI 2.0 present and SMBIOS 2.7 present (with further details of each).

Great, thanks!

[rustybird]

this was for someone else, should i have also disabled coreboot detection like you suggested? or edited a bootfile for the necessary parameters?

Just disabling the Qubes installer's coreboot detection would be the cleanest.

If you don't want to reinstall, I'd at least add the line GRUB_CMDLINE_XEN_DEFAULT="console=none dom0_mem=min:1024M dom0_mem=max:4096M" and rerun grub2-mkconfig. Not sure what to do about GRUB_CMDLINE_LINUX, because it is generated dynamically, depending on installation layout and LUKS UUIDs. But apparently it's not essential (unless they want to try AEM).

i run a heads machine which is coreboot without SeaBIOS. if i run biosdecode (as root) in dom0 it does not print the line BIOS32 Service Directory present, only ACPI 2.0 present and SMBIOS 2.7 present (with further details of each).

Great, thanks!

[marmarek]

There are two things:

• allowing /boot to be encrypted on coreboot(+grub2)
• not installing grub2 when it's already included as coreboot payload

Currently installed assumes that if coreboot is installed, it has grub2 payload, so does both of above things. Can we somehow detect grub2 payload presence (not only coreboot presence)? If not, I think at least second step should be disabled, so grub2 would be installed even if already included in coreboot payload.
Is the above (biosdecode | grep BIOS32) the way to detect SeaBIOS?

[marmarek]

There are two things:

• allowing /boot to be encrypted on coreboot(+grub2)
• not installing grub2 when it's already included as coreboot payload

Currently installed assumes that if coreboot is installed, it has grub2 payload, so does both of above things. Can we somehow detect grub2 payload presence (not only coreboot presence)? If not, I think at least second step should be disabled, so grub2 would be installed even if already included in coreboot payload.
Is the above (biosdecode | grep BIOS32) the way to detect SeaBIOS?

[rustybird]

• allowing /boot to be encrypted on coreboot(+grub2)

It is even encrypted automatically in this case, right?

Can we somehow detect grub2 payload presence (not only coreboot presence)?

No idea

Is the above (biosdecode | grep BIOS32) the way to detect SeaBIOS?

Yes, and it really is the wrong thing to check. For example, coreboot's GRUB payload could be chainloaded from SeaBIOS.

Maybe we should just scrap the automagic for now and add two independent boot parameters, encrypt_boot={0,1} and skip_grub={0,1}? There are just so many moving parts - coreboot, SeaBIOS, GRUB payload, @osresearch's Heads, #2442, ...

[rustybird]

• allowing /boot to be encrypted on coreboot(+grub2)

It is even encrypted automatically in this case, right?

Can we somehow detect grub2 payload presence (not only coreboot presence)?

No idea

Is the above (biosdecode | grep BIOS32) the way to detect SeaBIOS?

Yes, and it really is the wrong thing to check. For example, coreboot's GRUB payload could be chainloaded from SeaBIOS.

Maybe we should just scrap the automagic for now and add two independent boot parameters, encrypt_boot={0,1} and skip_grub={0,1}? There are just so many moving parts - coreboot, SeaBIOS, GRUB payload, @osresearch's Heads, #2442, ...

[marmarek]

allowing /boot to be encrypted on coreboot(+grub2)

It is even encrypted automatically in this case, right?

Yes.

Maybe we should just scrap the automagic for now and add two independent boot parameters, encrypt_boot={0,1} and skip_grub={0,1}? There are just so many moving parts - coreboot, SeaBIOS, GRUB payload, @osresearch's Heads, #2442, ...

Yes, it would be much less error-prone.

[marmarek]

allowing /boot to be encrypted on coreboot(+grub2)

It is even encrypted automatically in this case, right?

Yes.

Maybe we should just scrap the automagic for now and add two independent boot parameters, encrypt_boot={0,1} and skip_grub={0,1}? There are just so many moving parts - coreboot, SeaBIOS, GRUB payload, @osresearch's Heads, #2442, ...

Yes, it would be much less error-prone.

[rustybird]

OK, I'll submit a PR.

[rustybird]

OK, I'll submit a PR.

[rustybird]

Now I'm really confused:

1. The lines self.encryption_support = True and self.stage2_format_types += ["lvmlv"] do not automatically encrypt the boot partition. In retrospect that makes sense, because otherwise the "manual grub2-install" partial workaround for SeaBIOS would not be viable.

2. The two lines don't seem to change how Anaconda behaves for manual partitioning. With or without them, you can choose LVM and/or encryption for the boot partition. And there will actually be no encryption, even if you choose it.

3. I don't know how what to make of @tlaurion's instructions in #2118 (comment). Is the idea for Anaconda to create /boot as a regular folder on the encrypted root partition inside the LVM? If I follow the steps, enter / into the "Mount Point:" field for matrix-rootvol in the Manual Partitioning screen, and click Done, Anaconda clears the field and fails with "You have not defined a root partition (/)".

Has anyone successfully installed R3.2 with an encrypted boot partition?

[rustybird]

Now I'm really confused:

1. The lines self.encryption_support = True and self.stage2_format_types += ["lvmlv"] do not automatically encrypt the boot partition. In retrospect that makes sense, because otherwise the "manual grub2-install" partial workaround for SeaBIOS would not be viable.

2. The two lines don't seem to change how Anaconda behaves for manual partitioning. With or without them, you can choose LVM and/or encryption for the boot partition. And there will actually be no encryption, even if you choose it.

3. I don't know how what to make of @tlaurion's instructions in #2118 (comment). Is the idea for Anaconda to create /boot as a regular folder on the encrypted root partition inside the LVM? If I follow the steps, enter / into the "Mount Point:" field for matrix-rootvol in the Manual Partitioning screen, and click Done, Anaconda clears the field and fails with "You have not defined a root partition (/)".

Has anyone successfully installed R3.2 with an encrypted boot partition?

[marmarek]

In step "3" you need to click "update" first (right bottom corner). AFAIR without self.encryption_support = True and self.stage2_format_types += ["lvmlv"] Anaconda would complain that /boot cannot be on LVM and/or encrypted.
There was some method to make Anaconda not create separate /boot. Maybe it was some older version of this function?

[marmarek]

In step "3" you need to click "update" first (right bottom corner). AFAIR without self.encryption_support = True and self.stage2_format_types += ["lvmlv"] Anaconda would complain that /boot cannot be on LVM and/or encrypted.
There was some method to make Anaconda not create separate /boot. Maybe it was some older version of this function?

[rustybird]

In step "3" you need to click "update" first (right bottom corner).

This also just clears the Mount Point field for me, then I get the same error after clicking Done. Is it working for you?

[rustybird]

In step "3" you need to click "update" first (right bottom corner).

This also just clears the Mount Point field for me, then I get the same error after clicking Done. Is it working for you?

[marmarek]

I don't have coreboot machine handy, but just checked and setting "/" do work this way. If you're trying to re-use existing partition, you need to select also "reformat" checkbox (there is a warning at the bottom of screen about it, if you don't do that).

[marmarek]

I don't have coreboot machine handy, but just checked and setting "/" do work this way. If you're trying to re-use existing partition, you need to select also "reformat" checkbox (there is a warning at the bottom of screen about it, if you don't do that).

[rustybird]

Thanks! Somehow the checkbox looked greyed out to me. (PTSD intensifies)

[rustybird]

Thanks! Somehow the checkbox looked greyed out to me. (PTSD intensifies)

[DarkCoridor]

I installed Qubes 4 RC1 on a Lenovo x220 with Coreboot + SeaBIOS (1.9.1) successfully but booting stays stuck in endless boot loop. I tried the grub2-install /dev/sda trick (despite that being for Qubes 3.2?), however that does not make any different.

I know it's not my machine that has an issue, as it successfully runs Tails, Subgraph, and Debian. Help or pointers would be appreciated.

[DarkCoridor]

I installed Qubes 4 RC1 on a Lenovo x220 with Coreboot + SeaBIOS (1.9.1) successfully but booting stays stuck in endless boot loop. I tried the grub2-install /dev/sda trick (despite that being for Qubes 3.2?), however that does not make any different.

I know it's not my machine that has an issue, as it successfully runs Tails, Subgraph, and Debian. Help or pointers would be appreciated.

[marmarek]

Remove iommu=no-igfx option from xen parameters (in grub config).

[marmarek]

Remove iommu=no-igfx option from xen parameters (in grub config).

[DarkCoridor]

Thanks @marmarek I did that and it is no difference in the looping. Anything else I can try?

[DarkCoridor]

Thanks @marmarek I did that and it is no difference in the looping. Anything else I can try?

[h01ger]

On Sun, Sep 03, 2017 at 04:59:07PM -0700, DarkCoridor wrote: Thanks @marmarek I did that and it is no difference in the looping. Anything else I can try?

are you using the intel vga bios blob or the free firmware written in ADA? (i'm using the latter on my x230 and it works fine.)

…

-- cheers, Holger

[h01ger]

On Sun, Sep 03, 2017 at 04:59:07PM -0700, DarkCoridor wrote: Thanks @marmarek I did that and it is no difference in the looping. Anything else I can try?

are you using the intel vga bios blob or the free firmware written in ADA? (i'm using the latter on my x230 and it works fine.)

…

-- cheers, Holger

[DarkCoridor]

@h01ger I am unsure of that. Did you experience similar issue with one or other? Perhaps worth noting is I also used me_cleaner on the Coreboot BIOS

[DarkCoridor]

@h01ger I am unsure of that. Did you experience similar issue with one or other? Perhaps worth noting is I also used me_cleaner on the Coreboot BIOS

[h01ger]

On Mon, Sep 04, 2017 at 05:01:35PM +0000, DarkCoridor wrote: @h01ger I am unsure of that. Did you experience similar issue with one or other? Perhaps worth noting is I also used [me_cleaner](/corna/me_cleaner) on the Coreboot BIOS

I never used the intel vga bios blob (except with the original lenovo firmware of course) and have not yet any experience with me_cleaner.

…

-- cheers, Holger

[h01ger]

On Mon, Sep 04, 2017 at 05:01:35PM +0000, DarkCoridor wrote: @h01ger I am unsure of that. Did you experience similar issue with one or other? Perhaps worth noting is I also used [me_cleaner](/corna/me_cleaner) on the Coreboot BIOS

I never used the intel vga bios blob (except with the original lenovo firmware of course) and have not yet any experience with me_cleaner.

…

-- cheers, Holger

[zxyz]

I had the same problem as @DarkCoridor with my X220 and Qubes-Os 4.0 rc1 (corebot from master, Seabios from master and me_cleaner from master) using the free firmware. Using the VGA blob I get to see the grub menu, which is as expected - but the computer reboots as before. But editing the XEN parameters and removing the iommu=no-igfx option as suggested by @marmarek did the trick and Qubes finally boots.

For extracting the BIOS I followed:
https://nroach44.id.au/index.php/2016/12/11/thinkpad-x220-coreboot-and-me-removal/

[zxyz]

I had the same problem as @DarkCoridor with my X220 and Qubes-Os 4.0 rc1 (corebot from master, Seabios from master and me_cleaner from master) using the free firmware. Using the VGA blob I get to see the grub menu, which is as expected - but the computer reboots as before. But editing the XEN parameters and removing the iommu=no-igfx option as suggested by @marmarek did the trick and Qubes finally boots.

For extracting the BIOS I followed:
https://nroach44.id.au/index.php/2016/12/11/thinkpad-x220-coreboot-and-me-removal/