Boot xen.efi directly, bypass grub in EFI mode

[marmarek]

Qubes OS version:

R4.0

Description

xen.efi is rather picky about its environment and in fails to load when grub uses own filesystem drivers instead of EFI services. There is very little control over that mechanism in grub (see #794 (comment)). On many systems, booting xen.efi without grub (using rEFInd, EFI shell, or simply by renaming it over BOOTX64.efi) helps with boot problems. So, we decided to get rid of the grub, boot xen.efi directly, but enable known UEFI workarounds by default (dropping grub is one of them ;) ). This should be the most convenient for most users, even those having buggy UEFI.
The missing thing will be boot menu for choosing custom options or "rescue mode". But it is still possible to use installer for rescuing system - you just need to switch to the second console (alt-ctrl-f2) instead following installer instructions. And since UEFI workarounds will be enabled by default, most use cases for custom boot options should be handled.

And if that still isn't enough for some machine, there are still old workarounds available - including using rEFInd or modifying image as described above (both kernel and xen options). Or booting in legacy mode, where you have menu to modify boot options.

See this thread for details.

[qubesos-bot]

Automated announcement from builder-github

The package pykickstart-2.32-4.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package pykickstart-2.32-4.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package pykickstart-2.32-4.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package pykickstart-2.32-4.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[eduncan911]

@marmarek I've been waiting a long time for this!

How can I download a nightly build and test these changes for you?

I have been struggling getting Qubes R3.2 and now 4.0 rc2 and 4.0 rc4 loaded on two machines for some time now - and never have been successful.

I do have a Thinkpad Helix running R3.2. but can't create a rescue disk from it for the 4.0 rc4 stuff.

The two machines are:

Mid-2014 MacBook Pro Retina 15": 4.0rc4 install just reboots to the GRUB> prompt. There is no xen.cfg, I tried to build one manually and boot it and/or the grub.cfg without success. Been trying several Fedora, Qubes and Xen efi hacks without success.

I just don't know the format and directory structure 4.0 Xen and Fedora wants for EFI.

The other machine is an Asus Rampage IV BE desktop. The installer just doesn't boot under EFI - it just immediately power cycles and flashes to quick to see anything. Disabling EFI for Legacy BIOS has several issues (freezing during gui loads, freezing during package installs and even freezing sometimes during Xen init loading for the installer.

Ultimately, my goal has been to load Qubes on an SFF M.2 external SSD I have in an USB3 enclosure, so I can switch out machines (and test Qubes on other machines). Three machines I have are all EFI only, with the Asus and Thinkpad the only two yhst have Legacy options.

However, I'd be happy just to have 4.0 loaded on the MacBook Pro's SSD since it is my daily driver and absorbs most of my abuse and travels. I just get past the EFI portion and do not want to load rEFI orbany other multi-boot hack.

I'll try to keep digging and reading up on your work.

However if there is anything I can do to test a nightly build, please let me know!

[eduncan911]

@marmarek I've been waiting a long time for this!

How can I download a nightly build and test these changes for you?

I have been struggling getting Qubes R3.2 and now 4.0 rc2 and 4.0 rc4 loaded on two machines for some time now - and never have been successful.

I do have a Thinkpad Helix running R3.2. but can't create a rescue disk from it for the 4.0 rc4 stuff.

The two machines are:

Mid-2014 MacBook Pro Retina 15": 4.0rc4 install just reboots to the GRUB> prompt. There is no xen.cfg, I tried to build one manually and boot it and/or the grub.cfg without success. Been trying several Fedora, Qubes and Xen efi hacks without success.

I just don't know the format and directory structure 4.0 Xen and Fedora wants for EFI.

The other machine is an Asus Rampage IV BE desktop. The installer just doesn't boot under EFI - it just immediately power cycles and flashes to quick to see anything. Disabling EFI for Legacy BIOS has several issues (freezing during gui loads, freezing during package installs and even freezing sometimes during Xen init loading for the installer.

Ultimately, my goal has been to load Qubes on an SFF M.2 external SSD I have in an USB3 enclosure, so I can switch out machines (and test Qubes on other machines). Three machines I have are all EFI only, with the Asus and Thinkpad the only two yhst have Legacy options.

However, I'd be happy just to have 4.0 loaded on the MacBook Pro's SSD since it is my daily driver and absorbs most of my abuse and travels. I just get past the EFI portion and do not want to load rEFI orbany other multi-boot hack.

I'll try to keep digging and reading up on your work.

However if there is anything I can do to test a nightly build, please let me know!

[marmarek]

How can I download a nightly build and test these changes for you?

This is already included in 4.0rc4.

Mid-2014 MacBook Pro Retina 15": 4.0rc4 install just reboots to the GRUB> prompt.

How did you get this? There is no grub on installation media anymore. There is:

• xen.efi directly (named BOOTX64.efi) for EFI boot
• isolinux for legacy boot

The only case where you can get grub, is when you install the system in legacy mode. Then you'll have grub on your hard drive.

[marmarek]

How can I download a nightly build and test these changes for you?

This is already included in 4.0rc4.

Mid-2014 MacBook Pro Retina 15": 4.0rc4 install just reboots to the GRUB> prompt.

How did you get this? There is no grub on installation media anymore. There is:

• xen.efi directly (named BOOTX64.efi) for EFI boot
• isolinux for legacy boot

The only case where you can get grub, is when you install the system in legacy mode. Then you'll have grub on your hard drive.

[eduncan911]

Already in 4.0rc4? And no more grub? That seems pretty odd, yes.

I only have macOS on my laptop. Nothing else. I've never had Linux on this laptop before.

I just downloaded rc4 again this morning, went through the installer, and it shows grub> after the installer is done and system reboots.

If I remove the USB devices, no EFI option appears under Macs book manager (hold OPTION on power up).

Plug in the freshly formatted and installed Qubes USB HDD (where I installed Qubes 4.0 rc4 to), hold OPTION while power on, and EFI option appears. I select the new EFI entry, the light on the USB stick flashes and grub> prompt immediately appears.

How grub is getting installed must be from the Qubes installer and it doesn't show anywhere else?

However, running efibootmgr under chroot of the installer does show a number of entries (from memory):

• 0001 Qubes
• 0080 macOS
• 0081 macOS
• 0000 pci device

I've deleted the Qubes entry a few times, and manually ran the efibootmgr to get an entry, and order, set right without success.

I've hesitated paving my entire SSD to install on local disk. But I can if you would like.

Note: the USB3 HDD is one that uses the ASMedia chipset to emulate an external HDD, not a removable disk. I have a couple of M.2 SSDs that I can install into it, and been trying to get Qubes installed.

It shows up as /dev/sdd FYI

[eduncan911]

Already in 4.0rc4? And no more grub? That seems pretty odd, yes.

I only have macOS on my laptop. Nothing else. I've never had Linux on this laptop before.

I just downloaded rc4 again this morning, went through the installer, and it shows grub> after the installer is done and system reboots.

If I remove the USB devices, no EFI option appears under Macs book manager (hold OPTION on power up).

Plug in the freshly formatted and installed Qubes USB HDD (where I installed Qubes 4.0 rc4 to), hold OPTION while power on, and EFI option appears. I select the new EFI entry, the light on the USB stick flashes and grub> prompt immediately appears.

How grub is getting installed must be from the Qubes installer and it doesn't show anywhere else?

However, running efibootmgr under chroot of the installer does show a number of entries (from memory):

• 0001 Qubes
• 0080 macOS
• 0081 macOS
• 0000 pci device

I've deleted the Qubes entry a few times, and manually ran the efibootmgr to get an entry, and order, set right without success.

I've hesitated paving my entire SSD to install on local disk. But I can if you would like.

Note: the USB3 HDD is one that uses the ASMedia chipset to emulate an external HDD, not a removable disk. I have a couple of M.2 SSDs that I can install into it, and been trying to get Qubes installed.

It shows up as /dev/sdd FYI

[marmarek]

I just downloaded rc4 again this morning, went through the installer, and it shows grub> after the installer is done and system reboots.

This means you've installed in legacy mode. Or some leftover from previous installation. If you install on external USB anyway and don't have anything valuable there, try zeroing first block first (boot record and partition table):

dd if=/dev/zero of=/dev/sdX bs=512 count=1

Double check if that's the right device before doing so

You can also try this: https://www.qubes-os.org/doc/uefi-troubleshooting/#boot-device-not-recognized-after-installing

[marmarek]

I just downloaded rc4 again this morning, went through the installer, and it shows grub> after the installer is done and system reboots.

This means you've installed in legacy mode. Or some leftover from previous installation. If you install on external USB anyway and don't have anything valuable there, try zeroing first block first (boot record and partition table):

dd if=/dev/zero of=/dev/sdX bs=512 count=1

Double check if that's the right device before doing so

You can also try this: https://www.qubes-os.org/doc/uefi-troubleshooting/#boot-device-not-recognized-after-installing

[eduncan911]

(Messaged you on Keybase)

I zero'd out the first block on both the USB installer stick as well as the external HDD device. Confirmed that performing this cleared all partitions on both devices.

Exact same result. Grub gets installed onto the external HDD.

Note that Grub2 is not used on the installer, which I think your commit actually removed - the installer now will boot directly via EFI with no "Test Media" option. This is now the ONLY way the installer loads on the Macbook Pro!!! You fixed it! 👍

However, it is not installing via Legacy mode. I know what Legacy mode looks like with Grub2 as I tried that a few times on my Asus motherboard-equipped desktop. I've seen the boot options, Rescue mode option, etc etc. None of that shows when installing on the Macbook Pro.

I do not get Grub2, nor any boot options, nor any "Test Media and Install" option when booting the EFI option on the Macbook Pro. It is all under EFI.

However, at the end of the installation, I reboot and there sits Grub2 on the external HDD - and I get the dreaded grub> prompt.

[eduncan911]

(Messaged you on Keybase)

I zero'd out the first block on both the USB installer stick as well as the external HDD device. Confirmed that performing this cleared all partitions on both devices.

Exact same result. Grub gets installed onto the external HDD.

Note that Grub2 is not used on the installer, which I think your commit actually removed - the installer now will boot directly via EFI with no "Test Media" option. This is now the ONLY way the installer loads on the Macbook Pro!!! You fixed it! 👍

However, it is not installing via Legacy mode. I know what Legacy mode looks like with Grub2 as I tried that a few times on my Asus motherboard-equipped desktop. I've seen the boot options, Rescue mode option, etc etc. None of that shows when installing on the Macbook Pro.

I do not get Grub2, nor any boot options, nor any "Test Media and Install" option when booting the EFI option on the Macbook Pro. It is all under EFI.

However, at the end of the installation, I reboot and there sits Grub2 on the external HDD - and I get the dreaded grub> prompt.

[marmarek]

However, at the end of the installation, I reboot and there sits Grub2 on the external HDD - and I get the dreaded grub> prompt.

Ok, that's really weird...

[marmarek]

However, at the end of the installation, I reboot and there sits Grub2 on the external HDD - and I get the dreaded grub> prompt.

Ok, that's really weird...

[marmarek]

Results of debugging with @eduncan911 : there is separate handler in anaconda for Mac EFI, which indeed install grub. And is broken, because expect Fedora not Qubes, Linux not Xen etc.

[marmarek]

Results of debugging with @eduncan911 : there is separate handler in anaconda for Mac EFI, which indeed install grub. And is broken, because expect Fedora not Qubes, Linux not Xen etc.

[eduncan911]

Thanks @marmarek !

I'm also currently working on a PR to fix it for my 2x Macbook Pro Retinas (mid-2014, 2016 w/touchbar) and 1x late-2015 iMac 27" I'll test it all with first.

Testing is slow going, and we need to be very careful not to break anything this late in the game before any additional releases since there already has been several 4.0 RCs and we don't want to break the next release.

[eduncan911]

Thanks @marmarek !

I'm also currently working on a PR to fix it for my 2x Macbook Pro Retinas (mid-2014, 2016 w/touchbar) and 1x late-2015 iMac 27" I'll test it all with first.

Testing is slow going, and we need to be very careful not to break anything this late in the game before any additional releases since there already has been several 4.0 RCs and we don't want to break the next release.

[eduncan911]

Thanks to @marmarek , Apple hardware has been fixed: QubesOS/qubes-installer-qubes-os#20

[eduncan911]

Thanks to @marmarek , Apple hardware has been fixed: QubesOS/qubes-installer-qubes-os#20