System boots into grub rescue

[MarcusWichelmann]

Hi,

after installing Debian Buster using your script on a SATA SSD ZFS-mirror the system boots into grub rescue.

If we run a normal (hd2,gpt3)/efi/debian/grub.cfg manually, we can boot into Debian successfully and fix this using grub-install and update-grub.

Do you have an idea what's wrong?

[ponasromas]

I ha e encountered similar problem with BIOS grub after adding 2 new disks into pool. And the problem that there is no grub at all after adding new disks.

[mjoconr]

I have the same problem, I was able to boot the system using the grub cli commands above (they where different.
normal (hd{0,1}),gpt2)/efi//grub.cfg
But I can not see how to use grub-install to fix the issue.
Reading the script it seems like I'm missing something.
Also I think a script is needed to update both drives in the mirror

[SoerenBusse]

I've currently spent about 6hours to figure out why this bug happens.
It seems like the debian team has changed the default behaviour of grub-install which now installs the signed grubx64.efi for secure boot by default. This grubx64.efi contains an early configuration which tries to load the grub.cfg from the EFI/debian directory. Normally the --bootloader-id parameter would change the embedded configuration so it will load the grub.cfg from the right directory which was specified in the bootloader-id parameter. This however isn't possible for a signed grubx64.efi, which will result in this bug we noticed here, because EFI/debian/grub.cfg simply doesn't exist.

When you know what you're looking for you find several people having the same issue:
http://forums.debian.net/viewtopic.php?f=5&t=141126&hilit=925309&start=0

There's also an bug report in the debian bug tracker, which doesn't seem to be fixed yet.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=%23925309

Workaround:
To support secure boot we need to change the bootloader-id to debian. This however has the disadvantage that we can't add a label like disk 1 or disk 2.

When we don't need secure boot we can add --no-uefi-secure-boot to grub-install

I think the best way would be to give the user the choice whether to use secure boot or not. What do you think @hn?

Edit:
When using debian as bootloader-id it only adds one of the drives to the NVRAM of the efi because the id must be unique. This is another issue we're facing when using secureboot

Edit2:
The best option would be to run grub-install with the --no-nvram option which prevents the entries to be written to the EFIs NVRAM. Instead you can run the command efibootmgr -c --label "Debian Buster Disk 0" --loader "\EFI\debian\shimx64.efi" --disk /dev/sda -p 2 (and for all other disks of course). This works absolutly fine in my environment.

[hn]

Hi @SoerenBusse, big thanks for investigating the issue! I'm quite busy right now, so it will probably take some days until I've time to understand all implications and to implement a fix.

Feel free to send patches :)

[SoerenBusse]

@hn I've already fixed the issue but while testing the fix before pushing I run into the following issue with QEMUs UEFI OVFM: Have you encountered anything like this before in your tests or have you tested on bare metal? https://bugzilla.tianocore.org/show_bug.cgi?id=2235

[gkisuze]

I've currently spent about 6hours to figure out why this bug happens.
It seems like the debian team has changed the default behaviour of grub-install which now installs the signed grubx64.efi for secure boot by default. This grubx64.efi contains an early configuration which tries to load the grub.cfg from the EFI/debian directory. Normally the --bootloader-id parameter would change the embedded configuration so it will load the grub.cfg from the right directory which was specified in the bootloader-id parameter. This however isn't possible for a signed grubx64.efi, which will result in this bug we noticed here, because EFI/debian/grub.cfg simply doesn't exist.

When you know what you're looking for you find several people having the same issue:
http://forums.debian.net/viewtopic.php?f=5&t=141126&hilit=925309&start=0

There's also an bug report in the debian bug tracker, which doesn't seem to be fixed yet.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=%23925309

Workaround:
To support secure boot we need to change the bootloader-id to debian. This however has the disadvantage that we can't add a label like disk 1 or disk 2.

When we don't need secure boot we can add --no-uefi-secure-boot to grub-install

I think the best way would be to give the user the choice whether to use secure boot or not. What do you think @hn?

Edit:
When using debian as bootloader-id it only adds one of the drives to the NVRAM of the efi because the id must be unique. This is another issue we're facing when using secureboot

Edit2:
The best option would be to run grub-install with the --no-nvram option which prevents the entries to be written to the EFIs NVRAM. Instead you can run the command efibootmgr -c --label "Debian Buster Disk 0" --loader "\EFI\debian\shimx64.efi" --disk /dev/sda -p 2 (and for all other disks of course). This works absolutly fine in my environment.

@SoerenBusse, @hn My setup installs ok when configured to install with legacy mode in BIOS using the script. It however doesn't allow to boot when installed in UEFI mode despite making appropriate changes in BIOS. The installation procedure is a bit unpredictable. It works okay on intel NUC i5, both UEFI and legacy while the script fails with dell optiplex 7040 micro in UEFI mode. For both machines, I've to manually run grub-install after the script has run through all the prompts to get a bootable environment.
Can you avail updates to the script to address all possible machines and both UEFI and legacy? I have limited experience with making changes to the bash scripts of this magnitude.
Thanks

[hn]

@SoerenBusse @gkisuze It seems there is no 'bullet proof' way to have UEFI working with the broad variety of hardware available on the market. I think it's best give the choice to the user for the boot mode: "Legacy BIOS" or "UEFI [EXPERIMENTAL]".

Unfortunately, I currently do not have time to implement and -- this might be the bigger part -- to thoroughly test those changes.

If you or others submit patches please keep them as small and unintrusive as possible and split them into multiple parts if possible.