MBR kexec failed due to "Cannot find the ESP partition mount point"

[enihcam]

Submission type

• Bug report

systemd version the issue has been seen with

236

Used distribution

Archlinux

In case of bug report: Expected behaviour you didn't see

'systemctl kexec' triggers the kexec boot successfully

In case of bug report: Unexpected behaviour you saw

'systemctl kexec' returns 'Cannot find the ESP partition mount point'

In case of bug report: Steps to reproduce the problem

Disable UEFI and boot the system in MBR mode; Load kexec module and kernel; Run 'systemctl kexec'

[yuwata]

Sorry, currentlysystemctl kexec requires the system is running with UEFI. We use systemd's boot loader config and ESP settings to determine which kernel and initrd are used.

[enihcam]

v235 had no such issue, but v236 release notes did not mention about this.

[yuwata]

4bb2e9d.

[yuwata]

Indeed. At least we should mention that in NEWS or somewhere.
@keszybz should we fallback to original process when the system does not support EFI?

[poettering]

Sorry, currentlysystemctl kexec requires the system is running with UEFI. We use systemd's boot loader config and ESP settings to determine which kernel and initrd are used.

Well, do note that if users load a kexec kernel with the kexec-tools before issuing "systemctl kexec", then our bootloader spec logic will not take effect.

What changed here really is that previously, if you didn't load a kexec kernel before issuing "systemctl kexec" we'd always just fallback to "systemctl reboot", while now we try to load a kernel, but that only works if the bootloader spec is used, and if we can't then we'll fail.

[yuwata]

Ah, thank you for the explanation.

[nos1609]

By the way, It works for me like mentioned above but have to type commands manually to take effect on a Legacy, not UEFI system.

sudo kexec -l /boot/vmlinuz-linux-ck-nehalem --initrd=/boot/initramfs-linux-ck-nehalem.img --reuse-cmdline
sudo systemctl kexec

After such operations, it successfully loads another kernel while initially being booted with GRUB.

Didn't really get why for such limitations have been introduced...

[sigaev]

I use the pair

sudo kexec <...>
sudo systemctl kexec

on a pre-EFI machine. In systemd 235, this works.

After 4bb2e9d that is included in systemd 236, while executing the above commands, systemd insists on locating an ESP partition and reading certain systemd boot loader configuration files. If that fails for any reason (non-EFI system, not using the systemd boot loader), it appears that kexec cannot be used.

I'm curious why this regression was deemed OK? @keszybz ?

[poettering]

Hmm, I must say I wonder too, whether old behaviour wasn't better: "systemctl kexec" meant that we'd use kexec if it was setup, and did a plain reboot otherwise. That way "systemctl kexec" was an optimization primarily, and the outcome of having a rebooted kernel would always hold, though not necessarily through kexec.

@keszybz I think we should revisit this change of behaviour!

[keszybz]

Behaviour was changed on purpose — with previous logic it was too easy to not load the kernel by mistake and try to execute kexec and get a normal reboot instead. I think if want to call kexec, you know (or at least expect) the kernel to be loaded. Why would you call kexec if you haven't loaded the kernel before? You'd have to rely on the off-chance that something loaded the right kernel. I think any kind of scripted or manual use of systemctl kexec will include loading of the kernel, at least in the case where it can't be loaded automatically, so then whoever is calling kexec either knows that the kernel should be there, or knows not to call kexec.

All in all, kexec is a low-level tool, you need to know what you are doing, and it's not equivalent to a normal reboot. In particular, a normal reboot goes through the boot-loader and uses the options there, and kexec allows for a different kernel / different options. So I think it's good to keep those two things separate.

[keszybz]

Hmm, not that I look at this again, something is fishy. Doing kexec -l ... && systemctl kexec should work, because before load_kexec_kernel first checks if the kexec kernel is loaded and exits immediately if yes. So if that fails that'd mean that either the kernel wasn't loaded properly or the check is broken somehow. @sigaev can you attach the logs for such a failed reboot?

So... after playing with this a bit, I still think that if systemctl kexec is called, and the kernel cannot be loaded, the command should fail. But when we are actually doing the shutdown, and for whatever reason kexec fails, we should proceed with a normal reboot. Network is down in this case already when we attempt to do the kexec, so rebooting normally is a better solution. I'll send a PR to proceed with a normal reboot if --force is given (systemd-kexec.service calls systemctl kexec --force).

[BtbN]

I used to be using a kexec-load.service like described in the arch wiki: https://wiki.archlinux.org/index.php/kexec#Separate_.2Fboot_partition

Updated to systemd with the mentioned commit in it today, and now I don't seem to be able to systemctl kexec anymore, because it can't find the nonexistent ESP.
Formally, the service would take care of loading the kernel just fine, and it seems like this whole setup is entirely impossible now unless I manually load kernel and initrd every time.

[ghost]

it seems like this whole setup is entirely impossible now unless I manually load kernel and initrd every time

A workaround: kexec-load before reboot and make systemd-reboot.service do kexec. My Ubuntu 18.04 setup:

# cat /etc/systemd/system/systemd-reboot.service.d/override.conf 
[Service]
Type=oneshot
ExecStart=
ExecStart=/bin/systemctl --force kexec

# cat /etc/systemd/system/kexec-load.service 
[Unit]
Description=load kernel into the current kernel
Documentation=man:kexec(8)
DefaultDependencies=no
Before=shutdown.target umount.target final.target reboot.target systemctl-reboot.service

[Service]
Type=oneshot
ExecStart=/sbin/kexec -d -l /vmlinuz --initrd=/initrd.img --reuse-cmdline

[Install]
WantedBy=reboot.target

(some of the Before= services might be unnecessary, I was adding them until it stopped unmounting NFS root filesystem before kernel and initrd are loaded from it.)

[cbz]

The last just results in a system hang on reboot for me.

[bmwiedemann]

In openSUSE we also loaded the kexec kernel during the shutdown initiated by systemctl kexec
https://bugzilla.opensuse.org/show_bug.cgi?id=1153790#c3 has a hack to make it work again in newer systemd

But it would be even nicer, if we could have the kexec-load.service file run between systemctl kexec and the check for a loaded kexec kernel.

[paulmenzel]

I just hit the same issue. The error/warning is just confusing. I suggest to extend it.

No kexec kernel configured. Automatic selection currently only works for systems following the boot loader specification.