Can't boot after upgrading

[krysclarke]

systemd version the issue has been seen with

255 rc2-1

Used distribution

Debian Sid

Linux kernel version used

unsure

CPU architectures issue was seen on

x86_64

Component

other

Expected behaviour you didn't see

System boots up fine

Unexpected behaviour you saw

VERY early in the boot process, my desktop computer stops booting.
Thankfully, I have both a desktop and a laptop computer, so the following is manually "copy & pasted" - Ctrl-C, Ctrl-V isn't available to copy the text off my desktop computer and on to my laptop computer, so apologies for any spelling/capitalisation errors.

[ 1.410816] systemd[1]: Inserted module 'autofs4'
[ 1.418158] systemd[1]: systemd 255~rc2-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 1.419059] systemd[1]: Detected architecture x86-64.
[ 1.419365] systemd[1]: Running in initrd.

Welcome to Debian GNU/Linux trixie/sid dracut-059-4 (Initramfs)!

[ 1.420802] systemd[1]: No hostname configured, using default hostname.
[ 1.421161] systemd[1]: Hostname set to .
[ 1.421507] systemd[1]: Initializing machine ID from random generator.
[ 1.459899] systemd[1]: Failed to open executor binary '/usr/lib/systemd/systemd-executor': No such file or directory
[ 1.460870] systemd[1]: Failed to allocate manager object: No such file or directory
[!!!!!!] Failed to allocated manager object.
[ 1.462055] systemd[1]: Freezing execution

Hitting the 'reset' button on my computer produces the same messages, with very slightly different timestamps, and an extra 3 lines:

[ 1.608864] tsc: Refined TSC clocksource calibration: 3792.873 MHz
[ 1.609464] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x6d5818a734c, max_idle_ns: 881590694765 ns
[ 1.610090] clocksource: Switched to clocksource tsc

Steps to reproduce the problem

1. apt-get update
2. apt-get upgrade
3. Reboot

NOTE: There was several hours between the apt-get upgrade and the reboot (which, technically, was me shutting down the computer just before I went to bed and turning it back on soon after breakfast this morning), I can't remember which packages were upgraded (though I do remember I had a handful of grub packages that had been held back for several days until yesterday) and I do remember that there were no packages that were being held back from being upgraded.

Additional program output to the terminal or log subsystem illustrating the issue

No response

[vcaputo]

This seems likely to be a Debian packaging / update process bug. systemd-executor is a new addition via commit bb5232b only present in v255.

Since your error is so early, still in the initramfs, I suspect Debian failed to pull in the added systemd-executor executable when rebuilding your initrd.img post-upgrade.

Probably best to report it to Debian upstream instead.

Edit: BTW, in the interests of helping you get your machine booted, you might be able to get back in by simply selecting an alternative kernel+initrd combo from your boot menu. Debian usually leaves around the old stuff in my experience. But I'm not sure off the top of my head how happy systemd will be with an older version incoming from initramfs vs. what's on the fully booted environment's filesystem.

[YHNdnzj]

As stated in NEWS, we now rely on a new systemd-executor binary for process spawning. Therefore, it needs to be included in the initrd. For dracut, you need dracutdevs/dracut#2535 which hasn't made into a stable release yet.

[krysclarke]

Edit: BTW, in the interests of helping you get your machine booted, you might be able to get back in by simply selecting an alternative kernel+initrd combo from your boot menu. Debian usually leaves around the old stuff in my experience. But I'm not sure off the top of my head how happy systemd will be with an older version incoming from initramfs vs. what's on the fully booted environment's filesystem.

You are correct, Debian does leave the previous kernel version to be installed lying around, so that is an option for me.
The big problem for me here (and not an issue for this bit of GitHub), is that I've set GRUB to 0sec timeout and to not show the menu, and I've no idea of the correct key(s) to use to get it to show to me.

[vcaputo]

Edit: BTW, in the interests of helping you get your machine booted, you might be able to get back in by simply selecting an alternative kernel+initrd combo from your boot menu. Debian usually leaves around the old stuff in my experience. But I'm not sure off the top of my head how happy systemd will be with an older version incoming from initramfs vs. what's on the fully booted environment's filesystem.

You are correct, Debian does leave the previous kernel version to be installed lying around, so that is an option for me. The big problem for me here (and not an issue for this bit of GitHub), is that I've set GRUB to 0sec timeout and to not show the menu, and I've no idea of the correct key(s) to use to get it to show to me.

I've always used holding down Shift for this, but haven't really used grub since the uefi transition. According to https://askubuntu.com/questions/16042/how-to-get-to-the-grub-menu-at-boot-time you use Esc on uefi to get the prompt.

Easy to try...

[krysclarke]

Someone else beat me to reporting it to Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1056059

As this is confirmed to not be a systemd bug, I'm closing this.

[TriMoon]

@krysclarke

Used distribution

Debian Sid

Maybe sid isn't a fit for you 😉
(Try a more stable version of debian, because sid will break many times due to the nature of that version...)

[xiacunshun]

Same problem on OpencloudOS.