Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kernel-install doesn't work for kernel tree-produced rpms #29568

Closed
davide125 opened this issue Oct 13, 2023 · 5 comments
Closed

kernel-install doesn't work for kernel tree-produced rpms #29568

davide125 opened this issue Oct 13, 2023 · 5 comments
Labels
bug 🐛 Programming errors, that need preferential fixing kernel-install not-our-bug

Comments

@davide125
Copy link
Contributor

systemd version the issue has been seen with

254 (254.5-2.fc40)

Used distribution

Fedora Rawhide

Linux kernel version used

6.6.0-0.rc5.40.fc40.aarch64

CPU architectures issue was seen on

aarch64

Component

kernel-install

Expected behaviour you didn't see

/boot/vmlinuz-$kver exists after the kernel RPM is installed

Unexpected behaviour you saw

/boot/vmlinuz-$kver does not exist after the kernel RPM is installed

Steps to reproduce the problem

On a Fedora Rawhide (or Fedora ELN) system:

# build the kernel rpm
$ git clone linux.git
$ make defconfig
$ make rpm-pkg
# look at the scripts
$ rpm -q --scripts rpmbuild/RPMS/aarch64/kernel-6.6.0_rc5_00234_g8cb1f10d8c4b_dirty-2.aarch64.rpm
# install the kernel
$ sudo dnf install -y rpmbuild/RPMS/aarch64/kernel-6.6.0_rc5_00234_g8cb1f10d8c4b_dirty-2.aarch64.rpm
# notice how vmlinuz is missing in /boot
$ ls -la /boot

What's happening here is that the kernel RPM spec includes this logic: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3c9c7a14b627 which moves vmlinuz away before calling installkernel. This then leads to errors in the output (coming from /usr/lib/kernel/install.d/20-grub.install I believe), but these don't interrupt the install:

Successfully forked off '(direxec)' as PID 18420.
grub2-mkrelpath: error: failed to get canonical path of `/boot/vmlinuz-6.6.0_rc5_00234_g8cb1f10d8c4b_dirty'.
dirname: missing operand
Try 'dirname --help' for more information.
/usr/lib/kernel/install.d/20-grub.install succeeded.

and at the end of it, /boot/vmlinuz-6.6.0_rc5_00234_g8cb1f10d8c4b_dirty is missing. We originally discovered this on Fedora ELN, but I've verified it repros on Rawhide as well (and likely any other distro with 254). I suspect this is a side effect of the kernel-install rewrite in C.

Additional program output to the terminal or log subsystem illustrating the issue

No response
@davide125 davide125 added the bug 🐛 Programming errors, that need preferential fixing label Oct 13, 2023
@davide125
Copy link
Contributor Author

Also filed https://bugzilla.redhat.com/show_bug.cgi?id=2244131 for this to track it downstream

@Exotic0015 Exotic0015 mentioned this issue Oct 15, 2023
Closed
@davide125
Copy link
Contributor Author

I've submitted https://lore.kernel.org/linux-kbuild/20231103-rpmpost-v1-1-9c18afab47f4@meta.com/ to simplify the %post on the kernel end, which would avoid triggering this issue. Nonetheless, this should still be fixed on the systemd side as previously created RPMs will still be impacted.

intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Nov 3, 2023
@davide125 @intel-lab-lkp
rpm-pkg: simplify installkernel %post
30c5384 
The %post currently does a shuffling dance before calling installkernel.
This isn't actually necessary afaict, and the current implementation
ends up triggering downstream issues such as
systemd/systemd#29568

This commit simplifies the logic to remove the shuffling. For reference,
the original logic was added in commit 3c9c7a1("rpm-pkg: add %post
section to create initramfs and grub hooks").

Signed-off-by: Davide Cavalca <dcavalca@meta.com>
intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Dec 12, 2023
@jtornosm @intel-lab-lkp
rpm-pkg: simplify installkernel %post
806f203 
A new installkernel application is now included in systemd-udev package
and it has been improved to allow simplifications.

For the new installkernel application, as Davide says:
<<The %post currently does a shuffling dance before calling installkernel.
This isn't actually necessary afaict, and the current implementation
ends up triggering downstream issues such as
systemd/systemd#29568
This commit simplifies the logic to remove the shuffling. For reference,
the original logic was added in commit 3c9c7a1("rpm-pkg: add %post
section to create initramfs and grub hooks").>>

But we need to keep the old behavior as well, because the old installkernel
application from grubby package, does not allow this simplification and
we need to be backward compatible to avoid issues with the different
packages. So the easiest solution is to check the package that provides
the installkernel application, and simplify (and fix for this
application at the same time), only if the package is systemd-udev.

cc: stable@vger.kernel.org
Co-Developed-by: Davide Cavalca <dcavalca@meta.com>
Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Dec 19, 2023
@jtornosm @intel-lab-lkp
rpm-pkg: simplify installkernel %post
b08d8b1 
The new installkernel application that is now included in systemd-udev
package allows installation although destination files are already present
in the boot directory of the kernel package, but is failing with the
implemented workaround for the old installkernel application from grubby
package.

For the new installkernel application, as Davide says:
<<The %post currently does a shuffling dance before calling installkernel.
This isn't actually necessary afaict, and the current implementation
ends up triggering downstream issues such as
systemd/systemd#29568
This commit simplifies the logic to remove the shuffling. For reference,
the original logic was added in commit 3c9c7a1("rpm-pkg: add %post
section to create initramfs and grub hooks").>>

But we need to keep the old behavior as well, because the old installkernel
application from grubby package, does not allow this simplification and
we need to be backward compatible to avoid issues with the different
packages.

Mimic Fedora shipping process and store vmlinuz, config amd System.map
in the module directory instead of the boot directory. In this way, we will
avoid the commented problem for all the cases, because the new destination
files are not going to exist in the boot directory of the kernel package.

Replace installkernel tool with kernel-install tool, because the latter is
more complete. Suitable manual actions are added as a default if tool is not
present (unusual).

Special installation case for discontinued architecture ia64 has been
removed.

cc: stable@vger.kernel.org
Co-Developed-by: Davide Cavalca <dcavalca@meta.com>
Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Dec 19, 2023
@jtornosm @intel-lab-lkp
rpm-pkg: simplify installkernel %post
e1074da 
The new installkernel application that is now included in systemd-udev
package allows installation although destination files are already present
in the boot directory of the kernel package, but is failing with the
implemented workaround for the old installkernel application from grubby
package.

For the new installkernel application, as Davide says:
<<The %post currently does a shuffling dance before calling installkernel.
This isn't actually necessary afaict, and the current implementation
ends up triggering downstream issues such as
systemd/systemd#29568
This commit simplifies the logic to remove the shuffling. For reference,
the original logic was added in commit 3c9c7a1("rpm-pkg: add %post
section to create initramfs and grub hooks").>>

But we need to keep the old behavior as well, because the old installkernel
application from grubby package, does not allow this simplification and
we need to be backward compatible to avoid issues with the different
packages.

Mimic Fedora shipping process and store vmlinuz, config amd System.map
in the module directory instead of the boot directory. In this way, we will
avoid the commented problem for all the cases, because the new destination
files are not going to exist in the boot directory of the kernel package.

Replace installkernel tool with kernel-install tool, because the latter is
more complete. Suitable manual actions are added as a default if tool is not
present (unusual).

cc: stable@vger.kernel.org
Co-Developed-by: Davide Cavalca <dcavalca@meta.com>
Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
@keszybz
Copy link
Member

keszybz commented Jan 9, 2024

https://bugzilla.redhat.com/show_bug.cgi?id=2244131#c1 has a longer debugging session. I'll paste just the gist here:

$ /sbin/installkernel -v 6.7.0+ /boot/.vmlinuz-6.7.0+-rpm /boot/.System.map-6.7.0+-rpm
...
kernel version (6.7.0+) set via command line.
kernel image file (/boot/.vmlinuz-6.7.0+-rpm) set via command line.
...
Plugin arguments: add 6.7.0+ /boot/fb878d0057c640258a53deec11a1c2f7/6.7.0+ /boot/.vmlinuz-6.7.0+-rpm
...
About to execute /usr/lib/kernel/install.d/10-devicetree.install add 6.7.0+ /boot/fb878d0057c640258a53deec11a1c2f7/6.7.0+ /boot/.vmlinuz-6.7.0+-rpm
...
About to execute /usr/lib/kernel/install.d/20-grub.install add 6.7.0+ /boot/fb878d0057c640258a53deec11a1c2f7/6.7.0+ /boot/.vmlinuz-6.7.0+-rpm
...
grub2-mkrelpath: error: failed to get canonical path of `/boot/vmlinuz-6.7.0+'.
dirname: missing operand
Try 'dirname --help' for more information.
/usr/lib/kernel/install.d/20-grub.install succeeded.
About to execute /usr/lib/kernel/install.d/50-depmod.install add 6.7.0+ /boot/fb878d0057c640258a53deec11a1c2f7/6.7.0+ /boot/.vmlinuz-6.7.0+-rpm
...

There is no /boot/vmlinuz-6.7.0+, there's only /boot/.vmlinuz-6.7.0+-rpm.
It seems that the scriptlet is called correctly. 20-grub.install tries to guess
the name and gets it wrong.

@keszybz keszybz closed this as not planned Won't fix, can't repro, duplicate, stale Jan 9, 2024
@DaanDeMeyer
Copy link
Contributor

@keszybz We've already figured out this regression was caused by your change in grubby to drop the /sbin/installkernel script: https://src.fedoraproject.org/rpms/grubby/c/aa9b1b454fd0c792226cefb65d744fadfaa8acda?branch=rawhide. kernel-install isn't a drop-in replacement for the grubby script since the grubby script also copied the kernel to /boot which kernel-install doesn't do unconditionally. Regardless, this is not a bug in kernel-install though.

@keszybz
Copy link
Member

keszybz commented Jan 9, 2024

kernel-install isn't a drop-in replacement for the grubby script since the grubby script also copied the kernel to /boot which kernel-install doesn't do unconditionally.

kernel-install calls 20-grub.install or 90-loaderentry.install that are supposed to install the kernel in whatever place they like. 20-grub.install is an unholy mess, but it has a line like cp --remove-destination --preserve=timestamps -T "${KERNEL_IMAGE}" "/boot/vmlinuz-${KERNEL_VERSION}", so I think it's supposed to do that.

Regardless, this is not a bug in kernel-install though.

Yes.

intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Jan 14, 2024
@jtornosm @intel-lab-lkp
rpm-pkg: simplify installkernel %post
b9b1bd6 
The new installkernel application that is now included in systemd-udev
package allows installation although destination files are already present
in the boot directory of the kernel package, but is failing with the
implemented workaround for the old installkernel application from grubby
package.

For the new installkernel application, as Davide says:
<<The %post currently does a shuffling dance before calling installkernel.
This isn't actually necessary afaict, and the current implementation
ends up triggering downstream issues such as
systemd/systemd#29568
This commit simplifies the logic to remove the shuffling. For reference,
the original logic was added in commit 3c9c7a1("rpm-pkg: add %post
section to create initramfs and grub hooks").>>

But we need to keep the old behavior as well, because the old installkernel
application from grubby package, does not allow this simplification and
we need to be backward compatible to avoid issues with the different
packages.

Mimic Fedora shipping process and store vmlinuz, config amd System.map
in the module directory instead of the boot directory. In this way, we will
avoid the commented problem for all the cases, because the new destination
files are not going to exist in the boot directory of the kernel package.

Replace installkernel tool with kernel-install tool, because the latter is
more complete.

Besides, after installkernel tool execution, check to complete if suitable
(same release and build) vmlinuz, System.map and config files are present
in /boot directory, and if necessary, copy manually for install operation
or remmove manually for remove operation.

Tested with Fedora 38, Fedora 39, RHEL 9, Oracle Linux 9.3,
openSUSE Tumbleweed and openMandrive ROME, using dnf/zypper and rpm tools.

cc: stable@vger.kernel.org
Co-Developed-by: Davide Cavalca <dcavalca@meta.com>
Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
teknoraver pushed a commit to teknoraver/linux that referenced this issue Jan 20, 2024
@jtornosm @teknoraver
rpm-pkg: simplify installkernel %post
27840ed 
A new installkernel application is now included in systemd-udev package
and it has been improved to allow simplifications.

For the new installkernel application, as Davide says:
<<The %post currently does a shuffling dance before calling installkernel.
This isn't actually necessary afaict, and the current implementation
ends up triggering downstream issues such as
systemd/systemd#29568
This commit simplifies the logic to remove the shuffling. For reference,
the original logic was added in commit 3c9c7a1("rpm-pkg: add %post
section to create initramfs and grub hooks").>>

But we need to keep the old behavior as well, because the old installkernel
application from grubby package, does not allow this simplification and
we need to be backward compatible to avoid issues with the different
packages. So the easiest solution is to check the package that provides
the installkernel application, and simplify (and fix for this
application at the same time), only if the package is systemd-udev.

cc: stable@vger.kernel.org
Co-Developed-by: Davide Cavalca <dcavalca@meta.com>
Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
ColinIanKing pushed a commit to ColinIanKing/linux-next that referenced this issue Feb 1, 2024
@jtornosm @masahir0y
kbuild: rpm-pkg: simplify installkernel %post
358de8b 
The new installkernel application that is now included in systemd-udev
package allows installation although destination files are already present
in the boot directory of the kernel package, but is failing with the
implemented workaround for the old installkernel application from grubby
package.

For the new installkernel application, as Davide says:
<<The %post currently does a shuffling dance before calling installkernel.
This isn't actually necessary afaict, and the current implementation
ends up triggering downstream issues such as
systemd/systemd#29568
This commit simplifies the logic to remove the shuffling. For reference,
the original logic was added in commit 3c9c7a1("rpm-pkg: add %post
section to create initramfs and grub hooks").>>

But we need to keep the old behavior as well, because the old installkernel
application from grubby package, does not allow this simplification and
we need to be backward compatible to avoid issues with the different
packages.

Mimic Fedora shipping process and store vmlinuz, config amd System.map
in the module directory instead of the boot directory. In this way, we will
avoid the commented problem for all the cases, because the new destination
files are not going to exist in the boot directory of the kernel package.

Replace installkernel tool with kernel-install tool, because the latter is
more complete.

Besides, after installkernel tool execution, check to complete if the
correct package files vmlinuz, System.map and config files are present
in /boot directory, and if necessary, copy manually for install operation.
In this way, take into account if  files were not previously copied from
/usr/lib/kernel/install.d/* scripts and if the suitable files for the
requested package are present (it could be others if the rpm files were
replace with a new pacakge with the same release and a different build).

Tested with Fedora 38, Fedora 39, RHEL 9, Oracle Linux 9.3,
openSUSE Tumbleweed and openMandrive ROME, using dnf/zypper and rpm tools.

cc: stable@vger.kernel.org
Co-Developed-by: Davide Cavalca <dcavalca@meta.com>
Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
LorenzoBianconi pushed a commit to LorenzoBianconi/bpf-next that referenced this issue Feb 22, 2024
@jtornosm @LorenzoBianconi
kbuild: rpm-pkg: simplify installkernel %post
71c26c9 
The new installkernel application that is now included in systemd-udev
package allows installation although destination files are already present
in the boot directory of the kernel package, but is failing with the
implemented workaround for the old installkernel application from grubby
package.

For the new installkernel application, as Davide says:
<<The %post currently does a shuffling dance before calling installkernel.
This isn't actually necessary afaict, and the current implementation
ends up triggering downstream issues such as
systemd/systemd#29568
This commit simplifies the logic to remove the shuffling. For reference,
the original logic was added in commit 3c9c7a1("rpm-pkg: add %post
section to create initramfs and grub hooks").>>

But we need to keep the old behavior as well, because the old installkernel
application from grubby package, does not allow this simplification and
we need to be backward compatible to avoid issues with the different
packages.

Mimic Fedora shipping process and store vmlinuz, config amd System.map
in the module directory instead of the boot directory. In this way, we will
avoid the commented problem for all the cases, because the new destination
files are not going to exist in the boot directory of the kernel package.

Replace installkernel tool with kernel-install tool, because the latter is
more complete.

Besides, after installkernel tool execution, check to complete if the
correct package files vmlinuz, System.map and config files are present
in /boot directory, and if necessary, copy manually for install operation.
In this way, take into account if  files were not previously copied from
/usr/lib/kernel/install.d/* scripts and if the suitable files for the
requested package are present (it could be others if the rpm files were
replace with a new pacakge with the same release and a different build).

Tested with Fedora 38, Fedora 39, RHEL 9, Oracle Linux 9.3,
openSUSE Tumbleweed and openMandrive ROME, using dnf/zypper and rpm tools.

cc: stable@vger.kernel.org
Co-Developed-by: Davide Cavalca <dcavalca@meta.com>
Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
teknoraver pushed a commit to teknoraver/linux that referenced this issue Feb 24, 2024
@jtornosm @teknoraver
rpm-pkg: simplify installkernel %post
ce51995 
A new installkernel application is now included in systemd-udev package
and it has been improved to allow simplifications.

For the new installkernel application, as Davide says:
<<The %post currently does a shuffling dance before calling installkernel.
This isn't actually necessary afaict, and the current implementation
ends up triggering downstream issues such as
systemd/systemd#29568
This commit simplifies the logic to remove the shuffling. For reference,
the original logic was added in commit 3c9c7a1("rpm-pkg: add %post
section to create initramfs and grub hooks").>>

But we need to keep the old behavior as well, because the old installkernel
application from grubby package, does not allow this simplification and
we need to be backward compatible to avoid issues with the different
packages. So the easiest solution is to check the package that provides
the installkernel application, and simplify (and fix for this
application at the same time), only if the package is systemd-udev.

cc: stable@vger.kernel.org
Co-Developed-by: Davide Cavalca <dcavalca@meta.com>
Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug 🐛 Programming errors, that need preferential fixing kernel-install not-our-bug
Milestone
No milestone
Development

No branches or pull requests
3 participants
@keszybz @davide125 @DaanDeMeyer