Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

firmware: Enable firmware compression to conserve some disk space #3418

Merged

Conversation

foersleo
Copy link
Contributor

@foersleo foersleo commented Sep 6, 2023

Issue number:

Closes #3359

Description of changes:

When we introduced the linux-firmware package in #3296, we did not enable
compression as we had not have the necessary support in our kernels enabled. This PR fixes this by enabling the necessary
config options. Before linux kernel series 5.19 the firmware compression code only supported XZ compression, hence we do
not need (and cant) select a specific compression algorithm. With 6.1 we need to additionally select XZ compression.

This change saves about 4mb in the package size, and even 36 mb in unpacked form:

$ ls -ahl before after
after:
total 25M
drwxr-xr-x. 3 fedora fedora 4.0K Sep  6 12:08 .
drwxr-xr-x. 4 fedora fedora 4.0K Sep  6 11:38 ..
-rw-r--r--. 1 fedora fedora  25M Sep  6 11:43 bottlerocket-linux-firmware-20230625-1.x86_64.rpm
drwxr-xr-x. 3 fedora fedora 4.0K Sep  6 12:08 x86_64-bottlerocket-linux-gnu

before:
total 29M
drwxr-xr-x. 3 fedora fedora 4.0K Sep  6 12:08 .
drwxr-xr-x. 4 fedora fedora 4.0K Sep  6 11:38 ..
-rw-r--r--. 1 fedora fedora  29M Sep  6 11:40 bottlerocket-linux-firmware-20230625-1.x86_64.rpm
drwxr-xr-x. 3 fedora fedora 4.0K Sep  6 12:08 x86_64-bottlerocket-linux-gnu
$ du -sh before/x86_64-bottlerocket-linux-gnu/
61M	before/x86_64-bottlerocket-linux-gnu/
$ du -sh after/x86_64-bottlerocket-linux-gnu/
25M	after/x86_64-bottlerocket-linux-gnu/

Testing done:

Proper testing for this is tricky, as we do not have access to any of the hardware this package includes.

Only the expected config changes are recorded for this patch:

$ cat configs_firmware_compression/diff-report 
==> configs_firmware_compression/config-aarch64-metal-dev-diff <==
 FW_LOADER_COMPRESS n -> y
+FW_LOADER_COMPRESS_XZ y
+FW_LOADER_COMPRESS_ZSTD n

==> configs_firmware_compression/config-x86_64-metal-dev-diff <==
 FW_LOADER_COMPRESS n -> y
+FW_LOADER_COMPRESS_XZ y
+FW_LOADER_COMPRESS_ZSTD n

==> configs_firmware_compression/config-x86_64-metal-k8s-1.23-diff <==
 FW_LOADER_COMPRESS n -> y

==> configs_firmware_compression/config-x86_64-metal-k8s-1.27-diff <==
 FW_LOADER_COMPRESS n -> y

Apart from that we can not really
Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

Copy link
Contributor

@arnaldo2792 arnaldo2792 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there specific testing to be done for ARM? I remember that “self-decompressing” kernel images were only supported for x86_64, and for ARM images the kernel folks relied on the bootloader to decompress the image. I assume that decompressing the firmware is handled by the kernel and thus nothing special is needed. But it wouldn't hurt to confirm 😅

packages/linux-firmware/linux-firmware.spec Outdated Show resolved Hide resolved
Firmware binary blobs can add up to quite a lot of disk space usage.
With Bottlerocket we do aim to keep our images small, so enable the
kernel's firmware loading infrastructure to handle compressed firmware
files.

Signed-off-by: Leonard Foerster <foersleo@amazon.com>
Firmware binary blobs can add up to quite a lot of disk space usage.
With Bottlerocket we do aim to keep our images small, so enable the
kernel's firmware loading infrastructure to handle compressed firmware
files.

Signed-off-by: Leonard Foerster <foersleo@amazon.com>
Firmware binary blobs can add up to quite a lot of disk space usage.
With Bottlerocket we do aim to keep our images small, so enable the
kernel's firmware loading infrastructure to handle compressed firmware
files.

Previous to linux kernel series 5.19 option `CONFIG_FW_LOADER_COMPRESS`
was only supporting the XZ compression algorithm. Since then also ZSTD
compression is available which makes additional options necessary to
select the right algorithm. For now select the default XZ compression
to align all our supported kernels on the same. Once we are only
supporting kernels >=6.1 we may chose to switch compression algorithm.

Signed-off-by: Leonard Foerster <foersleo@amazon.com>
Recoup some image size by enabling firmware compression, now that our
kernels support it.

None of the compressed firmware files has the execute bit set, so we can
skip removing it when assembling the package.

Signed-off-by: Leonard Foerster <foersleo@amazon.com>
@foersleo
Copy link
Contributor Author

foersleo commented Sep 7, 2023

Is there specific testing to be done for ARM? I remember that “self-decompressing” kernel images were only supported for x86_64, and for ARM images the kernel folks relied on the bootloader to decompress the image. I assume that decompressing the firmware is handled by the kernel and thus nothing special is needed. But it wouldn't hurt to confirm 😅

Great pointer @arnaldo2792! I have double checked that none of the config options comes with any dependency on architecture and the code under drivers/base/firmware_loader in the linux kernel do not have any code restricted on Architecture. So we are fine in that regard.

@foersleo
Copy link
Contributor Author

foersleo commented Sep 7, 2023

⬆️ force-push rebases on latest develop to accommodate conflicting changes introduced through #3419 (just contextual changes in the config files) and addressed Ben's comment on necessity of removing the execute bits.

Copy link
Member

@markusboehme markusboehme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice space savings!

@foersleo foersleo merged commit db97cd3 into bottlerocket-os:develop Sep 7, 2023
48 checks passed
@foersleo foersleo deleted the feature/firmware_compression branch September 7, 2023 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

kernel/firmware: Add support for firmware compression
5 participants