Add package linux-firmware and use it on metal-* variants

[foersleo]

Issue number:

Closes #3246

Description of changes:

With introduction of Metal variants we opened up Bottlerocket to a wider variety of Hardware configurations. In the past we have added driver support for some server grade hardware, mostly in the 10G+ NIC realm. While adding the drivers, we overlooked that some of these devices may need firmware binaries to be available in order to update firmware and function with the driver version shipped with the kernel.

Add firmware from linux-firmware to our metal variants for drivers we support in Bottlerocket. I have made an effort to cut down the list of binaries to only the ones we currently supply drivers for, as the linux-firmware package is quite large in full. I was able to cut the size of the installed size on disk down from 426 MB for the full package to 29 MB for the stripped down version carrying only the needed firmware.

We could strip the size on disk down even further when using compressed firmware images. Currently ZSTD and XZ compression are supported. While ZSTD does not change the size significantly XZ compression can further reduce the size from 29 MB (uncompressed) to 25 MB (xz compressed). However, doing that requires us to enable additional kernel config options CONFIG_FW_LOADER_COMPRESS and CONFIG_FW_LOADER_COMPRESS_XZ. That optimization can probably be part of a separate commit.

The resulting firmware package carries firmware for the following list of drivers and their corresponding config options we have configured for our metal kernels:

cxgb4 - CONFIG_CHELSIO_T4
tg3 - CONFIG_TIGON3
bnx2x - CONFIG_BNX2X
netxen_nic - CONFIG_NETXEN_NIC
qed - CONFIG_QED
myri10ge - CONFIG_MYRI10GE
i915 - CONFIG_DRM_I915
ice - CONFIG_ICE

Testing done:

I have done some build testing. Unfortunately assessing if all the firmware is in the correct place and can be reached by the drivers is beyond what I can do as I do not have access to these devices.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[foersleo]

I have not put the source tar in the lookaside cache yet. Hence, the failing test runs. I want to check the licensing on these firmware blobs we ship first. The linux-firmware package is an interesting place with regards to licenses.

Edit: So all the binaries that will currently go into this package are marked as Redistributable with no modification permitted.

[foersleo]

⬆️ Force push was just rebasing on top of develop before tending comments.

[foersleo]

⬆️ Force push tended most comments by Ben. The only one I have not fixed yet is the one about the SPDX license specs because of the above mentioned reasons.

I went also ahead and removed unneeded license files in the patches in commit 2 of the series. That helps uncluttering the license directory in the installation and only ships the license files for firmware we actually ship.

[zmrow]

From the variant perspective this looks good - I don't have any additional comments about the package or patches!

[yeazelm]

I'm happy with the outcome of trimming this size down. I am curious if we know how difficult it will be to keep up with these updates as things are added to the linux-firmware bundle upstream. Is there anything special we need to track to ensure updates don't accidentally start bloating with more things or will we just need to be diligent to check each update doesn't include things we don't need at the time of reviewing the update?

[foersleo]

I'm happy with the outcome of trimming this size down. I am curious if we know how difficult it will be to keep up with these updates as things are added to the linux-firmware bundle upstream. Is there anything special we need to track to ensure updates don't accidentally start bloating with more things or will we just need to be diligent to check each update doesn't include things we don't need at the time of reviewing the update?

That is a great question and I am a bit concerned about that myself. In all honesty I do not know yet. It looks like there is a release for linux-firmware about every 4-6 weeks (roughly once a month). I will have a stab at what it takes to update once the next release is out.

In the extreme case where these updates are too burdensome to do every month, we could just go with a linux-firmware update when we introduce a new major kernel. As discussed in a comment to Ben's question about dependency to specific kernel versions, the firmware images would be fine for the kernels we currently support. I do not expect major driver updates that would rely on newer firmware files (The file names are usually hard-coded in the drivers) within a stable series.

In an ideal world we would spend some time to build tooling to automatically strip down the files according to some kernel config file. Given the structure of the firmware repo I'd assume this to be quite challenging with lots of corner cases. But it is something we might want to try our hands on when we find some time

[foersleo]

Firmware was confirmed to be at the right spot and the driver finding it properly by Max Böhm #3246 (comment)

Thanks a lot Max for helping out here!

[markusboehme]

I wanted to review those patches as applied to linux-firmware and there's a bit of an interesting problem there: They do apply fine for me via patch and git apply, but then I lose their history and commentary. When trying to apply them via git am, at least 0003-linux-firmware-bt-wifi-Remove-firmware-for-Bluetooth.patch fails. The first failure is for the first line of the deleted LICENSE.nxp which does seem to use CRLF line endings. The patch seems to have converted those to just LF, and hence doesn't match.

I know Git can perform conversion of line endings, but it seems unexpected here. Can you check whether the original patches as generated by git format-patch were correct and this only happened when committing the patch files to the Bottlerocket repo? If so, I wonder whether disabling EOL conversion for *.patch files via .gitattributes (by treating those as binary files) would be desirable.

For the review, I just pass --ignore-whitespace to git am.

[markusboehme]

Thanks for the diligent work combing through all this! I hope it all plays well with updates and won't need to be redone. 🥲

I'm happy to approve once we reach a conclusion about the SPDX identifiers and the line endings (not caring at this point could be one).

[foersleo]

I wanted to review those patches as applied to linux-firmware and there's a bit of an interesting problem there: They do apply fine for me via patch and git apply, but then I lose their history and commentary. When trying to apply them via git am, at least 0003-linux-firmware-bt-wifi-Remove-firmware-for-Bluetooth.patch fails. The first failure is for the first line of the deleted LICENSE.nxp which does seem to use CRLF line endings. The patch seems to have converted those to just LF, and hence doesn't match.

I know Git can perform conversion of line endings, but it seems unexpected here. Can you check whether the original patches as generated by git format-patch were correct and this only happened when committing the patch files to the Bottlerocket repo? If so, I wonder whether disabling EOL conversion for *.patch files via .gitattributes (by treating those as binary files) would be desirable.

For the review, I just pass --ignore-whitespace to git am.

The line endings seems to be an interesting issue with git format-patch and git am. This is not an artifact of adding patches to a separate repository. The original patches do not apply to a clean checkout/unpack of linux firmware when trying to apply via git am without extra flags. We can work around this by passing flags --ignore-whitespace (as pointed out by you), or --keep-cr which will apply the patches, but create a warning about quoted CRLF:

$ for p in ../../linux-firmware-20230625/*.patch; do git am --keep-cr $p; done
Applying: linux-firmware: snd: remove firmware for snd/audio devices
Applying: linux-firmware: video: Remove firmware for video/broadcast devices
warning: quoted CRLF detected
Applying: linux-firmware: bt/wifi: Remove firmware for Bluetooth/WiFi devices
Applying: linux-firmware: scsi: Remove firmware for SCSI devices
Applying: linux-firmware: usb: remove firmware for USB/Serial/PCMCIA devices
Applying: linux-firmware: ethernet: Remove firmware for ethernet/IB devices
Applying: linux-firmware: Remove firmware for Accelarator devices
Applying: linux-firmware: gpu: Remove firmware for GPU devices
Applying: linux-firmware: various: Remove firmware for various devices
Applying: linux-firmware: amd-ucode: Remove amd microcode

Stackoverflow has some more detail about what is happening and what git tries to do to work around this. The quoted commit message in one of the answers is particularly enlightening about the odds and ins of CRLF. The gist of which is: git-am by default does not trust line endings coming in through mail because SMTP transport is CRLF-unsafe.

Unfortunately the proposed --transfer-encoding option seems to be not available with git format-patch so it seems there is not really anything we can do about this on the patch creation side.

[foersleo]

⬆️ Force push simplified %prep as suggested by @markusboehme and dropped %changelog as suggested by @yeazelm

[markusboehme]

@foersleo Thanks for looking closer into it! TIL about git am --keep-cr which seems to be the better alternative.

[foersleo]

Converting this one to draft for now to reduce risk of accidental merge while I am sorting out the questions regarding the Licenses.

[foersleo]

⬆️ Force push included fixing up the list of SPDX identifiers as discussed with OSPO and added the force-upstream flag to the package as introduced in #3333 .
Unfortunately I did mix in some catch up with the upstream develop branch and so the comparison to the previous state is littered with a lot of other changes.

Will do an additional rebase shortly to base it on top of current develop branch.

[markusboehme]

Still looking good to me with the updated SPDX license specifiers.

[foersleo]

⬆️ tended the comments mentioned by Arnaldo.

In addition I removed the force-upstream falg from the linux-firmware package. Discussing it with Ben we should be fine hosting a copy in the look-aside cache and we want to do that for as many packages as possible.

[foersleo]

Uploaded the source artifacts to lookaside cache and checked the automated test builds succeed. Merging.