Newer bunzip2 won't decompress Container Linux release artifacts

[bgilbert]

Issue Report

Bug

Container Linux Version

Any

Environment

Any

Expected Behavior

bunzip2 correctly decompresses Container Linux release artifacts.

Actual Behavior

bunzip2: Data integrity error when decompressing.

Reproduction Steps

1. Launch Container Linux 2163.0.0
2. wget https://stable.release.core-os.net/amd64-usr/2079.5.1/coreos_production_image.bin.bz2
3. bunzip2 coreos_production_image.bin.bz2

Other Information

lbzip2 has long created archives with more than 18002 selectors, which is incorrect. This was fixed in 2017 but Container Linux has still been using an unfixed version to compress its release artifacts. bunzip2 is recently maintained again, and has merged a patch refusing to read affected archives. That patch has been in Gentoo since April, is not yet in Fedora, and will likely propagate to other distros over time.

Many Container Linux release artifacts are affected, over the entire history of Container Linux. Alpha 2163.0.0 includes updated versions of both lbzip2 and bunzip2, so that 2163.0.0 release artifacts are properly formatted but a 2163.0.0 system can't read older artifacts.

Plan

1. We've stopped the rollout of 2163.0.0 and are releasing 2163.1.0 with the bunzip2 change reverted, allowing alpha to read older artifacts again.
2. We've backported the lbzip2 fix to the beta 2135 branch. We'll do a release on that branch next week, and it will promote to stable the following week. Those releases will have correctly compressed release artifacts.
3. We will eventually stop reverting the bunzip2 change, causing newer Container Linux releases to be unable to read older artifacts. Timing has not yet been decided; it'll probably happen in either the stable release after 2135, or the one after that.

Workaround

If it's necessary to read older release artifacts on newer systems, decompress them with lbunzip2.

[bgilbert]

Beta 2135.3.1 and stable 2079.6.0 were just released. Both have a fixed lbzip2 and should have correctly compressed artifacts.