This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
As part of Reproducible Builds effort, make the images produced by lorax reproducible, given the same set of inputs (packages, configuration, SOURCE_DATE_EPOCH variable).
See individual commits for explanation of specific changes.
The last commit require matching anaconda change, as it change image layout. It should be possible to make anaconda support both old and new format. Is there any actual reason why ext4 image is packaged into squashfs instead of using squashfs directly? I imagine historically it could be lack of overlayfs in vanilla kernel and the need to use dm-snapshot. But it is no longer the case. Anything else?
One thing not solved here is efiboot.img, because I didn't managed to reproducibly build FAT filesystem. Even after eliminating obvious metadata differences (volume ID, file mtimes, files order), there are still some differences near the end of the image, which I didn't identified.
By default mkfs.mksdos choose volume id based on current time. If
SOURCE_DATE_EPOCH is set, use that instead.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Some files are created in non-reproducible way, including including
random data explicitly (/etc/machine-id), timestamps (fontconfig cache,
ldconfig aux-cache, certs cache), or entries in random order (groups,
systemd catalog, package list).
Fix this by either making the files reproducible, or removing them.
Overall this looks pretty simple. ISTR there was a good reason we didn't switch to plain squashfs for the install.img but cannot remember exactly why. @wgwoods may remember though.
Changing the on-disk layout needs to be coordinated with Fedora and especially Anaconda so you may want to join the fedora-devel or anaconda-devel mailing lists and introduce yourself there.
I'm not convinced that reproducible builds are worth the extra effort, in general, compared with good gpg signatures on everything, but these patches are pretty clean so I'm not opposed to merging them in the future if you can get the needed changes into Anaconda.
Apart from making the build reproducible, this also reduce size and complexity of the image. Not requiring dmsquash-live dracut module (which include dmsetup etc) make initramfs noticeably smaller. Normally it isn't a big deal, but if you need to put that + kernel into efiboot.img, it's important. Because boot image on ISO9660 is limited to 32MB.
I'll write on fedora-devel about this.
If you want, I can extract this one commit to a separate PR.
Is there any actual reason why ext4 image is packaged into squashfs instead of using squashfs directly? I imagine historically it could be lack of overlayfs in vanilla kernel and the need to use dm-snapshot. But it is no longer the case. Anything else?
Correct, the problem with bare squashfs images was that anaconda still needs its root filesystem to be writeable (for updates.img mainly), and squashfs doesn't support the write() system call at all, so there's no way to make a squashfs filesystem writeable using device-mapper overlays. And overlayfs wasn't in the mainstream kernel at the time (Fedora 15-16 / kernel 2.6-3.1) - that only became an option in Fedora 22 / kernel 4.0.
I've always thought the ext4-inside-squashfs image payload was needlessly complex and would be happy to see a simpler solution.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.
marmarek
force-pushed
the
reproducible-builds
branch
from
As part of Reproducible Builds effort, make the images produced by lorax reproducible, given the same set of inputs (packages, configuration, SOURCE_DATE_EPOCH variable).
See individual commits for explanation of specific changes.
The last commit require matching anaconda change, as it change image layout. It should be possible to make anaconda support both old and new format. Is there any actual reason why ext4 image is packaged into squashfs instead of using squashfs directly? I imagine historically it could be lack of overlayfs in vanilla kernel and the need to use dm-snapshot. But it is no longer the case. Anything else?
One thing not solved here is
efiboot.img, because I didn't managed to reproducibly build FAT filesystem. Even after eliminating obvious metadata differences (volume ID, file mtimes, files order), there are still some differences near the end of the image, which I didn't identified.