Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
Calculate sha256 on the fly on dockerTools.buildLayeredImage #89075
Motivation for this change
Previously, we were calculating layer checksums at a separate pass after
This commit calculates the sha256 in parallel when creating the tarball
Relevant NixOS tests (
Previously, we were calculating layer checksums at a separate pass after the creation of the tarball. This is not ideal when packaging containers with large layers, since it requires reading the whole tarball again from disk in order to create the checksum. This commit calculates the sha256 in parallel when creating the tarball and saves it to a file for later use by buildLayeredImage function.
Please hold off on merging this since there seems to be a scary non-deterministic error somewhere. On every 2-3rd try, the tests fail with an error like:
I couldn't figure out the reason yet, so I'd appreciate another set of eyes.
Thank you @nlewo.
I think so. The error is not deterministic, so I can't be 100% sure, but I ran things without this patch multiple times and couldn't reproduce the error.
No, I just got the same error for layer 6.
I built and loaded it 6 times and I can't reproduce locally:
(I ran the gc before each attempt)
Would you be able to keep a copy of each produced image in order to compare them? You could first try to run sha256 on each produced image to see if they are equal. If not, you could then use diffoscope to identity the differences (it know how to unpack Docker images).