Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial support for reproducible builds #3255

Merged
merged 9 commits into from Dec 30, 2018
Merged

Initial support for reproducible builds #3255

merged 9 commits into from Dec 30, 2018

Conversation

rn
Copy link
Member

@rn rn commented Dec 29, 2018

This PR adds initial support for reproducible linuxkit builds (wip #2813).

  • Files created during the build have their timestamp fixed
  • Slices/Arrays are sorted before being marshalled to JSON

Currently, the tar, tar-kernel-initrd, docker, and kernel+initrd output formats support reproducible builds and tests for tar and kernel+initrd are added with this PR.

ladybugs

When creating files for the "intermediate" tar ball,
fix the ModTime. This reduces the difference between
LinuxKit images build from identical inputs.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
Currently 'docker export' is used to convert a linuxkit entry
in the YAML file to a tar file of the root filesystem. This
process creates a number of files and directories which have
the timestamp of when the 'docker export' is run. Fix 'em up.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
Sort list of capabilities that go into config.json.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
Sort the list of mount points by destination. This makes the list
deterministic for reproducible builds and also ensures that, e.g.,
the mount for /dev happens before the mount for /dev/pts.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
Test the 'tar' and 'kernel+initrd' formats for now.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
@deitch
Copy link
Collaborator

deitch commented Dec 29, 2018

This is rather nice!

Is there concern that the modtime now is just the timestamp that lkt open sourced? Might we want to have a cmdline option to set the date-time as an input, with a default for that time, and maybe an option that says, "now"?

E.g.

$ lkt build --modtime now ...
$ lkt build --modtime 20181229T175500Z ...

Or -timestamp or some other flag name, but that idea? And if not specified, just uses the date/time lkt was open-sourced?

@rn
Copy link
Member Author

rn commented Dec 29, 2018

Yeah, was considering making the timestamp configurable. But rather than via the commandline, we should probably support this proposal and make it configurable via the environment variable SOURCE_DATE_EPOCH. Just want to get the basics in place and then iterate over refinements...

@deitch
Copy link
Collaborator

deitch commented Dec 29, 2018

If that’s where it’s going as a standard, I guess. I like command line options as they are explicit. But not going to be a stickler about it. :-)

@justincormack
Copy link
Member

LGTM as a first pass. I think setting all times to the git commit time of the linuxkit build itself might also be a reasonable option in future, but a static time is fine for now.

@rn
Copy link
Member Author

rn commented Dec 30, 2018

I like the idea of using the git commit time as the default. That way it is different for different linuxkit versions. Might look at doing this as an improvement later along with support for SOURCE_DATE_EPOCH.

@rn rn merged commit 2b826be into linuxkit:master Dec 30, 2018
@rn rn deleted the repeat branch December 30, 2018 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants