New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reproducible builds proposal: make gem define SOURCE_DATE_EPOCH itself #2290

Open
anthraxx opened this Issue May 14, 2018 · 12 comments

Comments

Projects
None yet
6 participants
@anthraxx
Contributor

anthraxx commented May 14, 2018

I would like to suggest to make gem itself a potential SOURCE_DATE_EPOCH declarer instead of "only" making reproducible artifacts whenever the outside world defines the SOURCE_DATE_EPOCH environment variable.

While above works perfectly for all distros, as they and teir build tools and pipelines itself define SOURCE_DATE_EPOCH it would be awesome if the gem tool/script could define SOURCE_DATE_EPOCH itself.
This proposal would allow every gem aquired from the rubygems repository purely build with gem instead of any distro or other packaging related tool defining SOURCE_DATE_EPOCH to be independently reproduced.

This would need to check SOURCE_DATE_EPOCH in the gem command line tool, and if it is not yet define, it should define it to the current utc timestamp.

Example like it is done in Arch Linux's makepkg:
https://git.archlinux.org/pacman.git/tree/scripts/makepkg.sh.in#n93

This issue is related to:

  • Network problems
  • Installing a library
  • Publishing a library
  • The command line gem
  • Other

Related to pull requests:
#2289 #2278

Spec:
https://reproducible-builds.org/specs/source-date-epoch/

Buy-in:
https://reproducible-builds.org/docs/buy-in/

I will abide by the code of conduct.

@anthraxx

This comment has been minimized.

Contributor

anthraxx commented May 14, 2018

I just want to know if this is something that would be wanted/accepted by rubygems, then I can propose a pull request for this as well.

Awaiting feedback
CC @hsbt @segiddins @duckinator @lamby

@lamby

This comment has been minimized.

lamby commented May 14, 2018

Just to underline that it should only set SOURCE_DATE_EPOCH if it's not already set (as you can see here: https://git.archlinux.org/pacman.git/tree/scripts/makepkg.sh.in#n91)

@anthraxx

This comment has been minimized.

Contributor

anthraxx commented May 14, 2018

@lamby thanks to make sure this happens that way, but thats literally what i wrote in the 3th block 🐱

@lamby

This comment has been minimized.

lamby commented May 14, 2018

@anthraxx I saw exactly that, hence my "underline" :)

@bronzdoc

This comment has been minimized.

Member

bronzdoc commented Jul 18, 2018

This would need to check SOURCE_DATE_EPOCH in the gem command line tool, and if it is not yet define, it should define it to the current utc timestamp.

Rubygems already does that, see https://github.com/rubygems/rubygems/blob/master/lib/rubygems/package.rb#L163

@bronzdoc bronzdoc closed this Jul 18, 2018

@anthraxx

This comment has been minimized.

Contributor

anthraxx commented Jul 18, 2018

@anthraxx

This comment has been minimized.

Contributor

anthraxx commented Jul 18, 2018

@hsbt hsbt reopened this Aug 11, 2018

@duckinator

This comment has been minimized.

Member

duckinator commented Oct 24, 2018

Sorry for the late response, @anthraxx. I only just now saw this issue.

My understanding of the issue is that, every place RubyGems checks for SOURCE_DATE_EPOCH, it does something along the lines of:

time = ENV["SOURCE_DATE_EPOCH"] ? Time.at(ENV["SOURCE_DATE_EPOCH"].to_i) : Time.now

... which means if you don't explicitly define it, then the times won't be consistent throughout the entire process.

This means that you can only get reproducible builds via RubyGems by setting SOURCE_DATE_EPOCH outside of RubyGems.

Whereas if we check it in that way once, and set the environment variable ourselves if it is not already set, then we're defaulting to reproducible builds.

Is that correct, @anthraxx?

@anthraxx

This comment has been minimized.

Contributor

anthraxx commented Oct 24, 2018

Yes, the first set of commits i did just ensured everything works fine if we define SOURCE_DATE_EPOCH outside (which we do in our distro), which was the priority for me so i can have reproducible packages.

This is about making every gem package produced through gem and published to rubygems potentially reproducible.

To achieve this, as you summarized correctly, gem needs to define SOURCE_DATE_EPOCH env var once in an early stage (only if it is not yet defined from the outside) to the same uniform value for SOURCE_DATE_EPOCH and not Time.now. Doing this over an env var is important so any build process that invokes other libs or subprocesses that may respect SOURCE_DATE_EPOCH are also able to respect the same value.

@duckinator

This comment has been minimized.

Member

duckinator commented Oct 25, 2018

@anthraxx 👍 okay. I'm definitely in favor of adding that.

@simi

This comment has been minimized.

Contributor

simi commented Oct 25, 2018

@anthraxx I did some tests and it will be probably enough to pass build_time in Gem::Package to Gem::Package::TarWriter and use it at needed places. Feel free to ping me if you'll need any help with this.

@anthraxx

This comment has been minimized.

Contributor

anthraxx commented Oct 25, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment