Skip to content

Comments

[BEAM-2727] Make beam builds reproducible#3685

Closed
iemejia wants to merge 3 commits intoapache:masterfrom
iemejia:BEAM-2727-reproducible-builds
Closed

[BEAM-2727] Make beam builds reproducible#3685
iemejia wants to merge 3 commits intoapache:masterfrom
iemejia:BEAM-2727-reproducible-builds

Conversation

@iemejia
Copy link
Member

@iemejia iemejia commented Aug 4, 2017

Follow this checklist to help us incorporate your contribution quickly and easily:

  • Make sure there is a JIRA issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes.
  • Each commit in the pull request should have a meaningful subject line and body.
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Run mvn clean verify to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

@iemejia
Copy link
Member Author

iemejia commented Aug 4, 2017

R: @kennknowles
CC: @jbonofre

@iemejia iemejia force-pushed the BEAM-2727-reproducible-builds branch from 3d0e230 to 892fd27 Compare August 4, 2017 13:25
@coveralls
Copy link

Coverage Status

Changes Unknown when pulling 892fd27 on iemejia:BEAM-2727-reproducible-builds into ** on apache:master**.

<executions>
<execution>
<goals>
<goal>strip-jar</goal>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you describe the differences to the jar? I tried a little of this a while ago and was not sure about what was OK to remove.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well the thing is that java adds timestamps in some of the jar elements, e.g. the MANIFEST file and in some cases in the files that are updated with last modification date. The plugin covers these and other cases, you can take a look at the presentation he did (is really short) for some more detail. https://zlika.github.io/presentations/devoxx_fr_2016/reproducible-builds/slides_en.html#/_pom_properties

Copy link
Member

@kennknowles kennknowles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems generally good to me. I mostly am concerned about the maintenance of the plugin, having a dependency, etc. And I am not so expert in Maven so it would be good to solicit feedback from e.g. @davorbonaci. But I am comfortable merging and if anything goes wrong, we just revert.

I would only want to revert the part using the reproducible build plugin, but keep the changes for javadoc and the changes to the Java test. Consider splitting the commits so we can revert just the one. But it is easy enough to manually alter the pom so I am not that worried.

}

// delete the directory to not let stuff around after the test execution
FileUtils.deleteDirectory(baseDir);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably this should use @Rule TemporaryFolder

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, done.

This enables the Maven reproducible-build plugin that strips tiemstamps
and other information to achieve reproducible builds.
@iemejia iemejia force-pushed the BEAM-2727-reproducible-builds branch from 892fd27 to b8eddd2 Compare August 21, 2017 16:12
@iemejia
Copy link
Member Author

iemejia commented Aug 21, 2017

I agree with you, I have some preference for more community maintained plugins however this one makes the job and I reported one issue that was fixed and released really quickly so I think we can give it a try.
Please don't resolve/close the JIRA issue if this works because I would like still to add the JDK/maven information to the template vote. (I wonder if proposing the releaser to use the docker developer image might solve the reproduciblity issue, as well to force us to keep the docker image updated).

@coveralls
Copy link

Coverage Status

Coverage remained the same at 69.967% when pulling b8eddd2 on iemejia:BEAM-2727-reproducible-builds into d338b44 on apache:master.

@kennknowles
Copy link
Member

LGTM. Be sure to mention this when the next release thread comes up, as I guess if anything is going to show up as a problem, it will be in the built artifacts then.

@asfgit asfgit closed this in 62294e6 Aug 22, 2017
@iemejia iemejia deleted the BEAM-2727-reproducible-builds branch August 22, 2017 07:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants