Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8238650: Allow to override buildDate with SOURCE_DATE_EPOCH #99

Draft
wants to merge 1 commit into
base: master
from

Conversation

@bmwiedemann
Copy link

bmwiedemann commented Jan 29, 2020

Allow to override buildDate with SOURCE_DATE_EPOCH
in order to make builds reproducible.
See https://reproducible-builds.org/ for why this is good
and https://reproducible-builds.org/specs/source-date-epoch/
for the definition of this variable.

This PR was done while working on reproducible builds for openSUSE.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8238650: Allow to override buildDate with SOURCE_DATE_EPOCH

Download

$ git fetch https://git.openjdk.java.net/jfx pull/99/head:pull/99
$ git checkout pull/99

@bridgekeeper bridgekeeper bot added the oca label Jan 29, 2020
@bridgekeeper
Copy link

bridgekeeper bot commented Jan 29, 2020

Hi bmwiedemann, welcome to this OpenJDK project and thanks for contributing!

We do not recognize you as Contributor and need to ensure you have signed the Oracle Contributor Agreement (OCA). If you have not signed the OCA, please follow the instructions. Please fill in your GitHub username in the "Username" field of the application. Once you have signed the OCA, please let us know by writing /signed in a comment in this pull request.

If you already are an OpenJDK Author, Committer or Reviewer, please click here to open a new issue so that we can record that fact. Please use "Add GitHub user bmwiedemann" as summary for the issue.

If you are contributing this work on behalf of your employer and your employer has signed the OCA, please let us know by writing /covered in a comment in this pull request.

@kevinrushforth
Copy link
Member

kevinrushforth commented Jan 29, 2020

In addition to a signed OCA, we will need a bug filed to track this. Please read the CONTRIBUTING guidelines.

@bmwiedemann
Copy link
Author

bmwiedemann commented Jan 29, 2020

/signed

I filed one bug and got review ID : 9063488

@bridgekeeper bridgekeeper bot added the oca-verify label Jan 29, 2020
@bridgekeeper
Copy link

bridgekeeper bot commented Jan 29, 2020

Thank you! Please allow for up to two weeks to process your OCA, although it is usually done within one to two business days. Also, please note that pull requests that are pending an OCA check will not usually be evaluated, so your patience is appreciated!

@kevinrushforth
Copy link
Member

kevinrushforth commented Jan 29, 2020

I filed one bug and got review ID : 9063488

OK.

@bmwiedemann
Copy link
Author

bmwiedemann commented Feb 7, 2020

So the contributor agreement is done. How can I find the bug?

@kevinrushforth
Copy link
Member

kevinrushforth commented Feb 7, 2020

The OCA needs to be validated and recorded. As for the bug, it has been transferred to the JDK project as JDK-8238650.

in order to make builds reproducible.
See https://reproducible-builds.org/ for why this is good
and https://reproducible-builds.org/specs/source-date-epoch/
for the definition of this variable.

Signed-off-by: Bernhard M. Wiedemann <bwiedemann@suse.de>
@bmwiedemann bmwiedemann force-pushed the bmwiedemann:date branch from 2caff6e to e7f71fb Feb 7, 2020
@bmwiedemann bmwiedemann changed the title Allow to override buildDate with SOURCE_DATE_EPOCH 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH Feb 7, 2020
@openjdk openjdk bot added the rfr label Feb 7, 2020
@bmwiedemann
Copy link
Author

bmwiedemann commented Feb 7, 2020

The OCA needs to be validated and recorded.

https://www.oracle.com/technetwork/community/oca-486395.html#w now has
Bernhard Wiedemann - OpenJDK - bmwiedemann

As for the bug, it has been transferred to the JDK project as JDK-8238650.

Thanks, somehow I had not got an email notification for that.
Also worth noting that this PR only fixes one of the sources of non-determinism in openjfx. Do I have to open a separate bug for each of them?

@mlbridge
Copy link

mlbridge bot commented Feb 7, 2020

Webrevs

@kevinrushforth
Copy link
Member

kevinrushforth commented Feb 7, 2020

Do I have to open a separate bug for each of them?

Every fix (meaning each pull request) needs a unique bug ID.

@kevinrushforth
Copy link
Member

kevinrushforth commented Feb 7, 2020

/reviewers 2

@kevinrushforth kevinrushforth self-requested a review Feb 7, 2020
@openjdk
Copy link

openjdk bot commented Feb 7, 2020

@kevinrushforth
The number of required reviews for this PR is now set to 2 (with at least 1 of role reviewers).

@kevinrushforth
Copy link
Member

kevinrushforth commented Feb 7, 2020

As an optional override, I am OK with the concept of having a way for the build to be reproducible.

FWIW, I have scripts that will unpack the modular jar files and diff each class as well as doing the same for a src.zip, and it's pretty easy to tell if only VersionInfo (which is the class that records the time stamps) has changed.

I note that in practice, this is useful for a certain class of builds (e.g., CI or nightly test builds), but each released build is necessarily going to be different because you want a unique time stamp and build number associated with it.

I will review this (probably some time next week) and would like @johanvos to review as well.

@bmwiedemann
Copy link
Author

bmwiedemann commented Feb 8, 2020

FWIW, I have scripts that will unpack the modular jar files and diff each class

I agree that such specialized diff tools have some value, yet, there are also some limitations and downsides to them. E.g. you cannot simply tell another party what the expected sha256sum of a build result is.

https://www.suse.com/c/?p=42014 also has a section on problems with "the use of specialized comparison tools like [openSUSE's] ‘build-compare‘ "

I probably should write an FAQ entry about that topic...

each released build is necessarily going to be different because you want a unique time stamp and build number associated with it.

For release builds, it is important that other people can take the released sources and reproduce the same original binaries with the same release number (and ideally same timestamps) to easily verify that the build was clean (not corrupted by bad CPUs/RAM/HDDs or someone messing with the build machine).
I heard, some people even use that to save network bandwidth: add a small patch locally+remotely, build it locally, tell the world the new build hash, but have others upload their binaries with the right hash.

@bmwiedemann
Copy link
Author

bmwiedemann commented Feb 25, 2020

Hi, did you find time to review this?

@kevinrushforth
Copy link
Member

kevinrushforth commented Feb 25, 2020

No, I'm pretty backed up on reviews. It's on my queue, though.

Copy link
Member

kevinrushforth left a comment

Finally getting to this (sorry for the delay). This seems OK to me, as it is not an intrusive change. I do not think that actually setting this property for production builds is a good idea, so I recommend against that (ultimately that will be up to Gluon or whoever produces builds).

I'll test it once you change it from an env variable to a gradle property.

@@ -539,7 +539,7 @@ if (jfxReleasePatchVersion == "0") {
defineProperty("RELEASE_VERSION", relVer)
defineProperty("RELEASE_VERSION_PADDED", "${jfxReleaseMajorVersion}.${jfxReleaseMinorVersion}.${jfxReleaseSecurityVersion}.${jfxReleasePatchVersion}")

def buildDate = new java.util.Date()
def buildDate = System.getenv("SOURCE_DATE_EPOCH") == null ? new java.util.Date() : new java.util.Date(1000 * Long.parseLong(System.getenv("SOURCE_DATE_EPOCH")))

This comment has been minimized.

@kevinrushforth

kevinrushforth Feb 27, 2020 Member

This should be defined as a gradle property using defineProperty like this:

defineProperty("SOURCE_DATE_EPOCH", "")

You can then test for SOURCE_DATE_EPOCH == ""

You would pass it into the build via gradle -PSOURCE_DATE_EPOCH=...

I also recommend wrapping the line since it's pretty long.

While you are at it, can you add BUILD_TIMESTAMP to the list of properties that are logged? Look for log.quiet.

@kevinrushforth
Copy link
Member

kevinrushforth commented Apr 14, 2020

@bmwiedemann are you planning to pursue this PR?

@bmwiedemann
Copy link
Author

bmwiedemann commented Apr 15, 2020

Long term, yes.

I am currently low on time and my Java-foo is not that great, so I was hoping that someone would pick it up.

@kevinrushforth
Copy link
Member

kevinrushforth commented May 8, 2020

I note that the JDK build recently implemented this feature in JDK 15 with JDK-8244592.

@kevinrushforth kevinrushforth marked this pull request as draft Aug 26, 2020
@kevinrushforth
Copy link
Member

kevinrushforth commented Aug 26, 2020

Converted to a Draft PR, so that it will not show up as rfr. Go ahead and resubmit it when you are ready to proceed.

@openjdk openjdk bot removed the rfr label Aug 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.