Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8272984: javadoc support for reproducible builds #7171

Closed

Conversation

jonathan-gibbons
Copy link
Contributor

@jonathan-gibbons jonathan-gibbons commented Jan 21, 2022

Please review a javadoc update to support a new --date option to support reproducible builds.

This pull request supersedes #6905. In that PR, the SOURCE_DATE_EPOCH environment variable was used to provide the time stamp, but review feedback suggested the use of a new command-line option, --date. The format of the argument of the --date option is that same as that of similar options for the jar and jmod tools, and the code to handle the value is based on code in the jar tool.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed
  • Change requires a CSR request to be approved

Issues

  • JDK-8272984: javadoc support for reproducible builds
  • JDK-8279048: javadoc support for reproducible builds (CSR)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/7171/head:pull/7171
$ git checkout pull/7171

Update a local copy of the PR:
$ git checkout pull/7171
$ git pull https://git.openjdk.java.net/jdk pull/7171/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 7171

View PR using the GUI difftool:
$ git pr show -t 7171

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/7171.diff

@jonathan-gibbons
Copy link
Contributor Author

@jonathan-gibbons jonathan-gibbons commented Jan 21, 2022

/csr

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Jan 21, 2022

👋 Welcome back jjg! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added rfr csr labels Jan 21, 2022
@openjdk
Copy link

@openjdk openjdk bot commented Jan 21, 2022

@jonathan-gibbons an approved CSR request is already required for this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Jan 21, 2022

@jonathan-gibbons The following label will be automatically applied to this pull request:

  • javadoc

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the javadoc label Jan 21, 2022
@mlbridge
Copy link

@mlbridge mlbridge bot commented Jan 21, 2022

Webrevs

@@ -338,6 +352,34 @@ public boolean process(String opt, List<String> args) {
}
},

new XOption(resources, "--date", 1) {
// Valid --date range: within a year of now
Copy link
Contributor

@AlanBateman AlanBateman Jan 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding "--date" looks reasonable and makes it consistent with jar/jmod. I'm just not sure about restricting the date to "within a year of now". I assume projects will use the SOURCE_DATE_EPOCH to create a value for --date so they will run into issues if someone tries to build the project again, in a year.

Copy link
Contributor Author

@jonathan-gibbons jonathan-gibbons Jan 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change the range to 10 years.

I was expecting that the common use of reproducible builds was to compare builds created within a reasonably short period of time. I was wanting to set a range that was less than the seemingly arbitrary range of 1980-2099 allowed by the jar tool.

Copy link
Contributor

@AlanBateman AlanBateman Jan 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was expecting that the common use of reproducible builds was to compare builds created within a reasonably short period of time. I was wanting to set a range that was less than the seemingly arbitrary range of 1980-2099 allowed by the jar tool.

The restriction that the year be in the range 1980-2099 is because of the legacy ZIP format and the use of extended DOS time stamps. It would require introducing new APIs to support a wider date range. I don't think javadoc needs to put any restriction on the range. I don't have any objection to 10 years, I was mostly curious why it is needed.

Copy link
Contributor Author

@jonathan-gibbons jonathan-gibbons Jan 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was following the jmod code, which uses a range, and I guessed (apparently correctly) that the chosen range was zip-specific.

I think it is worthwhile having a sanity-check on the provided range.

Copy link
Member

@hns hns left a comment

Generally looks good, but I have a few questions and found a lone typo.

String arg = args.get(0);
try {
date = ZonedDateTime.parse(arg, DateTimeFormatter.ISO_ZONED_DATE_TIME)
.withZoneSameInstant(ZoneOffset.UTC);
Copy link
Member

@hns hns Jan 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I notice that the new option takes a date-time with timezone, but the timezone is discarded here (and the date-time is later converted to a java.util.Date). Since the purpose of this change is to enable reproducible builds, I guess it should be possible to define the timezone used in the "Generated by" comment (which as far as I can tell is the timezone of the local host).

Copy link
Contributor Author

@jonathan-gibbons jonathan-gibbons Jan 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went out of my way to ensure that the output form was not affected, in case there is anyone relying on the existing format ... and yes, we have tests for that as well.

I'd be open to formally changing the format of the output timestamp, but I think that should be a separate RFE.

Copy link
Member

@hns hns Jan 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant was not to change the format used in javadoc output but rather to preserve the time zone of the timestamp passed as --date option argument. Since java.util.Date.toString() always uses the default time zone, to reproduce a documentation bundle generated in a different time zone one would have to set the default time zone in addition to using the --date option. This could be avoided by retaining the ZonedDateTime with the original time zone instead of converting it to a java.util.Date. The following DateTimeFormatter instance could then be used to generate a timestamp string identical to the one produced by Date.toString() but with the correct time zone:

 DateTimeFormatter.ofPattern("EEE MMM dd HH:mm:ss zzz yyyy").withLocale(Locale.US);

Copy link
Contributor Author

@jonathan-gibbons jonathan-gibbons Jan 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion; that helps.

Improve command-line help message
Clean up imports in new test
hns
hns approved these changes Jan 26, 2022
Copy link
Member

@hns hns left a comment

Looks good to me.

The CSR should be updated to reflect the changed option help message. Talking about that change: I'm a bit unsure about it, as ISO 8601 does not include the optional zone ID, which is an extension in DateTimeFormatter.ISO_ZONED_DATE_TIME. So maybe the previous message was more correct and useful after all?

@jonathan-gibbons
Copy link
Contributor Author

@jonathan-gibbons jonathan-gibbons commented Jan 26, 2022

Looks good to me.

The CSR should be updated to reflect the changed option help message. Talking about that change: I'm a bit unsure about it, as ISO 8601 does not include the optional zone ID, which is an extension in DateTimeFormatter.ISO_ZONED_DATE_TIME. So maybe the previous message was more correct and useful after all?

The tool guide (man page) is a good place to discuss the exact format, give examples, and maybe even tie it back to SOURCE_DATE_EPOCH.

@openjdk openjdk bot removed the csr label Jan 28, 2022
@openjdk
Copy link

@openjdk openjdk bot commented Jan 28, 2022

@jonathan-gibbons This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8272984: javadoc support for reproducible builds

Reviewed-by: hannesw

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 87 new commits pushed to the master branch:

  • a1d1e47: 8280823: Remove NULL check in DumpTimeClassInfo::is_excluded
  • 094db1a: 8277948: AArch64: Print the correct native stack if -XX:+PreserveFramePointer when crash
  • 7857405: 8280744: Allow SuppressWarnings to be used in all declaration contexts
  • 40a2ce2: 8270476: Make floating-point test infrastructure more lambda and method reference friendly
  • 6d242e4: 8280835: jdk/javadoc/tool/CheckManPageOptions.java depends on source hierarchy
  • ece89c6: 8280366: (fs) Restore Files.createTempFile javadoc
  • b94ebaa: 8280686: Remove Compile::print_method_impl
  • a3a0dcd: 8280353: -XX:ArchiveClassesAtExit should print warning if base archive failed to load
  • cab5905: 8280583: Always build NMT
  • 7f68759: 8280719: G1: Remove outdated comment in RemoveSelfForwardPtrObjClosure::apply
  • ... and 77 more: https://git.openjdk.java.net/jdk/compare/293fb46f7cd28f2a08055e3eb8ec9459d64e9688...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Jan 28, 2022
@jonathan-gibbons
Copy link
Contributor Author

@jonathan-gibbons jonathan-gibbons commented Jan 31, 2022

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Jan 31, 2022

Going to push as commit 96d0df7.
Since your change was applied there have been 114 commits pushed to the master branch:

  • ee3be0b: 8280488: doclint reference checks withstand warning suppression
  • 74921e8: 8280738: Minor cleanup for HtmlStyle
  • 3916561: 8280543: Update the "java" and "jcmd" tool specification for CDS
  • f991891: 8280949: Correct the references for the Java Security Standard Algorithm Names specification
  • 319b774: 8277101: jcmd VM.cds dynamic_dump should not regenerate holder classes
  • 993a248: 8280450: Add task queue printing to STW Full GCs
  • dcc666d: 8280139: Report more detailed statistics about task stealing in task queue stats
  • bdda43e: 8280705: Parallel: Full gc mark stack draining should prefer to make work available to other threads
  • 091aff9: 8278908: [macOS] Unexpected text normalization on pasting from clipboard
  • 61794c5: 8280817: Clean up and unify empty VM operations
  • ... and 104 more: https://git.openjdk.java.net/jdk/compare/293fb46f7cd28f2a08055e3eb8ec9459d64e9688...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated label Jan 31, 2022
@openjdk openjdk bot closed this Jan 31, 2022
@openjdk openjdk bot removed ready rfr labels Jan 31, 2022
@openjdk
Copy link

@openjdk openjdk bot commented Jan 31, 2022

@jonathan-gibbons Pushed as commit 96d0df7.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@jonathan-gibbons jonathan-gibbons deleted the 8272984.date-option branch Jan 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated javadoc
3 participants