8277069: [REDO] JDK-8276743 Make openjdk build Zip Archive generation "reproducible"

[magicus]

This is basically Andrew's old patch that was backed out, with a single change: I'm using ZipFile instead of ZipInputStream to read in the original zip. The latter is not capable of reading the extended attributes that contain the unix permissions. (Why this is so, is not fully clear to me. What's worse, it's by no means clear from the documentation. We should probably file a follow-up bug to improve the Javadoc for ZipInputStream, warning the users to stay away if they can.)

I have also added the possibility to opt-out of reproducible building by an argument to SetupZipArchive. This is used in the closed Oracle part of the build, where we create some temporary zip files that are later discarded, and not part of the deliverables. This saves us some unnecessary overhead.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8277069: [REDO] JDK-8276743 Make openjdk build Zip Archive generation "reproducible"

Reviewers

• Erik Joelsson (@erikj79 - Reviewer)

Contributors

• Andrew Leonard <aleonard@openjdk.org>
• Magnus Ihse Bursie <ihse@openjdk.org>

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/6673/head:pull/6673
$ git checkout pull/6673

Update a local copy of the PR:
$ git checkout pull/6673
$ git pull https://git.openjdk.java.net/jdk pull/6673/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 6673

View PR using the GUI difftool:
$ git pr show -t 6673

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/6673.diff

[bridgekeeper]

👋 Welcome back ihse! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@magicus The following label will be automatically applied to this pull request:

• build

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 00: Full (1e1ed849)

[magicus]

I tried to find a way to make GitHub display the difference wrt the original, backed out, patch, but utterly failed. So here's a simple diff. This is the only way in which this patch differs from the original.

--- ./make/jdk/src/classes/build/tools/makezipreproducible/MakeZipReproducible.java	2021-12-02 19:57:38.000000000 +0100
+++ ../../jdk-ALT/open/make/jdk/src/classes/build/tools/makezipreproducible/MakeZipReproducible.java	2021-12-02 15:11:22.000000000 +0100
@@ -162,13 +162,8 @@

     // Process input zip file and add to sorted entries set
     boolean processInputEntries(File inFile) throws IOException {
-        try (FileInputStream fis = new FileInputStream(inFile);
-             ZipInputStream  zis = new ZipInputStream(fis)) {
-            ZipEntry entry;
-            while ((entry = zis.getNextEntry()) != null) {
-                entries.put(entry.getName(), entry);
-            }
-        }
+        ZipFile zipFile = new ZipFile(inFile);
+        zipFile.stream().forEach(entry -> entries.put(entry.getName(), entry));

         return true;
     }

[magicus]

/contributor add aleonard
/contributor add ihse

[openjdk]

@magicus
Contributor Andrew Leonard <aleonard@openjdk.org> successfully added.

[openjdk]

@magicus
Contributor Magnus Ihse Bursie <ihse@openjdk.org> successfully added.

[openjdk]

@magicus This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8277069: [REDO] JDK-8276743 Make openjdk build Zip Archive generation "reproducible"

Co-authored-by: Andrew Leonard <aleonard@openjdk.org>
Co-authored-by: Magnus Ihse Bursie <ihse@openjdk.org>
Reviewed-by: erikj

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 12 new commits pushed to the master branch:

• 4f15be2: 8277965: Enclosing instance optimization affects serialization
• 8f196a2: 8278037: Clean up PPC32 related code in C1
• 8b042d1: 8257856: Make ClassFileVersionsTest.java robust to JDK version updates
• ad1ff27: 8277422: tools/jar/JarEntryTime.java fails with modified time mismatch
• 7c4ef3a: 8277985: G1: Compare max_parallel_refinement_threads to UINT_MAX
• dc2abc9: 8278080: Add --with-cacerts-src='user cacerts folder' to enable deterministic cacerts generation
• 8d9cb2e: 8278078: Cannot reference super before supertype constructor has been called
• 65960f7: 8277753: Long*VectorTests.java fail with "bad AD file" on x86_32 after JDK-8276162
• 3889af3: 8277906: Incorrect type for IV phi of long counted loops after CCP
• d2b16c8: 8276126: Dump time class transformation causes heap objects of non-boot classes to be archived
• ... and 2 more: https://git.openjdk.java.net/jdk/compare/b79554bb5cef14590d427543a40efbcc60c66548...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[magicus]

FWIW, I've opened https://bugs.openjdk.java.net/browse/JDK-8278165 to at least get some documentation on the limitations of ZipInputStream.

[magicus]

/integrate

[openjdk]

Going to push as commit c93552c.
Since your change was applied there have been 20 commits pushed to the master branch:

• b8ac0d2: 8276681: Additional malformed Javadoc inline tags in JDK source
• b310f30: 8277396: [TESTBUG] In DefaultButtonModelCrashTest.java, frame is accessed from main thread
• 15345e3: 8276661: (fs) UserDefinedFileAttributeView no longer works with long path (win)
• 652b5f8: 8276674: Malformed Javadoc inline tags in JDK source
• e0f1fc7: 8277358: Accelerate CRC32-C
• 73a9654: 8276806: Use Objects.checkFromIndexSize where possible in java.base
• 30087cc: 8275342: Change nested classes in java.prefs to static nested classes
• 669c90e: 8278099: two sun/security/pkcs11/Signature tests failed with AssertionError
• 4f15be2: 8277965: Enclosing instance optimization affects serialization
• 8f196a2: 8278037: Clean up PPC32 related code in C1
• ... and 10 more: https://git.openjdk.java.net/jdk/compare/b79554bb5cef14590d427543a40efbcc60c66548...master

Your commit was automatically rebased without conflicts.

[openjdk]

@magicus Pushed as commit c93552c.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.