8279536: jdk/nio/zipfs/ZipFSOutputStreamTest.java timed out

[jaikiran]

Can I please get a review for this test only change which helps drastically improve the time taken by the jdk/nio/zipfs/ZipFSOutputStreamTest.java testcase?

This jdk/nio/zipfs/ZipFSOutputStreamTest.java testcase was initially introduced in #4607 to reproduce and verify the bug fix for https://bugs.openjdk.java.net/browse/JDK-8190753. The test does the following:

• Using ZipFS creates a zip file with entries of varying size. Each of these entries use random byte content.
• One of the entry size is Integer.MAX_SIZE + 1 byte to trigger/verify the issue noted in JDK-8190753.
• Finally the test verifies the contents of the generated entries in the zip file by matching it against the previously generated random content. The slowness resides in the way this testing/verification is done in this test case. Currently there's a loop which compares one byte at a time for each of these entries. This is unncessary and this adds up especially for the Integer.MAX_SIZE + 1 sized entry.

The change in this PR, improves this verification logic by using a fixed data to write out the entries and then read/verify that (fixed) content. This allows for better/quicker comparison of the entries.

Without this change, the testcase used to consistently take around 3 minutes 30 seconds to 3 minutes 50 seconds on my local setup. With this change the test now consistently completes (successfully) in just around 40 to 42 seconds (i.e. within a minute).

The original configuration of this test case uses a timeout of 300 seconds (5 minutes). That timeout was used based on the numbers I was seeing for this test completion. With this change, I don't expect it to require that much amount of time to complete (even on slow setups). However, I decided not to change that value just yet in this PR (since it anyway is a "maximum" time).

Additionally this PR adds some diagnostic logs for any future use if/when this test times out.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8279536: jdk/nio/zipfs/ZipFSOutputStreamTest.java timed out

Reviewers

• Lance Andersen (@LanceAndersen - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/7010/head:pull/7010
$ git checkout pull/7010

Update a local copy of the PR:
$ git checkout pull/7010
$ git pull https://git.openjdk.java.net/jdk pull/7010/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 7010

View PR using the GUI difftool:
$ git pr show -t 7010

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/7010.diff

[bridgekeeper]

👋 Welcome back jpai! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@jaikiran The following label will be automatically applied to this pull request:

• nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 00: Full (58d8dd37)

[LanceAndersen]

The changes look reasonable. I am going to run this a several hundred times to see where we get.

[LanceAndersen]

I think you are good to go.

I ran your change 1800 times across the various Mach5 platforms that we have without failure. 37 of the runs were over a minute, with the max run of 3 minutes, 27 seconds. Only 2 of the runs were over 3 minutes.

In case you are interested, here is the output from the slowest run:

Wrote entry f1 of bytes 2147483648 in 30330 milli seconds
Wrote entry f2 of bytes 26214400 in 889 milli seconds
Wrote entry d1/d2/f4 of bytes 0 in 0 milli seconds
Wrote entry d1/f3 of bytes 1234 in 0 milli seconds
Read entry f1 of bytes 2147483648 in 28988 milli seconds
Read entry f2 of bytes 26214400 in 62 milli seconds
Read entry d1/d2/f4 of bytes 0 in 0 milli seconds
Read entry d1/f3 of bytes 1234 in 0 milli seconds
test ZipFSOutputStreamTest.testOutputStream(java.util.ImmutableCollections$MapN@15951e35): success
config ZipFSOutputStreamTest.tearDown(): success
config ZipFSOutputStreamTest.setUp(): success
Wrote entry f1 of bytes 2147483648 in 11572 milli seconds
Wrote entry f2 of bytes 26214400 in 145 milli seconds
Wrote entry d1/d2/f4 of bytes 0 in 1 milli seconds
Wrote entry d1/f3 of bytes 1234 in 0 milli seconds
Read entry f1 of bytes 2147483648 in 4783 milli seconds
Read entry f2 of bytes 26214400 in 64 milli seconds
Read entry d1/d2/f4 of bytes 0 in 0 milli seconds
Read entry d1/f3 of bytes 1234 in 0 milli seconds
test ZipFSOutputStreamTest.testOutputStream(java.util.ImmutableCollections$MapN@2092a6c3): success

[openjdk]

@jaikiran This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8279536: jdk/nio/zipfs/ZipFSOutputStreamTest.java timed out

Reviewed-by: lancea

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 44 new commits pushed to the master branch:

• ece98d8: 8278461: Use Executable.getSharedParameterTypes() instead of Executable.getParameterTypes() in trusted code
• 525b20f: 8279676: Dubious YMM register clearing in x86_64 arraycopy stubs
• 4f0b650: 8278581: Improve reference processing statistics log output
• bd339aa: 8277627: Fix copyright years in some jvmci files
• 319d230: 8277463: JFileChooser with Metal L&F doesn't show non-canonical UNC path in - Look in
• 13bfb49: 6496103: isFileHidingEnabled return false by default
• f16f6a9: 8279821: JFR: Log warnings properly when loading a misconfigured .jfc file
• 1c688f4: 8279900: compiler/vectorization/TestPopCountVectorLong.java fails due to vpopcntdq is not supported
• 3aaa098: 8276694: Pattern trailing unescaped backslash causes internal error
• 36f41cb: 8279884: Use better file for cygwin source permission check
• ... and 34 more: https://git.openjdk.java.net/jdk/compare/11d88ce82efd72d3d63f7c7271c285cd21b01217...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[jaikiran]

Thank you Lance for the review and running extensive tests. The numbers look reasonable to me.

[jaikiran]

/integrate

[openjdk]

Going to push as commit ff0cb98.
Since your change was applied there have been 44 commits pushed to the master branch:

• ece98d8: 8278461: Use Executable.getSharedParameterTypes() instead of Executable.getParameterTypes() in trusted code
• 525b20f: 8279676: Dubious YMM register clearing in x86_64 arraycopy stubs
• 4f0b650: 8278581: Improve reference processing statistics log output
• bd339aa: 8277627: Fix copyright years in some jvmci files
• 319d230: 8277463: JFileChooser with Metal L&F doesn't show non-canonical UNC path in - Look in
• 13bfb49: 6496103: isFileHidingEnabled return false by default
• f16f6a9: 8279821: JFR: Log warnings properly when loading a misconfigured .jfc file
• 1c688f4: 8279900: compiler/vectorization/TestPopCountVectorLong.java fails due to vpopcntdq is not supported
• 3aaa098: 8276694: Pattern trailing unescaped backslash causes internal error
• 36f41cb: 8279884: Use better file for cygwin source permission check
• ... and 34 more: https://git.openjdk.java.net/jdk/compare/11d88ce82efd72d3d63f7c7271c285cd21b01217...master

Your commit was automatically rebased without conflicts.

[openjdk]

@jaikiran Pushed as commit ff0cb98.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.