Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8279536: jdk/nio/zipfs/ZipFSOutputStreamTest.java timed out #7010

Closed
wants to merge 1 commit into from

Conversation

jaikiran
Copy link
Member

@jaikiran jaikiran commented Jan 10, 2022

Can I please get a review for this test only change which helps drastically improve the time taken by the jdk/nio/zipfs/ZipFSOutputStreamTest.java testcase?

This jdk/nio/zipfs/ZipFSOutputStreamTest.java testcase was initially introduced in #4607 to reproduce and verify the bug fix for https://bugs.openjdk.java.net/browse/JDK-8190753. The test does the following:

  • Using ZipFS creates a zip file with entries of varying size. Each of these entries use random byte content.
  • One of the entry size is Integer.MAX_SIZE + 1 byte to trigger/verify the issue noted in JDK-8190753.
  • Finally the test verifies the contents of the generated entries in the zip file by matching it against the previously generated random content. The slowness resides in the way this testing/verification is done in this test case. Currently there's a loop which compares one byte at a time for each of these entries. This is unncessary and this adds up especially for the Integer.MAX_SIZE + 1 sized entry.

The change in this PR, improves this verification logic by using a fixed data to write out the entries and then read/verify that (fixed) content. This allows for better/quicker comparison of the entries.

Without this change, the testcase used to consistently take around 3 minutes 30 seconds to 3 minutes 50 seconds on my local setup. With this change the test now consistently completes (successfully) in just around 40 to 42 seconds (i.e. within a minute).

The original configuration of this test case uses a timeout of 300 seconds (5 minutes). That timeout was used based on the numbers I was seeing for this test completion. With this change, I don't expect it to require that much amount of time to complete (even on slow setups). However, I decided not to change that value just yet in this PR (since it anyway is a "maximum" time).

Additionally this PR adds some diagnostic logs for any future use if/when this test times out.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8279536: jdk/nio/zipfs/ZipFSOutputStreamTest.java timed out

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/7010/head:pull/7010
$ git checkout pull/7010

Update a local copy of the PR:
$ git checkout pull/7010
$ git pull https://git.openjdk.java.net/jdk pull/7010/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 7010

View PR using the GUI difftool:
$ git pr show -t 7010

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/7010.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Jan 10, 2022

👋 Welcome back jpai! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr label Jan 10, 2022
@openjdk
Copy link

@openjdk openjdk bot commented Jan 10, 2022

@jaikiran The following label will be automatically applied to this pull request:

  • nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the nio label Jan 10, 2022
@mlbridge
Copy link

@mlbridge mlbridge bot commented Jan 10, 2022

Webrevs

Copy link
Contributor

@LanceAndersen LanceAndersen left a comment

The changes look reasonable. I am going to run this a several hundred times to see where we get.

Copy link
Contributor

@LanceAndersen LanceAndersen left a comment

I think you are good to go.

I ran your change 1800 times across the various Mach5 platforms that we have without failure. 37 of the runs were over a minute, with the max run of 3 minutes, 27 seconds. Only 2 of the runs were over 3 minutes.

In case you are interested, here is the output from the slowest run:

Wrote entry f1 of bytes 2147483648 in 30330 milli seconds
Wrote entry f2 of bytes 26214400 in 889 milli seconds
Wrote entry d1/d2/f4 of bytes 0 in 0 milli seconds
Wrote entry d1/f3 of bytes 1234 in 0 milli seconds
Read entry f1 of bytes 2147483648 in 28988 milli seconds
Read entry f2 of bytes 26214400 in 62 milli seconds
Read entry d1/d2/f4 of bytes 0 in 0 milli seconds
Read entry d1/f3 of bytes 1234 in 0 milli seconds
test ZipFSOutputStreamTest.testOutputStream(java.util.ImmutableCollections$MapN@15951e35): success
config ZipFSOutputStreamTest.tearDown(): success
config ZipFSOutputStreamTest.setUp(): success
Wrote entry f1 of bytes 2147483648 in 11572 milli seconds
Wrote entry f2 of bytes 26214400 in 145 milli seconds
Wrote entry d1/d2/f4 of bytes 0 in 1 milli seconds
Wrote entry d1/f3 of bytes 1234 in 0 milli seconds
Read entry f1 of bytes 2147483648 in 4783 milli seconds
Read entry f2 of bytes 26214400 in 64 milli seconds
Read entry d1/d2/f4 of bytes 0 in 0 milli seconds
Read entry d1/f3 of bytes 1234 in 0 milli seconds
test ZipFSOutputStreamTest.testOutputStream(java.util.ImmutableCollections$MapN@2092a6c3): success

@openjdk
Copy link

@openjdk openjdk bot commented Jan 12, 2022

@jaikiran This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8279536: jdk/nio/zipfs/ZipFSOutputStreamTest.java timed out

Reviewed-by: lancea

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 44 new commits pushed to the master branch:

  • ece98d8: 8278461: Use Executable.getSharedParameterTypes() instead of Executable.getParameterTypes() in trusted code
  • 525b20f: 8279676: Dubious YMM register clearing in x86_64 arraycopy stubs
  • 4f0b650: 8278581: Improve reference processing statistics log output
  • bd339aa: 8277627: Fix copyright years in some jvmci files
  • 319d230: 8277463: JFileChooser with Metal L&F doesn't show non-canonical UNC path in - Look in
  • 13bfb49: 6496103: isFileHidingEnabled return false by default
  • f16f6a9: 8279821: JFR: Log warnings properly when loading a misconfigured .jfc file
  • 1c688f4: 8279900: compiler/vectorization/TestPopCountVectorLong.java fails due to vpopcntdq is not supported
  • 3aaa098: 8276694: Pattern trailing unescaped backslash causes internal error
  • 36f41cb: 8279884: Use better file for cygwin source permission check
  • ... and 34 more: https://git.openjdk.java.net/jdk/compare/11d88ce82efd72d3d63f7c7271c285cd21b01217...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Jan 12, 2022
@jaikiran
Copy link
Member Author

@jaikiran jaikiran commented Jan 12, 2022

Thank you Lance for the review and running extensive tests. The numbers look reasonable to me.

@jaikiran
Copy link
Member Author

@jaikiran jaikiran commented Jan 12, 2022

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Jan 12, 2022

Going to push as commit ff0cb98.
Since your change was applied there have been 44 commits pushed to the master branch:

  • ece98d8: 8278461: Use Executable.getSharedParameterTypes() instead of Executable.getParameterTypes() in trusted code
  • 525b20f: 8279676: Dubious YMM register clearing in x86_64 arraycopy stubs
  • 4f0b650: 8278581: Improve reference processing statistics log output
  • bd339aa: 8277627: Fix copyright years in some jvmci files
  • 319d230: 8277463: JFileChooser with Metal L&F doesn't show non-canonical UNC path in - Look in
  • 13bfb49: 6496103: isFileHidingEnabled return false by default
  • f16f6a9: 8279821: JFR: Log warnings properly when loading a misconfigured .jfc file
  • 1c688f4: 8279900: compiler/vectorization/TestPopCountVectorLong.java fails due to vpopcntdq is not supported
  • 3aaa098: 8276694: Pattern trailing unescaped backslash causes internal error
  • 36f41cb: 8279884: Use better file for cygwin source permission check
  • ... and 34 more: https://git.openjdk.java.net/jdk/compare/11d88ce82efd72d3d63f7c7271c285cd21b01217...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated label Jan 12, 2022
@openjdk openjdk bot closed this Jan 12, 2022
@openjdk openjdk bot removed ready rfr labels Jan 12, 2022
@openjdk
Copy link

@openjdk openjdk bot commented Jan 12, 2022

@jaikiran Pushed as commit ff0cb98.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@jaikiran jaikiran deleted the 8279536 branch Jan 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated nio
2 participants