Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8265078: jpackage tests on Windows leave large temp files #3473

Closed
wants to merge 3 commits into from

Conversation

@andyherrick
Copy link

@andyherrick andyherrick commented Apr 13, 2021

two changes:
One to jpackage, when recursively removing directory, when IOException occurs, record it and continue (deleting as much as possible) before throwing the exception.
The other to tests, when running jpackage via ProcessBuilder.execute(), set the "TMP" environment variable to the current value of System Property "java.io.tmpdir". This causes the sub-process (jpackage) to output tmp files to the tmp file location used by the test. (So the test harness can clean up after test exits).


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8265078: jpackage tests on Windows leave large temp files

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/3473/head:pull/3473
$ git checkout pull/3473

Update a local copy of the PR:
$ git checkout pull/3473
$ git pull https://git.openjdk.java.net/jdk pull/3473/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 3473

View PR using the GUI difftool:
$ git pr show -t 3473

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/3473.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Apr 13, 2021

👋 Welcome back herrick! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr label Apr 13, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Apr 13, 2021

@andyherrick The following label will be automatically applied to this pull request:

  • core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the core-libs label Apr 13, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Apr 13, 2021

Webrevs

Copy link
Member

@kevinrushforth kevinrushforth left a comment

Are you sure you need an AtomicReference to set the exception? I don't see any use of multiple threads, but I might be missing something. If you do need it, you need to fix the order of arguments.

Copy link
Member

@kevinrushforth kevinrushforth left a comment

This looks fine. I see that this bug is listed as Windows-specific. Are any similar changes needed for other platforms, or do they already write test files only to java.io.tmpdir?

@andyherrick
Copy link
Author

@andyherrick andyherrick commented Apr 14, 2021

Although jpackage failing to clean up has only been seen as an issue on Windows, allowing it to write outside the test directory (in default tmp dir) still exists on mac and linux - I will look into enhancing (what is now setWindowsTmpDir()) to be usefull on all platforms.

@alexeysemenyukoracle
Copy link
Member

@alexeysemenyukoracle alexeysemenyukoracle commented Apr 14, 2021

Are you sure you need an AtomicReference to set the exception? I don't see any use of multiple threads, but I might be missing something. If you do need it, you need to fix the order of arguments.

postVisitDirectory() and visitFile() methods can be potentially called concurrently by walkFileTree(). Javadoc https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/file/Files.html#walkFileTree(java.nio.file.Path,java.util.Set,int,java.nio.file.FileVisitor) doesn't say anything about synchronization, so I think it is fair to assume it is not guaranteed.

@openjdk
Copy link

@openjdk openjdk bot commented Apr 14, 2021

@andyherrick This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8265078: jpackage tests on Windows leave large temp files

Reviewed-by: asemenyuk, kcr

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 83 new commits pushed to the master branch:

  • 4c83d24: 8058176: [mlvm] tests should not allow code cache exhaustion
  • 9406744: 8264976: Minor numeric bug in AbstractSplittableWithBrineGenerator.makeSplitsSpliterator
  • 80026d8: 8265174: Update Class.getDeclaredMethods to discuss synthetic and bridge methods
  • ffb3771: 8265019: Update tests for additional TestNG test permissions
  • 7e4cd48: 8264821: DirectIOTest fails on a system with large block size
  • 0afcbd4: 8264412: AArch64: CPU description should refer DMI
  • 4661690: 8262883: doccheck: Broken links in java.base
  • 75b039a: 8263970: Manual test javax/swing/JTextField/JapaneseReadingAttributes/JapaneseReadingAttributes.java failed
  • f5b2f08: 8257836: Add additional test cases to TestSyncOnValueBasedClassEvent.java
  • 27dd88b: 8262957: (fs) Fail fast in UnixFileStore.isExtendedAttributesEnabled
  • ... and 73 more: https://git.openjdk.java.net/jdk/compare/5bd6c7454738d0a8a395cfe27a364cdb325eb37c...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Apr 14, 2021
@kevinrushforth
Copy link
Member

@kevinrushforth kevinrushforth commented Apr 14, 2021

postVisitDirectory() and visitFile() methods can be potentially called concurrently by walkFileTree()

I don't think they can.

Javadoc ... doesn't say anything about synchronization, so I think it is fair to assume it is not guaranteed.

Given that the Files class says nothing about running its various file walking operations in parallel, I think you can be confident that the visitor methods are only ever run on the same thread that calls walkFileTree. I would consider it a bug if the thread used to callback into the visitor were different from the calling thread.

Still, I think using AtomicReference is fine here, so this is a moot point for this PR.

@andyherrick
Copy link
Author

@andyherrick andyherrick commented Apr 14, 2021

/integrate

@openjdk openjdk bot closed this Apr 14, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Apr 14, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Apr 14, 2021

@andyherrick Since your change was applied there have been 86 commits pushed to the master branch:

  • 05f851e: 8265236: ProblemList java/foreign/TestUpcall.java on macosx-aarch64
  • d1b28e7: 8265231: (fc) ReadDirect and WriteDirect tests fail after fix for JDK-8264821
  • 57f86a0: 8265235: ProblemList java/foreign/TestIntrinsics.java on macosx-aarch64
  • 4c83d24: 8058176: [mlvm] tests should not allow code cache exhaustion
  • 9406744: 8264976: Minor numeric bug in AbstractSplittableWithBrineGenerator.makeSplitsSpliterator
  • 80026d8: 8265174: Update Class.getDeclaredMethods to discuss synthetic and bridge methods
  • ffb3771: 8265019: Update tests for additional TestNG test permissions
  • 7e4cd48: 8264821: DirectIOTest fails on a system with large block size
  • 0afcbd4: 8264412: AArch64: CPU description should refer DMI
  • 4661690: 8262883: doccheck: Broken links in java.base
  • ... and 76 more: https://git.openjdk.java.net/jdk/compare/5bd6c7454738d0a8a395cfe27a364cdb325eb37c...master

Your commit was automatically rebased without conflicts.

Pushed as commit e167577.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@andyherrick andyherrick deleted the andyherrick:JDK-8265078 branch Jun 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants