8308116: jdk.test.lib.compiler.InMemoryJavaCompiler.compile does not close files #14001

xmas92 · 2023-05-16T08:06:15Z

jdk.test.lib.compiler.InMemoryJavaCompiler.compile(String className, CharSequence sourceCode, String... options) creates a JavaFileManager but fails to close it after the CompilationTask is completed, leaving the file descriptors open until reference processing kicks in.

Graph below shows number of open file descriptors before and after this patch for a program which compiles dummy classes in a loop.

Testing: GHA and Oracle CI tier1-3. Tests using jdk.test.lib.compiler.InMemoryJavaCompiler also tested locally.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8308116: jdk.test.lib.compiler.InMemoryJavaCompiler.compile does not close files

Reviewers

Leonid Mesnik (@lmesnik - Reviewer) ⚠️ Review applies to a939e0c9
Stefan Karlsson (@stefank - Reviewer)
Jan Lahoda (@lahodaj - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/14001/head:pull/14001
$ git checkout pull/14001

Update a local copy of the PR:
$ git checkout pull/14001
$ git pull https://git.openjdk.org/jdk.git pull/14001/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 14001

View PR using the GUI difftool:
$ git pr show -t 14001

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/14001.diff

Webrev

Link to Webrev Comment

…close files

bridgekeeper · 2023-05-16T08:08:05Z

👋 Welcome back aboldtch! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2023-05-16T08:09:39Z

@xmas92 The following label will be automatically applied to this pull request:

hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2023-05-16T08:13:46Z

Webrevs

lmesnik

There are few nits inline.
Also, please update copyright years.

lmesnik · 2023-05-16T19:52:52Z

test/lib/jdk/test/lib/compiler/InMemoryJavaCompiler.java

+        return CloseableCompilationTask.createTask(null, new FileManagerWrapper(file, moduleOverride), null, opts, null, Arrays.asList(file));
+    }
+
+    private static class CloseableCompilationTask implements Closeable {


Does it make a sense to implement Callable as well? Just to make semantic clearer. (It is just suggestion.)

lmesnik · 2023-05-16T19:55:39Z

test/lib/jdk/test/lib/compiler/InMemoryJavaCompiler.java

+            return new CloseableCompilationTask(task, fileManager);
+        }
+
+        public void close() throws IOException {


Please add tag Override.

Might it be better to wrap IOException into RuntimeException here and don't have any 'throws' but it is up to you.

openjdk · 2023-05-16T20:04:02Z

@xmas92 This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8308116: jdk.test.lib.compiler.InMemoryJavaCompiler.compile does not close files

Reviewed-by: lmesnik, stefank, jlahoda

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 119 new commits pushed to the master branch:

eb11508: 8308281: Java snippets in the FFM API need to be updated
26227a6: 8305073: Fix VerifyLoopOptimizations - step 2 - verify idom
80d7de7: 8305582: Compiler crash when compiling record patterns with var
e559613: 8308500: ZStatSubPhase::register_start should not call register_gc_phase_start if ZAbort::should_abort()
bdd2402: 8260943: C2 SuperWord: Remove dead vectorization optimization added by 8076284
4f0f776: 8308403: [s390x] separate remaining_cargs from z_abi_160
06b0a5e: 8302652: [SuperWord] Reduction should happen after loop, when possible
69f508a: 8308300: enhance exceptions in MappedMemoryUtils.c
c440827: 8308093: Disable language preview features use in JDK
422128b: 8306992: [JVMCI] mitigate more against JVMCI related OOME causing VM to exit
... and 109 more: https://git.openjdk.org/jdk/compare/01892f9c6ed1ff6593f77b89fcc2c5da8d094abf...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

test/lib/jdk/test/lib/compiler/InMemoryJavaCompiler.java

Co-authored-by: Andrey Turbanov <turbanoff@gmail.com>

This reverts commit 3223959.

This reverts commit 932ebc0.

xmas92 · 2023-05-22T16:03:32Z

Got the suggestion to close the fileManager directly. It require some extra refactoring. But does not change the class hierarchy so no test need to be changed. Creating a cleaner patch.

There are some parts that could be extracted into helper methods, but the compile method seems small enough to not require this. And it is harder to keep such a refactoring clean without introducing changes to the class hierarchy.

lahodaj

Looks reasonable.

xmas92 · 2023-05-23T14:55:19Z

Thanks for the reviews
/integrate

openjdk · 2023-05-23T15:34:12Z

Going to push as commit e9320f3.
Since your change was applied there have been 125 commits pushed to the master branch:

97d3b27: 8307523: [vectorapi] Optimize MaskFromLongBenchmark.java
bb0ff48: 8305091: Change ChaCha20 cipher init behavior to match AES-GCM
c0c4d77: 8308544: Fix compilation regression from JDK-8306983 on musl libc
9e196b3: 8308565: HttpClient: Sanitize logging while stopping
582ddeb: 8308545: java/net/httpclient/ShutdownNow.java fails with "stream 1 cancelled"
1cfb265: 8307814: In the case of two methods with Record Patterns, the second one contains a line number from the first method
eb11508: 8308281: Java snippets in the FFM API need to be updated
26227a6: 8305073: Fix VerifyLoopOptimizations - step 2 - verify idom
80d7de7: 8305582: Compiler crash when compiling record patterns with var
e559613: 8308500: ZStatSubPhase::register_start should not call register_gc_phase_start if ZAbort::should_abort()
... and 115 more: https://git.openjdk.org/jdk/compare/01892f9c6ed1ff6593f77b89fcc2c5da8d094abf...master

Your commit was automatically rebased without conflicts.

openjdk · 2023-05-23T15:36:22Z

@xmas92 Pushed as commit e9320f3.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

xmas92 added 2 commits May 15, 2023 18:39

8308116: jdk.test.lib.compiler.InMemoryJavaCompiler.compile does not …

f53c52b

…close files

Fix tests

3223959

openjdk bot added the rfr Pull request is ready for review label May 16, 2023

openjdk bot added the hotspot-runtime hotspot-runtime-dev@openjdk.org label May 16, 2023

lmesnik approved these changes May 16, 2023

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label May 16, 2023

xmas92 added 3 commits May 17, 2023 06:42

Feedback: Update Copyright years

932ebc0

Feedback: Add @OverRide and Callable

6cd9dc8

Feedback: Throw RuntimeException in close

a939e0c

lmesnik approved these changes May 17, 2023

View reviewed changes

turbanoff reviewed May 18, 2023

View reviewed changes

test/lib/jdk/test/lib/compiler/InMemoryJavaCompiler.java Outdated Show resolved Hide resolved

xmas92 and others added 4 commits May 19, 2023 13:30

Apply suggestions from code review

99fde73

Co-authored-by: Andrey Turbanov <turbanoff@gmail.com>

Revert "Fix tests"

63844aa

This reverts commit 3223959.

Revert "Feedback: Update Copyright years"

8655bc4

This reverts commit 932ebc0.

Refactor to close JavaFileManager directly

78740c6

stefank approved these changes May 23, 2023

View reviewed changes

lahodaj approved these changes May 23, 2023

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label May 23, 2023

openjdk bot closed this May 23, 2023

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels May 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8308116: jdk.test.lib.compiler.InMemoryJavaCompiler.compile does not close files #14001

8308116: jdk.test.lib.compiler.InMemoryJavaCompiler.compile does not close files #14001

xmas92 commented May 16, 2023 •

edited by openjdk bot

Loading

bridgekeeper bot commented May 16, 2023

openjdk bot commented May 16, 2023

mlbridge bot commented May 16, 2023 •

edited

Loading

lmesnik left a comment

lmesnik May 16, 2023

lmesnik May 16, 2023

lmesnik May 16, 2023

openjdk bot commented May 16, 2023 •

edited

Loading

xmas92 commented May 22, 2023

lahodaj left a comment

xmas92 commented May 23, 2023

openjdk bot commented May 23, 2023

openjdk bot commented May 23, 2023

8308116: jdk.test.lib.compiler.InMemoryJavaCompiler.compile does not close files #14001

8308116: jdk.test.lib.compiler.InMemoryJavaCompiler.compile does not close files #14001

Conversation

xmas92 commented May 16, 2023 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

Webrev

bridgekeeper bot commented May 16, 2023

openjdk bot commented May 16, 2023

mlbridge bot commented May 16, 2023 • edited Loading

Webrevs

lmesnik left a comment

Choose a reason for hiding this comment

lmesnik May 16, 2023

Choose a reason for hiding this comment

lmesnik May 16, 2023

Choose a reason for hiding this comment

lmesnik May 16, 2023

Choose a reason for hiding this comment

openjdk bot commented May 16, 2023 • edited Loading

xmas92 commented May 22, 2023

lahodaj left a comment

Choose a reason for hiding this comment

xmas92 commented May 23, 2023

openjdk bot commented May 23, 2023

openjdk bot commented May 23, 2023

xmas92 commented May 16, 2023 •

edited by openjdk bot

Loading

mlbridge bot commented May 16, 2023 •

edited

Loading

openjdk bot commented May 16, 2023 •

edited

Loading