8308341: JNI_GetCreatedJavaVMs returns a partially initialized JVM #14139

dholmes-ora · 2023-05-25T05:02:19Z

We now track the in-progress and completed states of VM creation and only return a VM from JNI_GetCreatedJavaVMs when there is a fully initialized VM.

Testing:

new regression test
tiers 1-3 (sanity)

Thanks

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue
Change requires CSR request JDK-8308816 to be approved

Issues

JDK-8308341: JNI_GetCreatedJavaVMs returns a partially initialized JVM
JDK-8308816: JNI_GetCreatedJavaVMs returns a partially initialized JVM (CSR)

Reviewers

Johan Sjölen (@jdksjolen - Reviewer) ⚠️ Review applies to 8f6aa1f7
Gerard Ziemski (@gerard-ziemski - Committer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/14139/head:pull/14139
$ git checkout pull/14139

Update a local copy of the PR:
$ git checkout pull/14139
$ git pull https://git.openjdk.org/jdk.git pull/14139/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 14139

View PR using the GUI difftool:
$ git pr show -t 14139

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/14139.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2023-05-25T05:03:43Z

👋 Welcome back dholmes! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2023-05-25T05:06:18Z

@dholmes-ora The following labels will be automatically applied to this pull request:

build
hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

mlbridge · 2023-05-25T05:10:20Z

Webrevs

01: Full - Incremental (2bf95d8c)
00: Full (8f6aa1f7)

jdksjolen

Hi, over all looks good, but I have some questions regarding the tests and I wonder whether you missed a case.

src/hotspot/share/prims/jni.cpp

jdksjolen · 2023-05-25T09:56:39Z

src/hotspot/share/prims/jni.cpp

+  // We can be called by native libraries in the JDK during VM
+  // initialization, so only bail-out if something seems very wrong.
+  // Though how would we get here in that case?
+  if (vm_created == NOT_CREATED) {


Shouldn't we also handle IN_PROGRESS?

I guess my comment was not clear enough. :) It is normal to get here while IN_PROGRESS due to calls from the JDK native code during initialization. So we only consider it an error to call then when NOT_CREATED.

Sloppy reading from me! That makes sense, and the code doing the work (L3960-L3965) seems to take all precautions necessary to ensure that nothing goes wrong if we're IN_PROGRESS.

Just a minor suggestion: "during VM initialization" could be "while VM initialization is in progress" to mirror the language in the enum, but that's up to you.

jdksjolen · 2023-05-25T10:16:06Z

test/hotspot/jtreg/runtime/jni/getCreatedJavaVMs/exeGetCreatedJavaVMs.c

@@ -0,0 +1,104 @@
+/*


Can we make this test behave even worse than it already does and have you caused a crash with this test before applying your changes? If it has caused a crash, then I think we're OK with the current form of the test.

Two ideas to make it more likely to crash:

Have more threads racing

Let's say you spawn more threads, is it possible for printf to prevent data races as it's MT-Safe? More threads would increase likelihood of contention on the implicit lock, leading to some threads having been significantly slowed down by having to perform a more expensive locking procedure, is my thought process.

The test crashes an unpatched VM. We only need the two threads. One will initiate the creation of the VM and the other then tries to attach to it. We want the second thread to call GetCreatedJavaVMs very quickly to make it more likely the VM is not initialized, but not so quickly that GetCreatedJavaVMs reports zero VMs (regardless of the patch). The time it takes the main thread to create the second thread seems to handle that nicely. Of course it depends on number of CPUs etc but on my local machine the test, and the original version from the JBS issue, reliably crashes every time before the fix.

Alright, then I'm very happy with this.

dholmes-ora · 2023-05-26T01:16:38Z

Thanks for looking at this @jdksjolen !

jdksjolen

Thank you for this patch, everything looks good to me )including the build changes).

jdksjolen · 2023-05-26T08:36:46Z

src/hotspot/share/prims/jni.cpp

+  // We can be called by native libraries in the JDK during VM
+  // initialization, so only bail-out if something seems very wrong.
+  // Though how would we get here in that case?
+  if (vm_created == NOT_CREATED) {


Sloppy reading from me! That makes sense, and the code doing the work (L3960-L3965) seems to take all precautions necessary to ensure that nothing goes wrong if we're IN_PROGRESS.

Just a minor suggestion: "during VM initialization" could be "while VM initialization is in progress" to mirror the language in the enum, but that's up to you.

jdksjolen · 2023-05-26T08:38:09Z

test/hotspot/jtreg/runtime/jni/getCreatedJavaVMs/exeGetCreatedJavaVMs.c

@@ -0,0 +1,104 @@
+/*


Alright, then I'm very happy with this.

openjdk · 2023-05-26T08:42:58Z

@dholmes-ora This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8308341: JNI_GetCreatedJavaVMs returns a partially initialized JVM

Reviewed-by: jsjolen, gziemski

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 326 new commits pushed to the master branch:

fb0b1f0: 8051725: Improve expansion of Conv2B nodes in the middle-end
3eced01: 8304074: [JMX] Add an approximation of total bytes allocated on the Java heap by the JVM
15e0285: 8309110: Build failure after JDK-8307795 due to warnings in micro-benchmark StoreMaskTrueCount.java
4526282: 8308977: gtest:codestrings fails on riscv
f600d03: 8307795: AArch64: Optimize VectorMask.truecount() on Neon
07f2070: 8309095: Remove UTF-8 character from TaskbarPositionTest.java
2b186e2: 8309042: MemorySegment::reinterpret cleanup action is not called for all overloads
78aac24: 8308881: Strong CLD oop handle roots are demoted to non-roots concurrently
1f1f604: 8302670: use-after-free related to PhaseIterGVN interaction with Unique_Node_List and Node_Stack
d35a550: 8309077: Problemlist compiler/jvmci/TestUncaughtErrorInCompileMethod.java
... and 316 more: https://git.openjdk.org/jdk/compare/d9052b946682d1c0f2629455d73fe4e6b95b29db...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

gerard-ziemski

Looks fine to me.

I have one question though: why wouldn't it be enough to move vm_created = 1 from where we had it before, down to where we now have vm_created = COMPLETE ?

I'm not 100% sure why we need the intermediate step IN_PROGRESS

gerard-ziemski · 2023-05-26T18:01:51Z

test/hotspot/jtreg/runtime/jni/getCreatedJavaVMs/TestGetCreatedJavaVMs.java

+/*
+ * @test
+ * @library /test/lib
+ * @requires os.family == "Linux"


Why is it Linux only?

Does ProcessTools.createNativeTestProcessBuilder("GetCreatedJavaVMs"); only work on Linux?

Filed https://bugs.openjdk.org/browse/CODETOOLS-7903477 to have it implemented on macOS.

It was Linux only primarily because I copied another test. It should work on any platform that uses pthreads - so not Windows. I can expand the makefile logic to build the test code on all platforms.

dholmes-ora · 2023-05-29T02:15:05Z

I have one question though: why wouldn't it be enough to move vm_created = 1 from where we had it before, down to where we now have vm_created = COMPLETE ?

Because we need to prevent two threads from concurrently loading and initializing a VM in the same process.

dholmes-ora · 2023-05-29T02:21:46Z

Thanks for the review @jdksjolen

dholmes-ora · 2023-05-29T03:08:43Z

Thanks for looking at this @gerard-ziemski . The test now runs on all non-Windows platforms.

gerard-ziemski · 2023-05-30T14:39:21Z

I have one question though: why wouldn't it be enough to move vm_created = 1 from where we had it before, down to where we now have vm_created = COMPLETE ?

Because we need to prevent two threads from concurrently loading and initializing a VM in the same process.

Wouldn't it be less complex to use a lock or some "init once" mechanism (pthread_once) instead of a 3 stage atomic field? Do we really need to know whether the process is in the middle of initialization for any reason other than whether it's actually done?

Just to clarify - I'm OK with the fix you have proposed, I was just curious if you considered any other alternatives.

dholmes-ora · 2023-05-30T22:41:28Z

Thanks for the review @gerard-ziemski

Just to clarify - I'm OK with the fix you have proposed, I was just curious if you considered any other alternatives.

Atomics are the simplest cross-platform solution available at this stage of VM init (which is "nothing is initialized yet") - even Atomic use is limited to Atomic::xchg as we can't require any stubs. We have no VM locks available and anything external to the VM (like pthread_once) would require as OS-specific solution in shared code.

dholmes-ora · 2023-05-30T22:45:20Z

I'm aware from #14209 that this new test will fail on AIX but as it will need the same fix as being discussed there I'd rather patch it once we know we have a cross-platform solution. I will file a follow up bug for that (unless we can fix it in #14209.)

Thanks for the reviews.

/integrate

openjdk · 2023-05-30T22:46:13Z

Going to push as commit 1e6770f.
Since your change was applied there have been 335 commits pushed to the master branch:

cb40db0: 8309134: Augment test/langtools/tools/javac/versions/Versions.java for JDK 21 language changes
de7fd1c: 8307944: ClassFileDumper should only load java.nio.file.Path if enabled
7891de3: 8297885: misc sun/security/pkcs11 tests timed out
323d6ce: 8308960: Decouple internal Version and OperatingSystem classes
1b8e6bf: 8308987: Update java.lang.Class to use javadoc snippets
04b0e78: 8307648: java/net/httpclient/ExpectContinueTest.java timed out
6b90b05: 8297878: KEM: Implementation
21af8ba: 8290499: new File(parent, "/") breaks normalization – creates File with slash at the end
804f198: 8308992: New test TestHFA fails with zero
fb0b1f0: 8051725: Improve expansion of Conv2B nodes in the middle-end
... and 325 more: https://git.openjdk.org/jdk/compare/d9052b946682d1c0f2629455d73fe4e6b95b29db...master

Your commit was automatically rebased without conflicts.

openjdk · 2023-05-30T22:46:26Z

@dholmes-ora Pushed as commit 1e6770f.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

gerard-ziemski · 2023-05-31T15:27:51Z

Thanks for the review @gerard-ziemski

Just to clarify - I'm OK with the fix you have proposed, I was just curious if you considered any other alternatives.

Atomics are the simplest cross-platform solution available at this stage of VM init (which is "nothing is initialized yet") - even >Atomic use is limited to Atomic::xchg as we can't require any stubs. We have no VM locks available and anything external to the >VM (like pthread_once) would require as OS-specific solution in shared code.

I guess it's a compromise either way. You have chosen to use an OS independent mechanism, at the cost of exposing the implementation to the outside world, by introducing a new stage (needs CSR). I think, I would have prefer (with what I know at this time) with keeping the implementation details hidden from outside (no need for CSR), at the cost of using platform dependent locks here, or some sort of initialize once mechanism.

dholmes-ora · 2023-05-31T23:01:54Z

You have chosen to use an OS independent mechanism, at the cost of exposing the implementation to the outside world, by introducing a new stage (needs CSR).

That is not an accurate characterisation of this change. The CSR request is needed because of the change in behaviour it introduces (by only returning a fully initialized VM), and has nothing at all to do with the implementation. We still need to be able to distinguish when VM creation has started (to prevent concurrent attempt) and when it has completed (so GetCreatedJavaVMs can return a valid VM) - the mechanism by which that is achieved is immaterial.

gerard-ziemski · 2023-06-01T15:42:22Z

You have chosen to use an OS independent mechanism, at the cost of exposing the implementation to the outside world, >by introducing a new stage (needs CSR).

That is not an accurate characterisation of this change. The CSR request is needed because of the change in behaviour it introduces (by only returning a fully initialized VM), and has nothing at all to do with the implementation. We still need to be able to distinguish when VM creation has started (to prevent concurrent attempt) and when it has completed (so GetCreatedJavaVMs can return a valid VM) - the mechanism by which that is achieved is immaterial.

Sorry, I should have read https://bugs.openjdk.org/browse/JDK-8308816 instead of assuming the impact of this change on the outside.

dholmes-ora added 2 commits May 24, 2023 22:07

8308341: JNI_GetCreatedJavaVMs returns a partially initialized JVM

92cacf0

Added regression test

8f6aa1f

openjdk bot added csr Pull request needs approved CSR before integration rfr Pull request is ready for review labels May 25, 2023

openjdk bot added build build-dev@openjdk.org hotspot hotspot-dev@openjdk.org labels May 25, 2023

jdksjolen reviewed May 25, 2023

View reviewed changes

openjdk bot removed the csr Pull request needs approved CSR before integration label May 26, 2023

jdksjolen approved these changes May 26, 2023

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label May 26, 2023

gerard-ziemski reviewed May 26, 2023

View reviewed changes

Enable the test on all Posix platforms

2bf95d8

dholmes-ora mentioned this pull request May 30, 2023

JDK-8303549: [AIX] TestNativeStack.java is failing with exit value 1 #14209

Closed

3 tasks

gerard-ziemski approved these changes May 30, 2023

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label May 30, 2023

openjdk bot closed this May 30, 2023

openjdk bot removed the ready Pull request is ready to be integrated label May 30, 2023

openjdk bot removed the rfr Pull request is ready for review label May 30, 2023

dholmes-ora deleted the 8308341-created-javavm branch May 30, 2023 22:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8308341: JNI_GetCreatedJavaVMs returns a partially initialized JVM #14139

8308341: JNI_GetCreatedJavaVMs returns a partially initialized JVM #14139

dholmes-ora commented May 25, 2023 •

edited by openjdk bot

Loading

bridgekeeper bot commented May 25, 2023

openjdk bot commented May 25, 2023

mlbridge bot commented May 25, 2023 •

edited

Loading

jdksjolen left a comment

jdksjolen May 25, 2023

dholmes-ora May 26, 2023

jdksjolen May 26, 2023

jdksjolen May 25, 2023

dholmes-ora May 26, 2023

jdksjolen May 26, 2023

dholmes-ora commented May 26, 2023

jdksjolen left a comment

jdksjolen May 26, 2023

jdksjolen May 26, 2023

openjdk bot commented May 26, 2023 •

edited

Loading

gerard-ziemski left a comment

gerard-ziemski May 26, 2023 •

edited

Loading

dholmes-ora May 29, 2023

dholmes-ora commented May 29, 2023

dholmes-ora commented May 29, 2023

dholmes-ora commented May 29, 2023

gerard-ziemski commented May 30, 2023

dholmes-ora commented May 30, 2023

dholmes-ora commented May 30, 2023

openjdk bot commented May 30, 2023

openjdk bot commented May 30, 2023

gerard-ziemski commented May 31, 2023

dholmes-ora commented May 31, 2023

gerard-ziemski commented Jun 1, 2023

8308341: JNI_GetCreatedJavaVMs returns a partially initialized JVM #14139

8308341: JNI_GetCreatedJavaVMs returns a partially initialized JVM #14139

Conversation

dholmes-ora commented May 25, 2023 • edited by openjdk bot Loading

Progress

Issues

Reviewers

Reviewing

Webrev

bridgekeeper bot commented May 25, 2023

openjdk bot commented May 25, 2023

mlbridge bot commented May 25, 2023 • edited Loading

Webrevs

jdksjolen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dholmes-ora commented May 26, 2023

jdksjolen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openjdk bot commented May 26, 2023 • edited Loading

gerard-ziemski left a comment

Choose a reason for hiding this comment

gerard-ziemski May 26, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dholmes-ora commented May 29, 2023

dholmes-ora commented May 29, 2023

dholmes-ora commented May 29, 2023

gerard-ziemski commented May 30, 2023

dholmes-ora commented May 30, 2023

dholmes-ora commented May 30, 2023

openjdk bot commented May 30, 2023

openjdk bot commented May 30, 2023

gerard-ziemski commented May 31, 2023

dholmes-ora commented May 31, 2023

gerard-ziemski commented Jun 1, 2023

dholmes-ora commented May 25, 2023 •

edited by openjdk bot

Loading

mlbridge bot commented May 25, 2023 •

edited

Loading

openjdk bot commented May 26, 2023 •

edited

Loading

gerard-ziemski May 26, 2023 •

edited

Loading