Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8311557: [JVMCI] deadlock with JVMTI thread suspension #14799

Closed
wants to merge 5 commits into from

Conversation

tkrodriguez
Copy link
Contributor

@tkrodriguez tkrodriguez commented Jul 7, 2023

Java based JVMCI compiler threads are more like normal Java threads so they aren't hidden_from_external_view like the native compilers. This can leak to deadlocks if you use JVMTI to suspend all threads since this will block the compiler queue and can block execution if background compilation is disabled. It's reasonable to treat libgraal threads like native threads in this regard. Making jargraal threads hidden too would interfere with using profiling and debugging tool on them so I've left that alone but it might be worth changing the JVMTI suspend and resume functions to explicitly skip compiler threads as well.


Progress

  • Change must be properly reviewed (1 review required, with at least 1 Reviewer)
  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue

Issue

  • JDK-8311557: [JVMCI] deadlock with JVMTI thread suspension (Bug - P3)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/14799/head:pull/14799
$ git checkout pull/14799

Update a local copy of the PR:
$ git checkout pull/14799
$ git pull https://git.openjdk.org/jdk.git pull/14799/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 14799

View PR using the GUI difftool:
$ git pr show -t 14799

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/14799.diff

Webrev

Link to Webrev Comment

@bridgekeeper
Copy link

bridgekeeper bot commented Jul 7, 2023

👋 Welcome back never! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Jul 7, 2023
@openjdk
Copy link

openjdk bot commented Jul 7, 2023

@tkrodriguez The following label will be automatically applied to this pull request:

  • hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot-compiler hotspot-compiler-dev@openjdk.org label Jul 7, 2023
@mlbridge
Copy link

mlbridge bot commented Jul 7, 2023

Webrevs

Copy link
Member

@TobiHartmann TobiHartmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks reasonable to me.

Making jargraal threads hidden too would interfere with using profiling and debugging tool on them so I've left that alone but it might be worth changing the JVMTI suspend and resume functions to explicitly skip compiler threads as well.

Please file a follow-up issue for that.

@openjdk
Copy link

openjdk bot commented Jul 19, 2023

@tkrodriguez This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8311557: [JVMCI] deadlock with JVMTI thread suspension

Reviewed-by: thartmann, dnsimon

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 20 new commits pushed to the master branch:

  • 79be8d9: 8312259: StatusResponseManager unused code clean up
  • 1875b28: 8314061: [JVMCI] DeoptimizeALot stress logic breaks deferred barriers
  • bd1b942: 8313905: Checked_cast assert in CDS compare_by_loader
  • 9b53251: 8313654: Test WaitNotifySuspendedVThreadTest.java timed out
  • e7c83ea: 8312194: test/hotspot/jtreg/applications/ctw/modules/jdk_crypto_ec.java cannot handle empty modules
  • 23fe2ec: 8313616: support loading library members on AIX in os::dll_load
  • f47767f: 8313882: Fix -Wconversion warnings in runtime code
  • 0cb9ab0: 8313239: InetAddress.getCanonicalHostName may return ip address if reverse lookup fails
  • 028b3ae: 8313874: JNI NewWeakGlobalRef throws exception for null arg
  • 83adaf5: 8313421: [JVMCI] avoid locking class loader in CompilerToVM.lookupType
  • ... and 10 more: https://git.openjdk.org/jdk/compare/0eb0997ae4f81314b764241e69dae5c698dbb6c6...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Jul 19, 2023
@tkrodriguez
Copy link
Contributor Author

Thanks for the review. Part of the problem is that it's not clear what the relationship between things like can_call_java and hidden_from_external_view is and whether making some of these methods safer for JVMCI threads will confuse users of JVMTI. There will likely be some follow up issues related to this as we navigate these distinctions.

@tkrodriguez
Copy link
Contributor Author

I did some local testing and unsurprisingly this changes make it impossible to attach a debugger to these threads. In rare cases during development it might be necessary to this since there are still some helper Java calls with native JVMCI. So I've added a new flag LibJVMCICompilerThreadHidden to control this behaviour. I also refactored the code a bit because I figured I was missing some JVMCI_ONLY macros in the shared code. Could I get a rereview?

@@ -151,6 +151,8 @@ class AbstractCompiler : public CHeapObj<mtCompiler> {
bool is_jvmci() const { return _type == compiler_jvmci; }
CompilerType type() const { return _type; }

virtual bool is_hidden_from_external_view() const { return false; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if this had a comment explaining what "hidden from external view" implies. But I see that Thread::is_hidden_from_external_view has no documentation either so I guess there's no much that can be explained here if the broader concept is somewhat undefined.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment in CompilerThread.

@@ -103,6 +103,9 @@ class JVMCICompiler : public AbstractCompiler {
bool is_c1 () { return false; }
bool is_c2 () { return false; }

virtual bool is_hidden_from_external_view() const { return UseJVMCINativeLibrary && LibJVMCICompilerThreadHidden; }

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove extra blank line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

@@ -157,6 +157,11 @@ class fileStream;
"error data to this file" \
"[default: ./" LIBJVMCI_ERR_FILE "] (%p replaced with pid)") \
\
product(bool, LibJVMCICompilerThreadHidden, true, DIAGNOSTIC, \
"If true then native JVMCI compiler threads are hidden from " \
"JVMTI and FlightRecorder. This must be set to false if you" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you" -> you "
threads" -> threads."

As far as I can help, this help text is never printed (is it?) but it may as well be properly formatted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. It's not in the product but it does function a documentation for the options.

Copy link
Member

@TobiHartmann TobiHartmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still looks good to me.

@tkrodriguez
Copy link
Contributor Author

Thanks!

/integrate

@openjdk
Copy link

openjdk bot commented Aug 15, 2023

Going to push as commit 004651d.
Since your change was applied there have been 62 commits pushed to the master branch:

  • 9ded868: 8314114: Fix -Wconversion warnings in os code, primarily linux
  • a02d65e: 8310308: IR Framework: check for type and size of vector nodes
  • dff99f7: 8313782: Add user-facing warning if THPs are enabled but cannot be used
  • f4e72c5: 8313949: Missing word in GPLv2 license text in StackMapTableAttribute.java
  • 6338927: 8314197: AttachListener::pd_find_operation always returning nullptr
  • b7dee21: 8314244: Incorrect file headers in new tests from JDK-8312597
  • 37c6b23: 8308340: C2: Idealize Fma nodes
  • 583cb75: 8313406: nep_invoker_blob can be simplified more
  • 0074b48: 8312597: Convert TraceTypeProfile to UL
  • 1f1c5c6: 8314241: Add test/jdk/sun/security/pkcs/pkcs7/SignerOrder.java to ProblemList
  • ... and 52 more: https://git.openjdk.org/jdk/compare/0eb0997ae4f81314b764241e69dae5c698dbb6c6...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Aug 15, 2023
@openjdk openjdk bot closed this Aug 15, 2023
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Aug 15, 2023
@openjdk
Copy link

openjdk bot commented Aug 15, 2023

@tkrodriguez Pushed as commit 004651d.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@@ -157,6 +157,11 @@ class fileStream;
"error data to this file" \
"[default: ./" LIBJVMCI_ERR_FILE "] (%p replaced with pid)") \
\
product(bool, LibJVMCICompilerThreadHidden, true, EXPERIMENTAL, \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds more like a diagnostic flag than experimental.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It kind of is but because of the way the way JVMCIGlobals::enable_jvmci_product_mode works it seems easier to just make it experimental that becomes product like all the other special JVMCI flags.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-compiler hotspot-compiler-dev@openjdk.org integrated Pull request has been integrated
4 participants