8318986: Improve GenericWaitBarrier performance #16404

shipilev · 2023-10-27T15:40:11Z

See the symptoms, reproducer and analysis in the bug.

Current code waits on disarm(), which effectively stalls leaving the safepoint if some threads lag behind. Having more runnable threads than CPUs nearly guarantees that we would wait for quite some time, but it also reproduces well if you have enough threads near the CPU count.

This PR implements a more efficient GenericWaitBarrier to recover the performance. Most of the implementation discussion is in the code comments. The key observation that drives this work is that we want to reuse Semaphore and related counters without being stuck waiting for threads to leave. (AFAICS, futex-based LinuxWaitBarrier does roughly the same, but handles this reuse on futex side, by assigning the "address" per futex.)

This issue affects everything except Linux. I initially found this on my M1 Mac, but pretty sure it is easy to reproduce on Windows as well. The safepoints from the reproducer in the bug improved dramatically on a Mac, see the graph below. The new version gives orders of magnitude better safepoint times. This also translates to much more active GC and attainable allocating rate, because GC throughput is not blocked by overly long safepoints.

Additional testing:

MacOS AArch64 server fastdebug, tier1
Linux x86_64 server fastdebug, tier1 tier2 tier3 (generic wait barrier enabled explicitly)
Linux AArch64 server fastdebug, tier1 tier2 tier3 (generic wait barrier enabled explicitly)
MacOS AArch64 server fastdebug, tier2 tier3
Linux x86_64 server fastdebug, tier4 (generic wait barrier enabled explicitly)
Linux AArch64 server fastdebug, tier4 (generic wait barrier enabled explicitly)

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8318986: Improve GenericWaitBarrier performance (Enhancement - P3)

Reviewers

Robbin Ehn (@robehn - Reviewer)
Ivan Walulya (@walulyai - Reviewer) ⚠️ Review applies to 191c0dbb
Patricio Chilano Mateo (@pchilano - Reviewer) ⚠️ Review applies to 191c0dbb

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/16404/head:pull/16404
$ git checkout pull/16404

Update a local copy of the PR:
$ git checkout pull/16404
$ git pull https://git.openjdk.org/jdk.git pull/16404/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 16404

View PR using the GUI difftool:
$ git pr show -t 16404

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/16404.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2023-10-27T15:41:38Z

👋 Welcome back shade! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2023-10-27T15:43:24Z

@shipilev The following label will be automatically applied to this pull request:

hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2023-11-01T23:11:07Z

Webrevs

shipilev · 2023-11-02T08:19:35Z

@robehn, you might be interested in this :)

shipilev · 2023-11-02T12:33:43Z

Stress tests show there is still a race condition, which can be triggered by putting a small sleep on this path:

diff --git a/src/hotspot/share/utilities/waitBarrier_generic.cpp b/src/hotspot/share/utilities/waitBarrier_generic.cpp
index f886e7b4cf5..13ba12c39a6 100644
--- a/src/hotspot/share/utilities/waitBarrier_generic.cpp
+++ b/src/hotspot/share/utilities/waitBarrier_generic.cpp
@@ -174,6 +174,7 @@ void GenericWaitBarrier::wait(int barrier_tag) {
   OrderAccess::fence();
 
   if (Atomic::load_acquire(&_barrier_tag) == barrier_tag) {
+    if (UseNewCode) os::naked_short_nanosleep(100);
     Atomic::add(&cell._unsignaled_waits, 1);
 
     // Wait for notification.

It would then fail/hang this tier4 test:

$ CONF=linux-x86_64-server-fastdebug make images test TEST=vmTestbase/nsk/monitoring/ThreadInfo/isSuspended/issuspended002.java TEST_VM_OPTS="-XX:+UnlockDiagnosticVMOptions -XX:+UseNewCode"

Putting this issue to draft until I figure out the better way to do this.

robehn · 2023-11-02T12:50:56Z

@robehn, you might be interested in this :)

Yepp, thanks for pinging me in! I'll have a look as when you are ready!

Regarding the bug, accounting for context switches in every 'sub-state' usually gets you at least one time.
That is what the: Atomic::add(&_barrier_threads, 1);, did, "no marine left behind" :)

And while on this topic:
Note that there is an optimization that can be done on Linux also.
Waking all threads via futex the VM thread gets high runtime, so two safepoint very close to each other is slow.
Meaning if there are a new safepoint op depending, VM thread often will context switched out just after waking all threads.
Secondly the futex wake, involving visiting all run queues, can be parallelized similar to this by just waking a few thread and let them wake the rest. Intel actually did a draft of that just before meltdown/spectre, so it got lost.
I think that draft did wake like 6 or 8 at the time, one large system ~128 cores you could get the time to full utilization down by almost 50% (but you lose some latency for the JavaThreads doing the second round of wakening).

I mention this since you have setup measurements and graphs, so maybe you like to continue on this code :) (no jira issue for this)

shipilev · 2023-11-02T16:02:25Z

Regarding the bug, accounting for context switches in every 'sub-state' usually gets you at least one time. That is what the: Atomic::add(&_barrier_threads, 1);, did, "no marine left behind" :)

Yes. I think most of the race condition mess comes from juggling several counters at once, plus depending on barrier_tag. The new version fuses the important counters together and melds in the arm/disarm status into the counter. Which allows to manage things more easily but CAS-ing one counter only. I think that resolves race conditions I saw in previous implementation. It yields same performance. Now running it through functional testing.

shipilev · 2023-11-02T20:57:31Z

I think this is good for review. The reproducer that used to hang/fail on assert is now passing. tier1 tier2 tier3 are all passing. I am running more tests overnight.

shipilev · 2023-11-06T10:50:58Z

I think this is good for review. The reproducer that used to hang/fail on assert is now passing. tier1 tier2 tier3 are all passing. I am running more tests overnight.

Testing seems all good. I'll leave the Linux -> Generic switch in this PR, until the very last moment before integration to keep testing more easily.

dcubed-ojdk · 2023-11-06T21:20:03Z

@shipilev - I'm glad that:

vmTestbase/nsk/monitoring/ThreadInfo/isSuspended/issuspended002.java

has proven to useful. I had been thinking about removing it from my weekly stress
kit runs since it has been a long time since I've seen a failure flushed out by that
test running the stress config. I think I'll keep it around for longer...

robehn

Thanks for addressing this!
I had some comment.

src/hotspot/share/utilities/waitBarrier_generic.cpp

… methods

robehn

Thank you! I like the new protocol with tag == 0 as disarmed.
Looks good!

(a bit more tired than this morning, so I'll have new look tomorrow, just in case a missed something. silence == all good)

src/hotspot/share/utilities/waitBarrier_generic.hpp

cl4es · 2023-11-15T00:31:04Z

Agreed. Would you like to formally approve this PR?

Even with Robbin's prior approval I feel it would be presumptuous of me to approve a PR in a piece of code I'm completely unfamiliar with.

robehn · 2023-11-15T08:44:17Z

@pchilano can you have look ?

pchilano · 2023-11-15T22:54:14Z

@pchilano can you have look ?

I will. I might not finish the review until next week though.

walulyai

LGTM!

src/hotspot/share/utilities/waitBarrier_generic.cpp

shipilev · 2023-11-20T09:45:08Z

All right, thanks all. I think we are waiting for @pchilano's review, and then we can integrate.

pchilano

Looks good to me.

src/hotspot/share/utilities/waitBarrier_generic.cpp

shipilev · 2023-11-21T10:16:01Z

The performance improvements still hold. I would wait for some light testing to complete, and then I will integrate.

robehn

Thanks!

Did you want to tackle the futex version also?
I'll create a JBS anyhow to track it.

shipilev · 2023-11-21T10:53:06Z

Did you want to tackle the futex version also? I'll create a JBS anyhow to track it.

I have no plans to tackle futex version. Trying to finish the tasks already started this year :) But I would be surprised if there are things that can be improved on top what futex already does for us. Probably avalanche wakeups would be a thing to try?

robehn · 2023-11-21T11:05:22Z

Did you want to tackle the futex version also? I'll create a JBS anyhow to track it.

I have no plans to tackle futex version. Trying to finish the tasks already started this year :) But I would be surprised if there are things that can be improved on top what futex already does for us. Probably avalanche wakeups would be a thing to try?

Ok! Yes!

shipilev · 2023-11-21T16:39:21Z

@pchilano @dholmes-ora @cl4es, may I ask you to check if anyone at Oracle did the testing runs for this PR, and schedule a run if not? I am sure we are good with current testing, but additional safety would be nice to avoid surprises this close to JDK 22 RDP1.

pchilano · 2023-11-21T17:05:46Z

@pchilano @dholmes-ora @cl4es, may I ask you to check if anyone at Oracle did the testing runs for this PR, and schedule a run if not? I am sure we are good with current testing, but additional safety would be nice to avoid surprises this close to JDK 22 RDP1.

I'll schedule a round of testing for Tiers1-7 with the latest version.

shipilev · 2023-11-22T14:49:15Z

I'll schedule a round of testing for Tiers1-7 with the latest version.

Thanks! Any failures so far?

pchilano · 2023-11-22T15:08:45Z

I run Tiers[1-7] and there is one failure in tier5 in test vmTestbase/nsk/monitoring/stress/thread/strace016/TestDescription.java on windows-x64-debug. The output is:

#>  
#>  WARNING: switching log to verbose mode,
#>      because error is complained
#>  
ThreadMonitor> Test mode:	DIRECTLY access to MBean
ThreadController> number of created threads:	30
ThreadController> depth for all threads:	100
ThreadController> invocation type:	mixed

Starting threads.

ThreadController> locking threads

States of the threads are culminated.
# ERROR: 	Thread BLOCKED_ThreadMM001 wrong thread state: RUNNABLE
The following stacktrace is for failure analysis.
nsk.share.TestFailure:  Thread BLOCKED_ThreadMM001 wrong thread state: RUNNABLE
	at nsk.share.Log.logExceptionForFailureAnalysis(Log.java:431)
	at nsk.share.Log.complain(Log.java:402)
	at nsk.monitoring.stress.thread.strace010.runIt(strace010.java:148)
	at nsk.monitoring.stress.thread.strace010.run(strace010.java:99)
	at nsk.monitoring.stress.thread.strace010.main(strace010.java:95)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
	at java.base/java.lang.Thread.run(Thread.java:1570)

Checked 6 BLOCKED threads
# ERROR: Expected amount: 7 for BLOCKED threads actual: 6
Checked 7 WAITING threads
Checked 8 TIMED_WAITING threads
Checked 9 RUNNABLE threads
# ERROR: Expected amount: 8 for RUNNABLE threads actual: 9


Test FAILED


#>  
#>  SUMMARY: Following errors occured
#>      during test execution:
#>  
# ERROR: 	Thread BLOCKED_ThreadMM001 wrong thread state: RUNNABLE
# ERROR: Expected amount: 7 for BLOCKED threads actual: 6
# ERROR: Expected amount: 8 for RUNNABLE threads actual: 9

I re-run tier5 twice and the test alone 100 times but unfortunately couldn't reproduce the issue. I checked the history of failures and haven't seen this failed before. But it could also be that there is some race already in the test uncovered by this patch.

There are some jobs pending for macos-x64 (there is currently a bottleneck in the pipeline for this platform).

shipilev · 2023-11-22T15:53:45Z

Thanks for testing!

I run Tiers[1-7] and there is one failure in tier5 in test vmTestbase/nsk/monitoring/stress/thread/strace016/TestDescription.java on windows-x64-debug. I re-run tier5 twice and the test alone 100 times but unfortunately couldn't reproduce the issue. I checked the history of failures and haven't seen this failed before. But it could also be that there is some race already in the test uncovered by this patch.

Yes, I think so too. I ran this test hundreds of times without failure. The output implies there is a thread that should be "blocked", but instead it is "runnable". I think the test itself contains the race condition, submitted: https://bugs.openjdk.org/browse/JDK-8320599. I would not treat this failure as integration blocker then.

Do you think we should wait for Mac pipeline to complete?

robehn · 2023-11-22T15:55:56Z

@pchilano you can change the waitBarrier.hpp so Linux also uses the generic one, as @shipilev did when he tested:

#if defined(LINUX)
#include "waitBarrier_linux.hpp"

And just use "typedef GenericWaitBarrier WaitBarrierDefault;"

For better coverage.

pchilano · 2023-11-22T16:02:28Z

Thanks for testing!

I run Tiers[1-7] and there is one failure in tier5 in test vmTestbase/nsk/monitoring/stress/thread/strace016/TestDescription.java on windows-x64-debug. I re-run tier5 twice and the test alone 100 times but unfortunately couldn't reproduce the issue. I checked the history of failures and haven't seen this failed before. But it could also be that there is some race already in the test uncovered by this patch.

Yes, I think so too. I ran this test hundreds of times without failure. The output implies there is a thread that should be "blocked", but instead it is "runnable". I think the test itself contains the race condition, submitted: https://bugs.openjdk.org/browse/JDK-8320599. I would not treat this failure as integration blocker then.

Yes, I think the issue is in ThreadController.java with Blocker.block(). I'll keep investigating to see if I can reproduce it.

Do you think we should wait for Mac pipeline to complete?

I'm not sure when this tasks will finish. I think we should be good with all the testing done so far.

pchilano · 2023-11-22T16:07:52Z

@pchilano you can change the waitBarrier.hpp so Linux also uses the generic one, as @shipilev did when he tested:
#if defined(LINUX)
#include "waitBarrier_linux.hpp"
And just use "typedef GenericWaitBarrier WaitBarrierDefault;"

For better coverage.

I actually realized of this yesterday after the jobs have been running for a while. So I submitted extra runs with that change to test Linux too. I run Tiers[4-7]. Tier7 completed successfully, and Tiers[4-6] is almost done too with no failures. There are again some macos-x64 jobs that are pending.

shipilev · 2023-11-22T17:02:27Z

All right then. I will integrate today, hopefully within an hour. Thank you all!

shipilev · 2023-11-22T17:54:22Z

/integrate

openjdk · 2023-11-22T17:55:19Z

Going to push as commit 30462f9.
Since your change was applied there have been 30 commits pushed to the master branch:

407cdd4: 8320207: doclet incorrectly chooses code font for a See Also link
1629a90: 8320331: G1 Full GC Heap verification relies on metadata not reset before verification
93bdc2a: 8306055: Add a built-in Catalog to JDK XML module
a4bd9e4: 8319440: RISC-V: jdk can't be built with clang due to register keyword
524da14: 8320418: PPC64: invokevfinal_helper duplicates code to handle ResolvedMethodEntry
5d4a54b: 8319449: compiler/print/CompileCommandPrintMemStat.java fails on Graal
35526d0: 8257076: os::scan_pages is empty on all platforms
25cebe8: 8317799: AIX PPC64: FFI symbol lookup doesn't find symbols
c39d001: 8319137: release _object in ObjectMonitor dtor to avoid races
8b47a14: 8320309: AIX: pthreads created by foreign test library don't work as expected
... and 20 more: https://git.openjdk.org/jdk/compare/e055fae104a887c436da9f2924e88029518d5d96...master

Your commit was automatically rebased without conflicts.

openjdk · 2023-11-22T17:55:25Z

@shipilev Pushed as commit 30462f9.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

openjdk bot added the hotspot hotspot-dev@openjdk.org label Oct 27, 2023

shipilev force-pushed the JDK-8318986-generic-wait-barrier branch from b4dff05 to e39ba69 Compare November 1, 2023 14:11

Fix

a390610

shipilev force-pushed the JDK-8318986-generic-wait-barrier branch from a0694a8 to a390610 Compare November 1, 2023 16:07

shipilev marked this pull request as ready for review November 1, 2023 23:07

openjdk bot added the rfr Pull request is ready for review label Nov 1, 2023

Tigthen up memory ordering even more conservatively

ca88eb7

shipilev marked this pull request as draft November 2, 2023 12:33

openjdk bot removed the rfr Pull request is ready for review label Nov 2, 2023

Rework to a single atomic counter per cell

8a8990c

shipilev added 3 commits November 2, 2023 17:19

Tight up the comments

7e56314

More comments work

6220dd5

Touchups

dfafbf3

shipilev marked this pull request as ready for review November 2, 2023 20:56

openjdk bot added the rfr Pull request is ready for review label Nov 2, 2023

robehn suggested changes Nov 7, 2023

View reviewed changes

shipilev added 3 commits November 7, 2023 10:24

Merge branch 'master' into JDK-8318986-generic-wait-barrier

9372f9b

Simple review feedback fixes: tracking wakeup numbers, reflowing some…

284c3c4

… methods

Encode barrier tag into state, resolving another race condition

3cd53c1

robehn approved these changes Nov 7, 2023

View reviewed changes

src/hotspot/share/utilities/waitBarrier_generic.hpp Show resolved Hide resolved

walulyai approved these changes Nov 19, 2023

View reviewed changes

src/hotspot/share/utilities/waitBarrier_generic.cpp Show resolved Hide resolved

pchilano approved these changes Nov 20, 2023

View reviewed changes

src/hotspot/share/utilities/waitBarrier_generic.cpp Outdated Show resolved Hide resolved

shipilev added 2 commits November 21, 2023 10:26

Merge branch 'master' into JDK-8318986-generic-wait-barrier

c3a9f32

Do not SpinYield at disarm loop

32b0a9c

Merge branch 'master' into JDK-8318986-generic-wait-barrier

e56a2bf

robehn approved these changes Nov 21, 2023

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Nov 22, 2023

openjdk bot closed this Nov 22, 2023

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Nov 22, 2023

shipilev deleted the JDK-8318986-generic-wait-barrier branch November 28, 2023 14:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8318986: Improve GenericWaitBarrier performance #16404

8318986: Improve GenericWaitBarrier performance #16404

shipilev commented Oct 27, 2023 •

edited by openjdk bot

Loading

bridgekeeper bot commented Oct 27, 2023

openjdk bot commented Oct 27, 2023

mlbridge bot commented Nov 1, 2023 •

edited

Loading

shipilev commented Nov 2, 2023

shipilev commented Nov 2, 2023

robehn commented Nov 2, 2023 •

edited

Loading

shipilev commented Nov 2, 2023

shipilev commented Nov 2, 2023

shipilev commented Nov 6, 2023

dcubed-ojdk commented Nov 6, 2023

robehn left a comment

robehn left a comment

cl4es commented Nov 15, 2023

robehn commented Nov 15, 2023

pchilano commented Nov 15, 2023

walulyai left a comment

shipilev commented Nov 20, 2023

pchilano left a comment

shipilev commented Nov 21, 2023

robehn left a comment

shipilev commented Nov 21, 2023

robehn commented Nov 21, 2023

shipilev commented Nov 21, 2023 •

edited

Loading

pchilano commented Nov 21, 2023

shipilev commented Nov 22, 2023

pchilano commented Nov 22, 2023

shipilev commented Nov 22, 2023 •

edited

Loading

robehn commented Nov 22, 2023 •

edited

Loading

pchilano commented Nov 22, 2023

pchilano commented Nov 22, 2023

shipilev commented Nov 22, 2023

shipilev commented Nov 22, 2023

openjdk bot commented Nov 22, 2023

openjdk bot commented Nov 22, 2023

8318986: Improve GenericWaitBarrier performance #16404

8318986: Improve GenericWaitBarrier performance #16404

Conversation

shipilev commented Oct 27, 2023 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

Webrev

bridgekeeper bot commented Oct 27, 2023

openjdk bot commented Oct 27, 2023

mlbridge bot commented Nov 1, 2023 • edited Loading

Webrevs

shipilev commented Nov 2, 2023

shipilev commented Nov 2, 2023

robehn commented Nov 2, 2023 • edited Loading

shipilev commented Nov 2, 2023

shipilev commented Nov 2, 2023

shipilev commented Nov 6, 2023

dcubed-ojdk commented Nov 6, 2023

robehn left a comment

Choose a reason for hiding this comment

robehn left a comment

Choose a reason for hiding this comment

cl4es commented Nov 15, 2023

robehn commented Nov 15, 2023

pchilano commented Nov 15, 2023

walulyai left a comment

Choose a reason for hiding this comment

shipilev commented Nov 20, 2023

pchilano left a comment

Choose a reason for hiding this comment

shipilev commented Nov 21, 2023

robehn left a comment

Choose a reason for hiding this comment

shipilev commented Nov 21, 2023

robehn commented Nov 21, 2023

shipilev commented Nov 21, 2023 • edited Loading

pchilano commented Nov 21, 2023

shipilev commented Nov 22, 2023

pchilano commented Nov 22, 2023

shipilev commented Nov 22, 2023 • edited Loading

robehn commented Nov 22, 2023 • edited Loading

pchilano commented Nov 22, 2023

pchilano commented Nov 22, 2023

shipilev commented Nov 22, 2023

shipilev commented Nov 22, 2023

openjdk bot commented Nov 22, 2023

openjdk bot commented Nov 22, 2023

shipilev commented Oct 27, 2023 •

edited by openjdk bot

Loading

mlbridge bot commented Nov 1, 2023 •

edited

Loading

robehn commented Nov 2, 2023 •

edited

Loading

shipilev commented Nov 21, 2023 •

edited

Loading

shipilev commented Nov 22, 2023 •

edited

Loading

robehn commented Nov 22, 2023 •

edited

Loading