8349094: GenShen: Race between control and regulator threads may violate assertions #23475

earthling-amzn · 2025-02-05T22:30:35Z

There are several changes to the operation of Shenandoah's control threads here.

The reason for cancellation is now recorded in ShenandoahHeap::_cancelled_gc as a GCCause, instead of various member variables in the control thread.
- The cancellation handling is driven entirely by the cancellation cause
- The graceful shutdown, alloc failure, humongous alloc failure and preemption requested flags are all removed
- The shutdown sequence is simpler
The generational control thread uses a lock to coordinate updates to the requested cause and generation
- APIs have been simplified to avoid converting between the generation type and the actual generation instance
- The old heuristic, rather than the control thread itself, is now responsible for resuming old generation cycles
- The control thread doesn't loop on its own (unless the pacer is enabled).

Testing

jtreg hotspot_gc_shenandoah
dacapo, extremem, diluvian, specjbb2015, specjvm2018, heapothesys

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8349094: GenShen: Race between control and regulator threads may violate assertions (Task - P4)

Reviewers

Kelvin Nilsen (@kdnilsen - Committer) Review applies to 915ffbda
Y. Srinivas Ramakrishna (@ysramakrishna - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/23475/head:pull/23475
$ git checkout pull/23475

Update a local copy of the PR:
$ git checkout pull/23475
$ git pull https://git.openjdk.org/jdk.git pull/23475/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 23475

View PR using the GUI difftool:
$ git pr show -t 23475

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/23475.diff

Using Webrev

Link to Webrev Comment

…cycles

…threads

bridgekeeper · 2025-02-05T22:31:17Z

👋 Welcome back wkemper! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2025-02-05T22:31:50Z

@earthling-amzn This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8349094: GenShen: Race between control and regulator threads may violate assertions

Reviewed-by: ysr, kdnilsen

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 33 new commits pushed to the master branch:

99fb350: 8350654: (fs) Files.createTempDirectory should say something about the default file permissions when no file attributes specified
768b024: 8350682: [JMH] vector.IndexInRangeBenchmark failed with IndexOutOfBoundsException for size=1024
c4b516d: 8348322: AOT cache creation crashes with "All cached hidden classes must be aot-linkable" when AOTInvokeDynamicLinking is disabled
e470f47: 8350095: RISC-V: Refactor string_compare
f53de92: 8350931: RISC-V: remove unnecessary src register for fp_sqrt_d/f
e1fc14f: 8350940: RISC-V: remove unnecessary assert_different_registers in minmax_fp
79880e5: 8351033: RISC-V: TestFloat16ScalarOperations asserts with offset (4210) is too large to be patched in one beq/bge/bgeu/blt/bltu/bne instruction!
bb70896: 8350903: Remove explicit libjvm.so dependency for libVThreadEventTest
7c187b5: 8338737: Shenandoah: Reset marking bitmaps after the cycle
3602c8c: 8350955: Fix repetitions of the word "the" in runtime component comments
... and 23 more: https://git.openjdk.org/jdk/compare/e98df71d9c5120fbb73a4c2f49863775fe5db781...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

openjdk · 2025-02-05T22:32:12Z

@earthling-amzn The following labels will be automatically applied to this pull request:

hotspot-gc
shenandoah

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

mlbridge · 2025-02-05T22:35:21Z

Webrevs

… clear

…threads

earthling-amzn · 2025-02-14T18:15:19Z

/integrate delegate

openjdk · 2025-02-14T18:15:34Z

@earthling-amzn Integration of this pull request has been delegated and may be completed by any project committer using the /integrate pull request command.

kdnilsen

Thank you. This looks very clean to me.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp

ysramakrishna

Flushing the comments at EOD; will complete review later.

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp

src/hotspot/share/gc/shenandoah/shenandoahController.hpp

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.hpp

src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp

ysramakrishna · 2025-02-15T01:46:45Z

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp

-  ShenandoahGenerationType generation = GLOBAL;
-
-  uint age_period = 0;
+  const int64_t wait_ms = ShenandoahPacing ? ShenandoahControlIntervalMin : 0;


So we are supporting ShenandoahPacing with GenShenm(at least till we pull it in the future), but don't want to enable it by default, correct?

Added tag jdk-25+11 for changeset 0131c1b

ysramakrishna

A few random comments, mostly on documentation, and a few assertion suggestions.

Rest looks good. Thanks for tightening up the code via your changes in this PR.

I don't think I should need to re-review any changes you make stemming from my (rally quite minor) comments.

Reviewed; 🚢

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.hpp

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp

It may cause the thread to exit before it is requested to stop

…threads

ysramakrishna

🚢

earthling-amzn · 2025-03-04T00:41:00Z

/integrate

openjdk · 2025-03-04T00:41:40Z

Going to push as commit 3a8a432.
Since your change was applied there have been 33 commits pushed to the master branch:

99fb350: 8350654: (fs) Files.createTempDirectory should say something about the default file permissions when no file attributes specified
768b024: 8350682: [JMH] vector.IndexInRangeBenchmark failed with IndexOutOfBoundsException for size=1024
c4b516d: 8348322: AOT cache creation crashes with "All cached hidden classes must be aot-linkable" when AOTInvokeDynamicLinking is disabled
e470f47: 8350095: RISC-V: Refactor string_compare
f53de92: 8350931: RISC-V: remove unnecessary src register for fp_sqrt_d/f
e1fc14f: 8350940: RISC-V: remove unnecessary assert_different_registers in minmax_fp
79880e5: 8351033: RISC-V: TestFloat16ScalarOperations asserts with offset (4210) is too large to be patched in one beq/bge/bgeu/blt/bltu/bne instruction!
bb70896: 8350903: Remove explicit libjvm.so dependency for libVThreadEventTest
7c187b5: 8338737: Shenandoah: Reset marking bitmaps after the cycle
3602c8c: 8350955: Fix repetitions of the word "the" in runtime component comments
... and 23 more: https://git.openjdk.org/jdk/compare/e98df71d9c5120fbb73a4c2f49863775fe5db781...master

Your commit was automatically rebased without conflicts.

openjdk · 2025-03-04T00:41:52Z

@earthling-amzn Pushed as commit 3a8a432.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

earthling-amzn added 14 commits January 30, 2025 11:44

Make old heuristic (not control thread) responsible for resuming old …

650a6ac

…cycles

WIP: Simplifying state, removing races

2687dde

WIP: Introduce immutable request object for use without the lock

d7a065b

WIP: Carry over degenerated request

19bfa7d

WIP: Passes tier1, mostly passes tier2

53ef004

WIP: Simplify degenerated cycle handling

ccde1ca

WIP: Don't need request.mode anymore

f2ed553

WIP: Simplify shutdown protocol

5940821

Better names, better comments

3378b66

Restore reporting allocations to pacer

76b6d7e

Fix interrupted old cycle handling

481d3b1

Revert unnecessary changes

5747faa

Simplify shControlThread

dea41e4

Merge remote-tracking branch 'jdk/master' into fix-control-regulator-…

f97f257

…threads

openjdk bot added the rfr Pull request is ready for review label Feb 5, 2025

openjdk bot added hotspot-gc hotspot-gc-dev@openjdk.org shenandoah shenandoah-dev@openjdk.org labels Feb 5, 2025

earthling-amzn added 9 commits February 6, 2025 09:17

Remove invalid assert, alloc waiters wait until allocation failure is…

a7a6eea

… clear

Do not stomp on pending requests when running a degenerated cycle

b0538ee

Use logging tag 'thread' to help control debug volume

d199e97

Resuming an old cycle should not preempt a young cycle

ae20748

Make shutdown more robust, make better use of request lock

700fa4d

Simplify locking protocol

a6513bc

Fix includes

d16f6fd

Fix shutdown livelock error

8038691

Add event for control thread state changes

f11584d

earthling-amzn added 5 commits February 11, 2025 14:58

Make shutdown safer for threads requesting (or expecting) gc

047d6ff

Improve message for assertion

779492c

Merge remote-tracking branch 'jdk/master' into fix-control-regulator-…

51d0920

…threads

Old gen bootstrap cycle must make it to init mark

82f9609

Merge remote-tracking branch 'jdk/master' into fix-control-regulator-…

915ffbd

…threads

openjdk bot added the delegated May be integrated by another Committer label Feb 14, 2025

kdnilsen approved these changes Feb 14, 2025

View reviewed changes

src/hotspot/share/gc/shenandoah/shenandoahGenerationalControlThread.cpp Outdated Show resolved Hide resolved

ysramakrishna reviewed Feb 15, 2025

View reviewed changes

earthling-amzn added 2 commits February 24, 2025 12:49

Address review feedback (better comments, better names)

1d887fc

Merge tag 'jdk-25+11' into fix-control-regulator-threads

d7858de

Added tag jdk-25+11 for changeset 0131c1b

ysramakrishna approved these changes Feb 26, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Feb 26, 2025

Improve names and comments

fb7819d

openjdk bot removed the ready Pull request is ready to be integrated label Feb 26, 2025

earthling-amzn added 4 commits February 26, 2025 17:10

Remove duplicated field pointer for old generation

b91ce9c

Add assertions about old gen state when resuming old cycles

d2e90dd

Don't check for shutdown in control thread loop condition

150cb79

It may cause the thread to exit before it is requested to stop

Merge remote-tracking branch 'jdk/master' into fix-control-regulator-…

37e445d

…threads

ysramakrishna approved these changes Mar 3, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Mar 3, 2025

openjdk bot added the integrated Pull request has been integrated label Mar 4, 2025

openjdk bot closed this Mar 4, 2025

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review delegated May be integrated by another Committer labels Mar 4, 2025

8349094: GenShen: Race between control and regulator threads may violate assertions #23475

8349094: GenShen: Race between control and regulator threads may violate assertions #23475

Uh oh!

Conversation

earthling-amzn commented Feb 5, 2025 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing

Progress

Issue

Reviewers

Reviewing

Uh oh!

bridgekeeper bot commented Feb 5, 2025

Uh oh!

openjdk bot commented Feb 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openjdk bot commented Feb 5, 2025

Uh oh!

mlbridge bot commented Feb 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

earthling-amzn commented Feb 14, 2025

Uh oh!

openjdk bot commented Feb 14, 2025

Uh oh!

kdnilsen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ysramakrishna left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ysramakrishna Feb 15, 2025

Choose a reason for hiding this comment

Uh oh!

ysramakrishna left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ysramakrishna left a comment

Choose a reason for hiding this comment

Uh oh!

earthling-amzn commented Mar 4, 2025

Uh oh!

openjdk bot commented Mar 4, 2025

Uh oh!

openjdk bot commented Mar 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

earthling-amzn commented Feb 5, 2025 •

edited by openjdk bot

Loading

openjdk bot commented Feb 5, 2025 •

edited

Loading

mlbridge bot commented Feb 5, 2025 •

edited

Loading