8315503: G1: Code root scan causes long GC pauses due to imbalanced iteration

[GoeLin]

I backport this for parity with 21.0.4-oracle.

I had to resolve 3 files:

src/hotspot/share/gc/g1/heapRegion.cpp
Hunk #1 had to be resolved. "8140326: G1: Consider putting regions where evacuation failed into next collection set" is not in 21.
It adds the "keep_tracked" argument.

src/hotspot/share/gc/g1/heapRegionRemSet.cpp
Resolved hunk #2 because of "8140326: G1: Consider putting regions where evacuation failed into next collection set".
Resolved hunk #4 because "8313202: MutexLocker should disallow null Mutexes" is not in 21
(other Mutex class).

src/hotspot/share/gc/g1/heapRegionRemSet.cpp
Resulved hunk #2, dual to hunk #2 in .cpp file.

Resolved hunks are in a commit of their own.

I include the two direct follow ups
8317440: Lock rank checking fails when code root set is modified with the Servicelock held after JDK-8315503
8318720: G1: Memory leak in G1CodeRootSet after JDK-8315503

Both apply clean on top.

I'll backport the third follow-up
8323685: PrintSystemDictionaryAtExit has mutex rank assert
as dependend backport on top of these. It also applies clean,
but I think it is too large to merge it in here. See #477

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• JDK-8318720 needs maintainer approval
• JDK-8317440 needs maintainer approval
• JDK-8315503 needs maintainer approval
• Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issues

• JDK-8315503: G1: Code root scan causes long GC pauses due to imbalanced iteration (Enhancement - P4 - Approved)
• JDK-8317440: Lock rank checking fails when code root set is modified with the Servicelock held after JDK-8315503 (Bug - P2 - Approved)
• JDK-8318720: G1: Memory leak in G1CodeRootSet after JDK-8315503 (Bug - P3 - Approved)

Reviewers

• Severin Gehwolf (@jerboaa - Reviewer)
• Aleksey Shipilev (@shipilev - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk21u-dev.git pull/476/head:pull/476
$ git checkout pull/476

Update a local copy of the PR:
$ git checkout pull/476
$ git pull https://git.openjdk.org/jdk21u-dev.git pull/476/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 476

View PR using the GUI difftool:
$ git pr show -t 476

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk21u-dev/pull/476.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back goetz! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@GoeLin This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8315503: G1: Code root scan causes long GC pauses due to imbalanced iteration
8317440: Lock rank checking fails when code root set is modified with the Servicelock held after JDK-8315503
8318720: G1: Memory leak in G1CodeRootSet after JDK-8315503

Reviewed-by: sgehwolf, shade

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 36 new commits pushed to the master branch:

• 7743b6c: Merge
• 05f18b1: 8329838: [21u] Remove designator DEFAULT_PROMOTED_VERSION_PRE=ea for release 21.0.3
• 778a6c4: 8319851: Improve exception logging
• 517bd93: 8322122: Enhance generation of addresses
• 5378f80: 8318340: Improve RSA key implementations
• 8eba98a: 8315708: Enhance HTTP/2 client usage
• 8fa8e02: 8305931: jdk/jfr/jcmd/TestJcmdDumpPathToGCRoots.java failed with "Expected chains but found none"
• fc7d8f8: 8326521: JFR: CompilerPhase event test fails on windows 32 bit
• b980e1c: 8328988: [macos14] Problem list LightweightEventTest.java which fails due to macOS bug described in JDK-8322653
• e9e042b: 8327631: Update IANA Language Subtag Registry to Version 2024-03-07
• ... and 26 more: https://git.openjdk.org/jdk21u-dev/compare/35f291b9575620d2728193359d8a13aa10b1467b...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[openjdk]

This backport pull request has now been updated with issue from the original commit.

[mlbridge]

Webrevs

• 00: Full (6f9e1261)

[jerboaa]

@GoeLin Please use /issue add for the bugs included in this backport.

[jerboaa]

/reviewers 2

[openjdk]

@jerboaa
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

[jerboaa]

Seems OK to me, but I'd like for somebody else to review it too. @shipilev or @tstuefe perhaps?

[shipilev]

I looked at this change shortly after the initial integration into mainline, and thought it is quite a bit risky. I can look again, now with the history of related bugs. But I have to mention right now that it might still be risky.

[GoeLin]

That's what I thought, too, but it also seems to be quite an issue with 21.
We discussed this, and decided we want to have the whole set of changes addressing the class unloading issue in 21 following Oracle. To reduce the risk we ask for extra reviews :)
It will run through our testing tonight.

[jerboaa]

But I have to mention right now that it might still be risky.

@shipilev No doubt about that. For context: It's part of a larger change ported to Oracle JDK 21.0.4 to fix a class unloading perf regression. See https://bugs.openjdk.org/browse/JDK-8326092 and linked issues.

[tstuefe]

My fear when looking at the patch sequence was that Oracle probably did not test Shenandoah very thoroughly. I also did not particularly like the CLDG changes. And the patches have not had a lot of time to cook in mainline.

[GoeLin]

/issue 8317440

[openjdk]

@GoeLin
Adding additional issue to issue list: 8317440: Lock rank checking fails when code root set is modified with the Servicelock held after JDK-8315503.

[GoeLin]

/issue 8318720

[openjdk]

@GoeLin
Adding additional issue to issue list: 8318720: G1: Memory leak in G1CodeRootSet after JDK-8315503.

[GoeLin]

I don't think this change is a risk for Shenandoah, The shared changes are minimal.
Also, as 22 is released including this change and with all the testing it needed to undergo, and next LTS is only 25 in 1.5 years, I don't think the test coverage will get substantial better.

[GoeLin]

Ok, so now I got some risk assessments. But could I also please get reviews? I.e., is the backport as correct wrt 21 as it is in 22? I could as well ask someone else for a second review?

[shipilev]

All right, this is not very scary, just the normal amount of scary.

[openjdk]

⚠️ @GoeLin This change is now ready for you to apply for maintainer approval. This can be done directly in each associated issue or by using the /approval command.

[shipilev]

Actually, let's ask @tschatzl directly if he knows about any other dependencies that might break this patch in JDK 21, or some other caveats.

[GoeLin]

Hi @shipilev, I would assume Thomas checked this when he backported it, so that we see all follow ups anyways!

[GoeLin]

/integrate

[tschatzl]

Hi @shipilev, I would assume Thomas checked this when he backported it, so that we see all follow ups anyways!

All known to me follow-ups were backported. All of them are referenced in the main backport JDK-8326092

[openjdk]

Going to push as commit e63addb.
Since your change was applied there have been 39 commits pushed to the master branch:

• 686399a: 8318599: HttpURLConnection cache issues leading to crashes in JGSS w/ native GSS introduced by 8303809
• 6c5500b: 8318986: Improve GenericWaitBarrier performance
• f0f2e70: 8326936: RISC-V: Shenandoah GC crashes due to incorrect atomic memory operations
• 7743b6c: Merge
• 05f18b1: 8329838: [21u] Remove designator DEFAULT_PROMOTED_VERSION_PRE=ea for release 21.0.3
• 778a6c4: 8319851: Improve exception logging
• 517bd93: 8322122: Enhance generation of addresses
• 5378f80: 8318340: Improve RSA key implementations
• 8eba98a: 8315708: Enhance HTTP/2 client usage
• 8fa8e02: 8305931: jdk/jfr/jcmd/TestJcmdDumpPathToGCRoots.java failed with "Expected chains but found none"
• ... and 29 more: https://git.openjdk.org/jdk21u-dev/compare/35f291b9575620d2728193359d8a13aa10b1467b...master

Your commit was automatically rebased without conflicts.

[openjdk]

@GoeLin Pushed as commit e63addb.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

[shipilev]

All known to me follow-ups were backported. All of them are referenced in the main backport JDK-8326092

Excellent, this is good to now. I see some related things that are only in 21.0.4-oracle. I assume @GoeLin is handling those :)