8351464: Shenandoah: Hang on ShenandoahController::handle_alloc_failure when run test TestAllocHumongousFragment#generational

[earthling-amzn]

Failed allocations may race to cancel the GC with the collector who is working to clear the cancelled GC. When the GC wins this race, it will fail to notify threads that are waiting for the failed GC cycle to complete. This change sees allocation waiters notified any time a GC completes without being cancelled.

Testing

Ran TestAllocHumongousFragment#generational 6,500 times without failures.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8351464: Shenandoah: Hang on ShenandoahController::handle_alloc_failure when run test TestAllocHumongousFragment#generational (Bug - P3)

Reviewers

• Xiaolong Peng (@pengxiaolong - Author)
• Y. Srinivas Ramakrishna (@ysramakrishna - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/23997/head:pull/23997
$ git checkout pull/23997

Update a local copy of the PR:
$ git checkout pull/23997
$ git pull https://git.openjdk.org/jdk.git pull/23997/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 23997

View PR using the GUI difftool:
$ git pr show -t 23997

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/23997.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back wkemper! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@earthling-amzn This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8351464: Shenandoah: Hang on ShenandoahController::handle_alloc_failure when run test TestAllocHumongousFragment#generational

Reviewed-by: xpeng, ysr

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 87 new commits pushed to the master branch:

• 8e53063: 8352275: Clean up dead code in jpackage revealed with improved negative test coverage
• c8a11f2: 8352293: jpackage tests build rpm packages on Ubuntu test machines after JDK-8351372
• 4a02de8: 8352098: -Xrunjdwp fails on static JDK
• 355b2f3: 8351374: Improve comment about queue.remove timeout in CleanerImpl.run
• a3540be: 8352163: [AIX] SIGILL in AttachOperation::ReplyWriter::write_fully after 8319055
• 53c5b93: 8352180: AttachListenerThread causes many tests to timeout on Windows
• b025d8c: 8350663: AArch64: Enable UseSignumIntrinsic by default
• a37d843: 8352015: LIBVERIFY_OPTIMIZATION remove special optimization settings
• b891bfa: 8352022: RISC-V: Support Zfa fminm_h/fmaxm_h for float16
• 20f1bca: 8351187: Add JFR monitor notification event
• ... and 77 more: https://git.openjdk.org/jdk/compare/1dd9cf1018f33af34f1f822cb0d1dc706353bbdf...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[openjdk]

@earthling-amzn The following labels will be automatically applied to this pull request:

• hotspot-gc
• shenandoah

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 03: Full - Incremental (f72e71c4)
• 02: Full - Incremental (3a16131b)
• 01: Full - Incremental (cb9cd72f)
• 00: Full (d0168ca9)

[pengxiaolong]

The bug should also exist in classical Shenandoah w/o generation, I think ShenandoahControlThread also need to be updated to fix the bug, even it seems not happening in ShenandoahControlThread in the jtreg test.

[earthling-amzn]

Not sure I want to change ShenandoahControlThread. It uses a different mechanism to track whether or not to notify. It only notifies when it services the alloc failure request (it doesn't depend on the shared cancelled_gc state the same way the generational mode does). In the scenario that leads to this live lock for the generational mode, the default mode would not notify the waiters upon successful completion of the concurrent cycle. It would notify them after the subsequent degenerated cycle.

[pengxiaolong]

Not sure I want to change ShenandoahControlThread. It uses a different mechanism to track whether or not to notify. It only notifies when it services the alloc failure request (it doesn't depend on the shared cancelled_gc state the same way the generational mode does). In the scenario that leads to this live lock for the generational mode, the default mode would not notify the waiters upon successful completion of the concurrent cycle. It would notify them after the subsequent degenerated cycle.

It does use the shared cancelled_cause, see the code here https://github.com/openjdk/jdk/blob/master/src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp#L68 and at line 171, ShenandoahControlThread does have same problem, alloc_failure_pending is evaluated using shared cancelled_cause before starting a cycle.

[earthling-amzn]

@pengxiaolong , yes - I agree. Good catch.

[pengxiaolong]

Looks good to me.

[ysramakrishna]

Is the description in the PR still valid?

This change directly tracks the number of threads waiting due to an allocation failure, rather than indirectly tracking them through the cancelled gc state.

[earthling-amzn]

Fixed the description.

[ysramakrishna]

LGTM!

[earthling-amzn]

/integrate

[openjdk]

Going to push as commit 20d4fe3.
Since your change was applied there have been 87 commits pushed to the master branch:

• 8e53063: 8352275: Clean up dead code in jpackage revealed with improved negative test coverage
• c8a11f2: 8352293: jpackage tests build rpm packages on Ubuntu test machines after JDK-8351372
• 4a02de8: 8352098: -Xrunjdwp fails on static JDK
• 355b2f3: 8351374: Improve comment about queue.remove timeout in CleanerImpl.run
• a3540be: 8352163: [AIX] SIGILL in AttachOperation::ReplyWriter::write_fully after 8319055
• 53c5b93: 8352180: AttachListenerThread causes many tests to timeout on Windows
• b025d8c: 8350663: AArch64: Enable UseSignumIntrinsic by default
• a37d843: 8352015: LIBVERIFY_OPTIMIZATION remove special optimization settings
• b891bfa: 8352022: RISC-V: Support Zfa fminm_h/fmaxm_h for float16
• 20f1bca: 8351187: Add JFR monitor notification event
• ... and 77 more: https://git.openjdk.org/jdk/compare/1dd9cf1018f33af34f1f822cb0d1dc706353bbdf...master

Your commit was automatically rebased without conflicts.

[openjdk]

@earthling-amzn Pushed as commit 20d4fe3.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.