8342772: Assert in LateInlineMHCallGenerator::do_late_inline_check

[dafedafe]

Issue

The assertion

jdk/src/hotspot/share/opto/callGenerator.cpp

Line 421 in 1c16d8a

assert(!cg->is_late_inline() || cg->is_mh_late_inline() || cg->is_virtual_late_inline() || AlwaysIncrementalInline || StressIncrementalInlining, "we're doing late inlining");

in LateInlineMHCallGenerator::do_late_inline_check fails while running a 24h-long Renaissance benchmark test.

Cause

The assert failure is due to both !cg->is_late_inline() and cg->is_mh_late_inline() being false because the CallGenerator cg is of type LateInlineVirtualCallGenerator. cg is created just above by calling for_method_handle_inline. The only way for cg to be of type LateInlineVirtualCallGenerator is that for_method_handle_inline (which finds out that the intrinsic id is vmIntrinsics::_linkToInterface) calls optimize_virtual_call, which apparently cannot devirtualize the call, and then calls call_generator and this creates and returns a LateInlineVirtualCallGenerator (at the end of the method).

Fix

There seem to be no apparent reason why the CallGenerator returned by for_method_handle_inline couldn't be of type LateInlineVirtualCallGenerator. So the sensible fix is to relax the assert to accommodate this type of call generator.

Testing

Tier 1-3+
Failing test before and after.

Unfortunately it has proven impractical to create a specific test that consistently (or even only intermittently) reproduces the issue.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8342772: Assert in LateInlineMHCallGenerator::do_late_inline_check (Bug - P3)(⚠️ The fixVersion in this issue is [26] but the fixVersion in .jcheck/conf is 27, a new backport will be created when this pr is integrated.)

Reviewers

• Vladimir Ivanov (@iwanowww - Reviewer)
• Christian Hagedorn (@chhagedorn - Reviewer)
• Tobias Hartmann (@TobiHartmann - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/28793/head:pull/28793
$ git checkout pull/28793

Update a local copy of the PR:
$ git checkout pull/28793
$ git pull https://git.openjdk.org/jdk.git pull/28793/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 28793

View PR using the GUI difftool:
$ git pr show -t 28793

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/28793.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back dfenacci! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@dafedafe This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8342772: Assert in LateInlineMHCallGenerator::do_late_inline_check

Reviewed-by: vlivanov, chagedorn, thartmann

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 119 new commits pushed to the master branch:

• 5c6947f: 8373429: gc/g1/TestCodeCacheUnloadDuringConcCycle fails on various platforms
• 6b3c1e0: 8373833: "error.cert.not.found" and "error.explicit-sign-no-cert" errors duplicate each other
• 7c979c1: 8374308: ImageBufferCache has no effect and can be removed
• ... and 116 more: https://git.openjdk.org/jdk/compare/4f283f188c43cb25c4eafcdf22eb7f58eae286cc...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[openjdk]

@dafedafe The following label will be automatically applied to this pull request:

• hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 00: Full (bfb94ce0)

[iwanowww]

Looks good.

[chhagedorn]

Looks good to me, too.

[TobiHartmann]

Looks good.

[dafedafe]

Thanks @iwanowww @TobiHartmann @chhagedorn for reviewing!

[dafedafe]

/integrate

[openjdk]

Going to push as commit c1c0ac8.
Since your change was applied there have been 119 commits pushed to the master branch:

• 5c6947f: 8373429: gc/g1/TestCodeCacheUnloadDuringConcCycle fails on various platforms
• 6b3c1e0: 8373833: "error.cert.not.found" and "error.explicit-sign-no-cert" errors duplicate each other
• 7c979c1: 8374308: ImageBufferCache has no effect and can be removed
• ... and 116 more: https://git.openjdk.org/jdk/compare/4f283f188c43cb25c4eafcdf22eb7f58eae286cc...master

Your commit was automatically rebased without conflicts.

[openjdk]

@dafedafe Pushed as commit c1c0ac8.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

[dafedafe]

/backport :jdk26

[openjdk]

@dafedafe the backport was successfully created on the branch backport-dafedafe-c1c0ac87-jdk26 in my personal fork of openjdk/jdk. To create a pull request with this backport targeting openjdk/jdk:jdk26, just click the following link:

➡️ Create pull request

The title of the pull request is automatically filled in correctly and below you find a suggestion for the pull request body:

Hi all,

This pull request contains a backport of commit c1c0ac87 from the openjdk/jdk repository.

The commit being backported was authored by Damon Fenacci on 7 Jan 2026 and was reviewed by Vladimir Ivanov, Christian Hagedorn and Tobias Hartmann.

Thanks!

If you need to update the source branch of the pull then run the following commands in a local clone of your personal fork of openjdk/jdk:

$ git fetch https://github.com/openjdk-bots/jdk.git backport-dafedafe-c1c0ac87-jdk26:backport-dafedafe-c1c0ac87-jdk26
$ git checkout backport-dafedafe-c1c0ac87-jdk26
# make changes
$ git add paths/to/changed/files
$ git commit --message 'Describe additional changes made'
$ git push https://github.com/openjdk-bots/jdk.git backport-dafedafe-c1c0ac87-jdk26