8256368: Avoid repeated upcalls into Java to re-resolve MH/VH linkers/invokers

[iwanowww]

It's another attempt to fix JDK-8256368 (previous one was #1453).

@coleenp raised a question about resolution failure case (indy_resolution_failed()) and after a closer look it turned out the logic is broken there.

After thinking more about it, I decided to completely drop resolution failure case handling and always perform re-resolution.
For MethodHandle/VarHandle invokers resolution failure is very uncommon, so going through re-resolution step seems well-justified.

Original description:

Method linkage of invokehandle instructions involve an upcall into Java
(MethodHandleNatives::linkMethod), but the result is not cached. Unless the
upcall behaves idempotently (which is highly desirable, but not guaranteed),
repeated invokehandle resolution attempts enter a vicious cycle in tiered mode:
switching to a higher tier involves call site re-resolution in generated code,
but re-resolution installs a fresh method which puts execution back into
interpreter.

(Another prerequisite is no inlining through a invokehandle which doesn't
normally happen in practice - relevant methods are marked w/ @forceinline -
except some testing modes, -Xcomp in particular.)

Proposed fix is to inspect ConstantPoolCache first. Previous resolution
attempts from interpreter and C1 records their results there and it stabilises
the execution.

Testing: hs-tier1 - hs-tier4

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8256368: Avoid repeated upcalls into Java to re-resolve MH/VH linkers/invokers

Reviewers

• Dean Long (@dean-long - Reviewer)
• Vladimir Kozlov (@vnkozlov - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/7429/head:pull/7429
$ git checkout pull/7429

Update a local copy of the PR:
$ git checkout pull/7429
$ git pull https://git.openjdk.java.net/jdk pull/7429/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 7429

View PR using the GUI difftool:
$ git pr show -t 7429

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/7429.diff

[bridgekeeper]

👋 Welcome back vlivanov! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@iwanowww
The hotspot-runtime label was successfully added.

The hotspot-compiler label was successfully added.

[mlbridge]

Webrevs

• 00: Full (31547fa1)

[dean-long]

Looks good. Is there any reason why we had this optimization for invokedynamic but not invokehandle?

[openjdk]

@iwanowww This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8256368: Avoid repeated upcalls into Java to re-resolve MH/VH linkers/invokers

Reviewed-by: dlong, kvn

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 4 new commits pushed to the master branch:

• 84868e3: 8281275: Upgrading from 8 to 11 no longer accepts '/' as filepath separator in gc paths
• 58c2bd3: 8281536: JFR: Improve jdk.jfr.ContentType documentation
• 83b6e4b: 8281294: [vectorapi] FIRST_NONZERO reduction operation throws IllegalArgumentExcept on zero vectors
• 039313d: 8054449: Incompatible type in example code in TreePath

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[iwanowww]

Thanks for the reviews, Vladimir and Dean.

Is there any reason why we had this optimization for invokedynamic but not invokehandle?

It's optional for invokehandle, but mandatory for invokedynamic.

invokedynamic can be linked by arbitrary user code (as part of bootstrap method execution) and calling BSM multiple times would violate the JVMS. There are no guarantees about BSM behavior, so it can produce different results on repeated invocations.

In contrast, invokehandle is linked solely by trusted code (MethodHandleNatives.linkMethod) and it's fine to re-resolve the call site on failure (LinkageError), since the result should be the same (unless it was an intermittent JVM failure, like OOM or StackOverflowError).

[iwanowww]

/integrate

[openjdk]

Going to push as commit 534e557.
Since your change was applied there have been 36 commits pushed to the master branch:

• 95f198b: 8274980: Improve adhoc build version strings
• c61d629: 8281553: Ensure we only require liveness from mach-nodes with barriers
• 2597206: 8280783: Parallel: Refactor PSCardTable::scavenge_contents_parallel
• 2632d40: 8281637: Remove unused VerifyOption_G1UseNextMarking
• 46f5229: 8281539: IGV: schedule approximation computes immediate dominators wrongly
• 1ef45c5: 8280799: С2: assert(false) failed: cyclic dependency prevents range check elimination
• 483d4b9: 8281505: Add CompileCommand PrintIdealPhase
• adbe066: 8239927: Product variable PrefetchFieldsAhead is unused and should be removed
• eff5daf: 8274939: Incorrect size of the pixel storage is used by the robot on macOS
• 8acfbc2: 8281675: VMDeprecatedOptions test fails after JDK-8278423
• ... and 26 more: https://git.openjdk.java.net/jdk/compare/3ce1c5b6ce02749ef8f9d35409b7bcbf27f47203...master

Your commit was automatically rebased without conflicts.

[openjdk]

@iwanowww Pushed as commit 534e557.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.