8286302: Port JEP 425 to PPC64 #10961

reinrich · 2022-11-02T21:54:26Z

Hi,

this is the port of JEP 425: Virtual Threads (Preview)) to PPC64.
More precisely it is the port of vm continuations in hotspot to PPC64. It allows to run with -XX:+VMContinuations which is a prerequisit for 'real' virtual threads (oxymoron?).

Most of the shared code changes are related to a new platform dependent constant frame::metadata_words_at_top. It is either added or subtracted to a frame address or size.

The following is supposed to explain (without real life details) why it is needed in addition to the existing frame::metadata_words. The diagram shows a frame at SP and its stack arguments. The caller frame is located at callers_SP.

  X86 / AARCH64                             PPC64:

  :                 :                       :                 :
  :                 :                       :                 :
  |                 |                       |                 |
  |-----------------|                       |-----------------|
  |                 |                       |                 |
  | stack arguments |                       | stack arguments |
  |                 |<- callers_SP          |                 |
  ===================                       |-----------------|
  |                 |                       |                 |
  | metadata@bottom |                       | metadata@top    |
  |                 |                       |                 |<- callers_SP
  |-----------------|                       ===================
  |                 |                       |                 |
  |                 |                       |                 |
  |                 |                       |                 |
  |                 |                       |                 |
  |                 |<- SP                  |                 |
  ===================                       |-----------------|
                                            |                 |
                                            | metadata@top    |
                                            |                 |<- SP
                                            ===================

On X86 and AARCH64 metadata (that's return address, saved SP/FP, etc.) is stored at the frame bottom (metadata@bottom). On PPC64 it is stored at the frame top (metadata@top) where it affects size and address calculations. Shared code deals with this by making use of the platform dependent constant frame::metadata_words_at_top.

size required to 'freeze' a single frame with its stack arguments in a StackChunk:
sizeof(frame) + sizeof(stack arguments) + frame::metadata_words_at_top
address of stack arguments:
callers_SP + frame::metadata_words_at_top
The value of frame::metadata_words_at_top is 0 words on X86 and AARCH64. On PPC64 it is 4 words.

Please refer to comments I've added for more details (e.g. the comment on StackChunkFrameStream<frame_kind>::frame_size()). Recently I've given a talk about vm continuations and the PPC64 port to my colleagues. It's rather an overview than a deep dive. The slides might serve as well as an intro to the matter.

The pr includes the new test jdk/jdk/internal/vm/Continuation/BasicExp.java which I wrote while doing the port. The test cases vary from simple to not-so-simple. One of the main features is that a CompilationPolicy passed as argument controls which frames are compiled/interpreted when freezing by defining a sliding window of compiled / interpreted frames which produces interesting transitions with and without stack arguments. There is overlap with Basic.java and Fuzz.java. Let me know wether to remove or keep BasicExp.java. Runtime with fastdebug: 2m on Apple M1 Pro and 3m on Intel Xeon E5-2660 v3 @ 2.60GHz. Note that -XX:+VerifyContinuations is explicitly set as a found it very useful, it increases the runtime quite a bit though.

Testing: the change passed our CI testing: most JCK and JTREG tests, also in Xcomp mode, SPECjvm2008, SPECjbb2015, Renaissance Suite and SAP specific tests with fastdebug and release builds on the standard platforms plus PPC64. These tests do include hotspot_loom and jdk_loom JTREG tests which I've also run with TEST_VM_OPTS="-XX:+VerifyContinuations" on X86_64, PPC64, and AARCH64.

Thanks, Richard.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8286302: Port JEP 425 to PPC64

Reviewers

Tyler Steele (@backwaterred - Committer) ⚠️ Review applies to c1d2f878
Martin Doerr (@TheRealMDoerr - Reviewer) ⚠️ Review applies to 22430750

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/10961/head:pull/10961
$ git checkout pull/10961

Update a local copy of the PR:
$ git checkout pull/10961
$ git pull https://git.openjdk.org/jdk pull/10961/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 10961

View PR using the GUI difftool:
$ git pr show -t 10961

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/10961.diff

bridgekeeper · 2022-11-02T22:04:02Z

👋 Welcome back rrich! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

reinrich · 2022-11-02T22:06:24Z

/label hotspot-compiler,hotspot-runtime

openjdk · 2022-11-02T22:07:24Z

@reinrich
The hotspot-compiler label was successfully added.

The hotspot-runtime label was successfully added.

mlbridge · 2022-11-02T23:19:26Z

Webrevs

RealFYang · 2022-11-10T03:44:16Z

Hi, I also tested this on my Linux RISC-V platform which has support for virtual threads. Test results look good. Thanks.

Tier1-4 tests
Tests under test/jdk/java/lang/Thread/virtual with extra options "-XX:+VerifyContinuations -XX:+VerifyStack"
Non-trivial benchmark workloads, like Renaissance, SPECjvm2008, SPECjbb2015, etc.

reinrich · 2022-11-10T09:47:17Z

Hi, I also tested this on my Linux RISC-V platform which has support for virtual threads. Test results look good. Thanks.
* Tier1-4 tests

* Tests under test/jdk/java/lang/Thread/virtual with extra options "-XX:+VerifyContinuations -XX:+VerifyStack"

* Non-trivial benchmark workloads, like Renaissance, SPECjvm2008, SPECjbb2015, etc.

Great. Thanks for the testing.

Richard.

backwaterred · 2022-11-14T23:08:09Z

I see a couple failures on Linux/ppc64le when testing with -XX:+VerifyContinuations.

jdk/internal/vm/Continuation/Fuzz.java#default
jdk/internal/vm/Continuation/Fuzz.java#preserve-fp

It may be reasonable to add these to a ProblemList.txt and address them at a different time.

reinrich · 2022-11-15T08:35:03Z

I see a couple failures on Linux/ppc64le when testing with -XX:+VerifyContinuations.

jdk/internal/vm/Continuation/Fuzz.java#default jdk/internal/vm/Continuation/Fuzz.java#preserve-fp

It may be reasonable to add these to a ProblemList.txt and address them at a different time.

Thanks for looking at the pr. The failures are due to timeouts. Depending on the load on your test system it might be necessary to increment the timeout factor (JTREG="TIMEOUT_FACTOR=8"). There were issues with the test before (see https://bugs.openjdk.org/browse/JDK-8290211).

backwaterred · 2022-11-15T20:48:17Z

The failures are indeed time-dependant, and pass with JTREG="TIMEOUT_FACTOR=16".

backwaterred

These changes are significant. They appear well thought out and well executed. Thanks for submitting this PR.

src/hotspot/cpu/ppc/sharedRuntime_ppc.cpp

src/hotspot/share/oops/stackChunkOop.inline.hpp

test/jdk/jdk/internal/vm/Continuation/BasicExp.java

reinrich · 2022-11-16T10:08:39Z

These changes are significant. They appear well thought out and well executed. Thanks for submitting this PR.

Thanks for your review @backwaterred

TheRealMDoerr

Great work! Looks very good to me. I'm requesting precise node sizes in C2 (let's keep this invariant: nodes should either have a precisely precomputed size or use dynamic computation). The rest are just minor comments and suggestions.
I need more time to study all explanations and to review the test, but I can't see anything which prevents us from shipping it with JDK 20.
Thanks for the ASCII art pictures!

src/hotspot/cpu/aarch64/frame_aarch64.hpp

src/hotspot/cpu/ppc/continuationFreezeThaw_ppc.inline.hpp

src/hotspot/cpu/ppc/frame_ppc.hpp

src/hotspot/cpu/ppc/sharedRuntime_ppc.cpp

src/hotspot/cpu/ppc/stackChunkFrameStream_ppc.inline.hpp

src/hotspot/cpu/ppc/templateInterpreterGenerator_ppc.cpp

src/hotspot/share/runtime/continuationFreezeThaw.cpp

TheRealMDoerr

Thanks for the updates! I'll look more into the explanations and the test, but you don't have to wait for that after you got your 2nd review.

openjdk · 2022-11-18T11:17:17Z

@reinrich This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8286302: Port JEP 425 to PPC64

Reviewed-by: tsteele, mdoerr

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 17 new commits pushed to the master branch:

fd910f7: 8297384: Add IR tests for existing idealizations of arithmetic nodes
cfe5a37: 8297556: Parse::check_interpreter_type fails with assert "must constrain OSR typestate"
74d3bac: 8297417: Poly1305IntrinsicFuzzTest fails with tag mismatch exception
0ed8b33: 8297385: Remove duplicated null typos in javadoc
1fb9dad: 8296419: [REDO] JDK-8295319: pending_cards_at_gc_start doesn't include cards in thread buffers
2f47f83: 8297569: URLPermission constructor throws IllegalArgumentException: Invalid characters in hostname after JDK-8294378
87d1097: 8297530: java.lang.IllegalArgumentException: Negative length on strings concatenation
390e69a: 8297150: Add a @sealedGraph tag to Reference
8effaa8: 8223783: sun/net/www/http/HttpClient/MultiThreadTest.java sometimes detect threads+1 connections
ab1f9ff: 8051627: Invariants about java.net.URI resolve and relativize are wrong
... and 7 more: https://git.openjdk.org/jdk/compare/9c77e41b81ebd28bd92ea7adad605981a5519046...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

reinrich · 2022-11-18T11:17:22Z

Thanks for the updates! I'll look more into the explanations and the test, but you don't have to wait for that after you got your 2nd review.

Thanks a lot indeed!

TheRealMDoerr

Thanks for mitigating the risk of test timeouts!

reinrich · 2022-11-21T22:42:53Z

Thanks for mitigating the risk of test timeouts!

Yes I had to adjust the timeout a little. Also it's not necessary to set -XX:+VerifyContinuations again in the runs that call System.gc().

lmesnik

Not a complete review but some notes about test style.

test/jdk/jdk/internal/vm/Continuation/BasicExp.java

reinrich · 2022-11-22T17:38:32Z

Thanks for your feedback @lmesnik. It help reducing the runtime of the test significantly.

lmesnik · 2022-11-23T01:02:03Z

Thanks for fixing the test.
I haven't done the full review, not adding myself as a reviewer. Don't have any additional comments.

reinrich · 2022-11-25T11:08:19Z

Thanks a lot for the reviews and feedback!
My own testing and GHA have succeeded after merging master.
I'm intending to integrate the port on Monday since I don't expect more feedback.

reinrich · 2022-11-28T08:07:05Z

/integrate

openjdk · 2022-11-28T08:09:22Z

Going to push as commit 43d1173.
Since your change was applied there have been 28 commits pushed to the master branch:

d610211: 8297408: Consolidate code in runtime/ErrorHandling
2f83b5c: 8297640: Increase buffer size for buf (insert_features_names) in Abstract_VM_Version::insert_features_names
50f9043: 8297451: ProcessHandleImpl should assert privilege when modifying reaper thread
99d3840: 8297359: RISC-V: improve performance of floating Max Min intrinsics
6c05771: 8295447: NullPointerException with invalid pattern matching construct in constructor call
76a24c3: 8297145: Add a @sealedGraph tag to ConstantDesc
099b42f: 8297148: Add a @sealedGraph tag to CallSite
85ddd8f: 8295253: Remove kludge from v1_0/PerfDataBuffer.java
952e100: 8297431: [JVMCI] HotSpotJVMCIRuntime.encodeThrowable should not throw an exception
08e6a82: 8297590: [TESTBUG] HotSpotResolvedJavaFieldTest does not run
... and 18 more: https://git.openjdk.org/jdk/compare/9c77e41b81ebd28bd92ea7adad605981a5519046...master

Your commit was automatically rebased without conflicts.

openjdk · 2022-11-28T08:09:43Z

@reinrich Pushed as commit 43d1173.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Loom ppc64le port

c1564a2

openjdk bot added hotspot-compiler hotspot-compiler-dev@openjdk.org hotspot-runtime hotspot-runtime-dev@openjdk.org labels Nov 2, 2022

reinrich marked this pull request as ready for review November 2, 2022 23:11

openjdk bot added the rfr Pull request is ready for review label Nov 2, 2022

reinrich added 4 commits November 4, 2022 10:04

Use callers_sp for fsize calculation in recurse_freeze_interpreted_frame

0d12b05

Merge branch 'master' into 8286302_Port_JEP_425_to_PPC64

406b065

Changes lost in merge

4c27b76

Fix cpp condition and add PPC64

c1d2f87

backwaterred approved these changes Nov 15, 2022

View reviewed changes

src/hotspot/cpu/ppc/sharedRuntime_ppc.cpp Outdated Show resolved Hide resolved

src/hotspot/share/oops/stackChunkOop.inline.hpp Show resolved Hide resolved

Feedback from backwaterred

f42de6b

turbanoff reviewed Nov 16, 2022

View reviewed changes

test/jdk/jdk/internal/vm/Continuation/BasicExp.java Outdated Show resolved Hide resolved

turbanoff reviewed Nov 16, 2022

View reviewed changes

test/jdk/jdk/internal/vm/Continuation/BasicExp.java Outdated Show resolved Hide resolved

turbanoff reviewed Nov 16, 2022

View reviewed changes

test/jdk/jdk/internal/vm/Continuation/BasicExp.java Outdated Show resolved Hide resolved

Cleanup BasicExp.java

7276a8e

TheRealMDoerr suggested changes Nov 17, 2022

View reviewed changes

Feedback Martin

116839e

TheRealMDoerr approved these changes Nov 18, 2022

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Nov 18, 2022

Cleanup BasicExp test

2243075

TheRealMDoerr approved these changes Nov 21, 2022

View reviewed changes

lmesnik suggested changes Nov 22, 2022

View reviewed changes

Feedback Leonid

5cf9074

More Feedback Leonid

5f59f2c

reinrich added 2 commits November 24, 2022 10:15

Merge branch 'master' into 8286302_Port_JEP_425_to_PPC64

0b7e325

Rename test BasicExp -> BasicExt

817ec63

openjdk bot added the integrated Pull request has been integrated label Nov 28, 2022

openjdk bot closed this Nov 28, 2022

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Nov 28, 2022

reinrich deleted the 8286302_Port_JEP_425_to_PPC64 branch January 24, 2025 08:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8286302: Port JEP 425 to PPC64 #10961

8286302: Port JEP 425 to PPC64 #10961

reinrich commented Nov 2, 2022 •

edited by openjdk bot

Loading

bridgekeeper bot commented Nov 2, 2022

reinrich commented Nov 2, 2022

openjdk bot commented Nov 2, 2022

mlbridge bot commented Nov 2, 2022 •

edited

Loading

RealFYang commented Nov 10, 2022 •

edited

Loading

reinrich commented Nov 10, 2022

backwaterred commented Nov 14, 2022

reinrich commented Nov 15, 2022

backwaterred commented Nov 15, 2022

backwaterred left a comment

reinrich commented Nov 16, 2022

TheRealMDoerr left a comment

TheRealMDoerr left a comment

openjdk bot commented Nov 18, 2022 •

edited

Loading

reinrich commented Nov 18, 2022

TheRealMDoerr left a comment

reinrich commented Nov 21, 2022

lmesnik left a comment

reinrich commented Nov 22, 2022

lmesnik commented Nov 23, 2022

reinrich commented Nov 25, 2022

reinrich commented Nov 28, 2022

openjdk bot commented Nov 28, 2022

openjdk bot commented Nov 28, 2022

8286302: Port JEP 425 to PPC64 #10961

8286302: Port JEP 425 to PPC64 #10961

Conversation

reinrich commented Nov 2, 2022 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

bridgekeeper bot commented Nov 2, 2022

reinrich commented Nov 2, 2022

openjdk bot commented Nov 2, 2022

mlbridge bot commented Nov 2, 2022 • edited Loading

Webrevs

RealFYang commented Nov 10, 2022 • edited Loading

reinrich commented Nov 10, 2022

backwaterred commented Nov 14, 2022

reinrich commented Nov 15, 2022

backwaterred commented Nov 15, 2022

backwaterred left a comment

Choose a reason for hiding this comment

reinrich commented Nov 16, 2022

TheRealMDoerr left a comment

Choose a reason for hiding this comment

TheRealMDoerr left a comment

Choose a reason for hiding this comment

openjdk bot commented Nov 18, 2022 • edited Loading

reinrich commented Nov 18, 2022

TheRealMDoerr left a comment

Choose a reason for hiding this comment

reinrich commented Nov 21, 2022

lmesnik left a comment

Choose a reason for hiding this comment

reinrich commented Nov 22, 2022

lmesnik commented Nov 23, 2022

reinrich commented Nov 25, 2022

reinrich commented Nov 28, 2022

openjdk bot commented Nov 28, 2022

openjdk bot commented Nov 28, 2022

reinrich commented Nov 2, 2022 •

edited by openjdk bot

Loading

mlbridge bot commented Nov 2, 2022 •

edited

Loading

RealFYang commented Nov 10, 2022 •

edited

Loading

openjdk bot commented Nov 18, 2022 •

edited

Loading