8250597: G1: Improve inlining around trim_queue

[dchuyko]

This change improves latency with G1 in some cases and makes trim_queue() code structure closer to current mainline.

The original patch does not apply cleanly in particular because of https://bugs.openjdk.java.net/browse/JDK-8200545 . However the refactoring is rather simple because it is a rearrangement of code between functions with compiler hints. It has been reproduced (short description in the next comment).

The change is isolated and is a set of code moves so the risk is low.

Testing: tier1, tier2 on aarch64 and x86_64 in development, the change has been included in Liberica JDK 11u EA for some time.

As an example of performance improvement, Critical-jOPS on Graviton2 raised +1%:+6.2% for 8GB heap.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8250597: G1: Improve inlining around trim_queue

Reviewers

• Paul Hohensee (@phohensee - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk11u-dev pull/26/head:pull/26
$ git checkout pull/26

Update a local copy of the PR:
$ git checkout pull/26
$ git pull https://git.openjdk.java.net/jdk11u-dev pull/26/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 26

View PR using the GUI difftool:
$ git pr show -t 26

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk11u-dev/pull/26.diff

[bridgekeeper]

👋 Welcome back dchuyko! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

This backport pull request has now been updated with issue and summary from the original commit.

[mlbridge]

Webrevs

• 00: Full (1e8d51f8)

[dchuyko]

G1ParScanThreadState::trim_queue():
Moved from g1ParScanThreadState.cpp to g1ParScanThreadState.inline.hpp, 'do-while !is_empty()' eleiminated.

G1ParScanThreadState::do_oop_evac():
Moved from g1ParScanThreadState.inline.hpp to g1ParScanThreadState.cpp, copy_to_survivor_space() -> do_copy_to_survivor_space()

[G1ParScanThreadState::do_partial_array()]
G1ParScanThreadState::do_oop_partial_array():
Moved from g1ParScanThreadState.inline.hpp to g1ParScanThreadState.cpp, made not 'inline'.

[G1ParScanThreadState::dispatch_task()]
G1ParScanThreadState::dispatch_reference():
Moved from g1ParScanThreadState.inline.hpp to g1ParScanThreadState.cpp, made not 'inline'.

G1ParScanThreadState::needs_partial_trimming():
Made not 'inline' in g1ParScanThreadState.inline.hpp.

G1ParScanThreadState::trim_queue_to_threshold()
Declaration moved from g1ParScanThreadState.inline.hpp to g1ParScanThreadState.cpp, 'inline' changed to 'ATTRIBUTE_FLATTEN NOINLINE' in g1ParScanThreadState.cpp, got 'do-while !overflow_empty()', made not 'inline' in g1ParScanThreadState.hpp.

G1ParScanThreadState::steal_and_trim_queue():
Declaration moved from g1ParScanThreadState.inline.hpp to g1ParScanThreadState.cpp, with 'ATTRIBUTE_FLATTEN', comments, made not 'inline' in g1ParScanThreadState.hpp.

G1ParScanThreadState::allocate_copy_slow():
Added in g1ParScanThread_surviving_young_words[young_index] += word_szState.cpp as 'NOINLINE', extracted from copy_to_survivor_space/if (obj_ptr == NULL) except the case when handle_evacuation_failure_par() is needed, part before tracer is wrapped in if (!(dest_attr->is_old() && _old_gen_is_full)), tracing is wrapped in if (obj_ptr != NULL), declaration added in g1ParScanThreadState.hpp.

G1ParScanThreadState::undo_allocation():
Added in g1ParScanThreadState.cpp as 'NOINLINE', extracted from copy_to_survivor_space/_plab_allocator->undo_allocation, declaration added in g1ParScanThreadState.hpp.

G1ParScanThreadState::copy_to_survivor_space():
Renamed to do_copy_to_survivor_space() in g1ParScanThreadState.cpp + comment, copy_to_survivor_space() added as a wrapper in g1ParScanThreadState.cpp, do_copy_to_survivor_space() declaration added and copy_to_survivor_space() declaration corrected in g1ParScanThreadState.hpp.
assert(is_in_cset()) has been added.
if(_old_gen_is_full && dest_state.is_old()) block moved.
_surviving_young_words[young_index] += word_sz moved into new block around young_index > 0 asserts

G1ParScanThreadState::handle_evacuation_failure_par():
'NOINLINE' in g1ParScanThreadState.cpp.

G1ParScanThreadState::is_partially_trimmed():
Manually inlined into trim_queue_partially().

G1ParScanThreadState::trim_queue_partially():
'inline' in g1ParScanThreadState.hpp instead of g1ParScanThreadState.inline.hpp.
do-while moved (in the patch it is replaced by asserts).

G1ParScanThreadState::trim_queue():
Made 'inline' in g1ParScanThreadState.hpp.

ATTRIBUTE_FLATTEN
Defined in globalDefinitions.hpp, globalDefinitions_gcc.hpp.

Manually inline deal_with_reference()-s into G1ParScanThreadState::dispatch_reference().

[mlbridge]

Mailing list message from Dmitry Chuyko on jdk-updates-dev:

CC'ing hotspot-gc-dev.

On 6/11/21 5:08 PM, Dmitry Chuyko wrote:

[mlbridge]

Mailing list message from Dmitry Chuyko on jdk-updates-dev:

CC'ing hotspot-gc-dev.

On 6/11/21 5:08 PM, Dmitry Chuyko wrote:

[bridgekeeper]

@dchuyko This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[phohensee]

Lgtm.

[openjdk]

@dchuyko This change now passes all automated pre-integration checks.

After integration, the commit message for the final commit will be:

8250597: G1: Improve inlining around trim_queue

Refactor, using NOINLINE and (new) ATTRIBUTE_FLATTEN for control.

Reviewed-by: phh

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 488 new commits pushed to the master branch:

• 25c33bc: 8239422: [TESTBUG] compiler/c1/TestPrintIRDuringConstruction.java failed when C1 is disabled
• 00255bd: 8263068: Rename safefetch.hpp to safefetch.inline.hpp
• 1297c26: 8275051: Shenandoah: Correct ordering of requested gc cause and gc request flag
• 4b6e487: 8197800: Test java/awt/Focus/NonFocusableWindowTest/NoEventsTest.java fails on Windows
• 5a2b2ac: 8198619: java/awt/Focus/FocusTraversalPolicy/ButtonGroupLayoutTraversal/ButtonGroupLayoutTraversalTest.java fails on mac
• b449303: 8171998: javax/swing/JMenu/4692443/bug4692443.java fails on Windows
• cdc48c2: 8256201: java/awt/FullScreen/FullscreenWindowProps/FullscreenWindowProps.java failed
• 93e98ad: 8266579: Update test/jdk/java/lang/ProcessHandle/PermissionTest.java & test/jdk/java/sql/testng/util/TestPolicy.java
• 1772cf1: 8272708: [Test]: Cleanup: test/jdk/security/infra/java/security/cert/CertPathValidator/certification/BuypassCA.java no longer needs ocspEnabled
• 0f9d6bf: 8247510: typo in IllegalHandshakeMessage
• ... and 478 more: https://git.openjdk.java.net/jdk11u-dev/compare/9dc882645a6eb79e068fcaf018c763d9036d7da2...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@phohensee) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

[bridgekeeper]

@dchuyko This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[bridgekeeper]

@dchuyko This pull request has been inactive for more than 8 weeks and will now be automatically closed. If you would like to continue working on this pull request in the future, feel free to reopen it! This can be done using the /open pull request command.