JDK-8267562: G1: Missing BOT in Open Archive regions causes long pauses #4265

jianglizhou · 2021-05-31T04:09:40Z

Populate G1BlockOffsetTableParts and associated G1BlockOffsetTable::_offset_array entries for 'open' archive regions during VM initialization at runtime.

G1BlockOffsetTable divides the covered space (Java heap) into “N”-word subregions (“N” is from 2^”LogN”). It uses an _offset_array to tell how far back it must go to find the start of a block that contains the first word of a subregion. Every G1 region (is a G1ContiguousSpace) owns a G1BlockOffsetTablePart (associates to part of the _offset_array), which covers space of the current region. For a mapped open archive heap region with pre-populated Java objects, its G1BlockOffsetTablePart is never setup at runtime because there is no allocation done within the region. As a result, G1BlockOffsetTablePart::block_start() always does lookup from the start (bottom) of an open archive region when called at runtime, regardless if the given 'addr' is near the bottom, top or in the middle of the region. The lookup becomes linear, instead of O(2^LogN). Large heap region size makes the situation worse and young-gen pauses longer.

Removed G1FullGCPrepareTask::G1CalculatePointersClosure::update_bot (introduced by JDK-8264987 change, #3459). It's merged as G1BlockOffsetTablePart::update, which is used for Survivor-turned-to-Old regions in full gc and open archive region. Thanks Thomas Schatzl for pointing to JDK-8264987 change for code reuse.

Tested with tier1 and hotspot_gc tests on linux_x86. Appreciate any help for testing other platforms and more tiers. Thanks!

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8267562: G1: Missing BOT in Open Archive regions causes long pauses

Reviewers

Thomas Schatzl (@tschatzl - Reviewer) ⚠️ Review applies to 0ab6edc
Hamlin Li (@Hamlin-Li - Reviewer) ⚠️ Review applies to 297b3e0

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/4265/head:pull/4265
$ git checkout pull/4265

Update a local copy of the PR:
$ git checkout pull/4265
$ git pull https://git.openjdk.java.net/jdk pull/4265/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 4265

View PR using the GUI difftool:
$ git pr show -t 4265

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/4265.diff

bridgekeeper · 2021-05-31T04:10:43Z

👋 Welcome back jiangli! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2021-05-31T04:12:38Z

@jianglizhou The following label will be automatically applied to this pull request:

hotspot-gc

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2021-05-31T04:16:30Z

Webrevs

tschatzl

Lgtm apart from the comment issue.

tschatzl · 2021-05-31T08:27:03Z

src/hotspot/share/cds/filemap.cpp

+
+    // Populate the open archive regions' G1BlockOffsetTableParts. That ensures
+    // fast G1BlockOffsetTablePart::block_start operations for a given address
+    // within the open archive regions during G1RemSet::refine_card_concurrently.


Not only refinement is subject to the issue but any operation trying to find start of an object (e.g. card table scanning). Please fix the comment accordingly.

Updated comment based on the feedback from Thomas. Thanks for the review.

openjdk · 2021-05-31T08:28:25Z

@jianglizhou This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8267562: G1: Missing BOT in Open Archive regions causes long pauses

Reviewed-by: tschatzl, mli

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 26 new commits pushed to the master branch:

36dc268: 8268036: ProblemList gtest/AsyncLogGtest.java
74b70a5: 8036580: CHA: improve default method support
20479c3: 8065760: CHA: Improve abstract method support
68f3b3a: 8267807: C2: Downcast receiver to target holder during inlining
1c7c0e1: 8267937: Wrong indentation in G1 gc+phases log for free cset subphases
ffd28c4: 8267947: CI: Preserve consistency between has_subklass() and is_subclass_of()
229a6e2: 8267095: Miscellaneous cleanups in vm.runtime.defmeth tests
6149b9a: 8267914: Remove DeferredObjectToKlass workaround
4eb2168: 8267670: Update java.io, java.math, and java.text to use switch expressions
f5634fe: 8267979: C2: Fix verification code in SubTypeCheckNode::Ideal()
... and 16 more: https://git.openjdk.java.net/jdk/compare/66274320251f492b0bba79cdfef88ad5b0d104fa...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

Hamlin-Li

looks good.

jianglizhou · 2021-05-31T19:40:21Z

looks good.

Thanks for the review.

tschatzl

One last comment change :)

tschatzl · 2021-06-01T08:13:37Z

src/hotspot/share/cds/filemap.cpp

+    // This is only needed for open archive regions, but not the closed archive
+    // regions, because objects in closed archive regions are 'immutable'.


Suggested change

// This is only needed for open archive regions, but not the closed archive

// regions, because objects in closed archive regions are 'immutable'.

// This is only needed for open archive regions, but not the closed archive

// regions, because objects in closed archive regions never reference objects

// outside the closed archive regions and they are immutable, so we never

// need their BOT during garbage collection.

jianglizhou · 2021-06-01T16:51:35Z

One last comment change :)

Thank for the suggestion. That does explain more clearly. Updated as suggested.

jianglizhou · 2021-06-01T18:16:27Z

/integrate

openjdk · 2021-06-01T18:18:11Z

@jianglizhou Since your change was applied there have been 30 commits pushed to the master branch:

2b33835: 8265148: StackWatermarkSet being updated during AsyncGetCallTrace
64ba2f0: 8265431: Add -fno-delete-null-pointer-checks to clang builds
d4b4cae: 8268023: Improve diagnostic for HandshakeFailureTest
9d8ad2e: 8267990: Revisit some uses of synchronized in the HttpClient API
36dc268: 8268036: ProblemList gtest/AsyncLogGtest.java
74b70a5: 8036580: CHA: improve default method support
20479c3: 8065760: CHA: Improve abstract method support
68f3b3a: 8267807: C2: Downcast receiver to target holder during inlining
1c7c0e1: 8267937: Wrong indentation in G1 gc+phases log for free cset subphases
ffd28c4: 8267947: CI: Preserve consistency between has_subklass() and is_subclass_of()
... and 20 more: https://git.openjdk.java.net/jdk/compare/66274320251f492b0bba79cdfef88ad5b0d104fa...master

Your commit was automatically rebased without conflicts.

Pushed as commit b17b11e.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Fixed JDK-8267562

297b3e0

openjdk bot added the rfr Pull request is ready for review label May 31, 2021

openjdk bot added the hotspot-gc hotspot-gc-dev@openjdk.org label May 31, 2021

tschatzl approved these changes May 31, 2021

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label May 31, 2021

Hamlin-Li approved these changes May 31, 2021

View reviewed changes

Updated comment based on the review feedback from Thomas.

0ab6edc

tschatzl approved these changes Jun 1, 2021

View reviewed changes

Incorporated comment suggestion from Thomas.

f0fdc24

openjdk bot closed this Jun 1, 2021

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jun 1, 2021

jianglizhou mentioned this pull request Feb 25, 2022

Core algorithms and mechanism for general Java class pre-initialize support for JDK classes. adoptium/jdk11u-fast-startup-incubator#33

Merged

JDK-8267562: G1: Missing BOT in Open Archive regions causes long pauses #4265

JDK-8267562: G1: Missing BOT in Open Archive regions causes long pauses #4265

jianglizhou commented May 31, 2021 •

edited by openjdk bot

bridgekeeper bot commented May 31, 2021

openjdk bot commented May 31, 2021

mlbridge bot commented May 31, 2021 •

edited

tschatzl left a comment

tschatzl May 31, 2021

jianglizhou May 31, 2021

openjdk bot commented May 31, 2021 •

edited

Hamlin-Li left a comment

jianglizhou commented May 31, 2021

tschatzl left a comment

tschatzl Jun 1, 2021

jianglizhou commented Jun 1, 2021

jianglizhou commented Jun 1, 2021

openjdk bot commented Jun 1, 2021

		// This is only needed for open archive regions, but not the closed archive
		// regions, because objects in closed archive regions are 'immutable'.

JDK-8267562: G1: Missing BOT in Open Archive regions causes long pauses #4265

JDK-8267562: G1: Missing BOT in Open Archive regions causes long pauses #4265

Conversation

jianglizhou commented May 31, 2021 • edited by openjdk bot

Progress

Issue

Reviewers

Reviewing

bridgekeeper bot commented May 31, 2021

openjdk bot commented May 31, 2021

mlbridge bot commented May 31, 2021 • edited

Webrevs

tschatzl left a comment

Choose a reason for hiding this comment

tschatzl May 31, 2021

Choose a reason for hiding this comment

jianglizhou May 31, 2021

Choose a reason for hiding this comment

openjdk bot commented May 31, 2021 • edited

Hamlin-Li left a comment

Choose a reason for hiding this comment

jianglizhou commented May 31, 2021

tschatzl left a comment

Choose a reason for hiding this comment

tschatzl Jun 1, 2021

Choose a reason for hiding this comment

jianglizhou commented Jun 1, 2021

jianglizhou commented Jun 1, 2021

openjdk bot commented Jun 1, 2021

jianglizhou commented May 31, 2021 •

edited by openjdk bot

mlbridge bot commented May 31, 2021 •

edited

openjdk bot commented May 31, 2021 •

edited