8214455: Relocate CDS archived regions to the top of the G1 heap #3349

iklam · 2021-04-05T21:48:43Z

When the runtime heap and dump time heap have different sizes, it's possible that the archived heap regions are mapped to the middle of the runtime heap. Because the archived heap regions are pinned, this could reduce the size of the largest "humongous" array allocation. In the worst case, the maximum allocatable array length may be half of the optimal value.

The fix is to relocate the archived regions to the top of the G1 heap (currently archived regions are supported only by G1 on 64-bit JVM).

Note that usually the top of the G1 heap is placed just below the 32GB boundary. As a result, the archived heap regions are at the same location between run time and dump time, so no relocation is necessary.

In mach5 testing, we occasionally run into this problem (see https://bugs.openjdk.java.net/browse/JDK-8239089), probably because we fail to reserve the heap below 32GB due to address space layout randomization (ASLR).

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8214455: Relocate CDS archived regions to the top of the G1 heap

Reviewers

Thomas Schatzl (@tschatzl - Reviewer) ⚠️ Review applies to def7db7
Calvin Cheung (@calvinccheung - Reviewer) ⚠️ Review applies to bedd676

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/3349/head:pull/3349
$ git checkout pull/3349

Update a local copy of the PR:
$ git checkout pull/3349
$ git pull https://git.openjdk.java.net/jdk pull/3349/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 3349

View PR using the GUI difftool:
$ git pr show -t 3349

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/3349.diff

bridgekeeper · 2021-04-05T21:49:02Z

👋 Welcome back iklam! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2021-04-05T21:51:20Z

@iklam The following label will be automatically applied to this pull request:

hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

iklam · 2021-04-05T21:55:48Z

/label add hotspot-gc

openjdk · 2021-04-05T21:56:34Z

@iklam
The hotspot-gc label was successfully added.

mlbridge · 2021-04-05T22:02:41Z

Webrevs

tschatzl

Lgtm.

tschatzl · 2021-04-06T09:35:01Z

src/hotspot/share/memory/filemap.cpp

@@ -1837,6 +1842,9 @@ void FileMapInfo::map_heap_regions_impl() {
      log_info(cds)("the desired range " PTR_FORMAT " - "  PTR_FORMAT, p2i(range.start()), p2i(range.end()));
      log_info(cds)("is outside of the heap " PTR_FORMAT " - "  PTR_FORMAT, p2i(CompressedOops::begin()), p2i(CompressedOops::end()));
      _heap_pointers_need_patching = true;
+    } else if (header()->heap_end() != CompressedOops::end()) {
+      log_info(cds)("CDS heap data need to be relocated to the end of the runtime heap to reduce fragmentation");


s/need/needs

tschatzl · 2021-04-06T09:39:02Z

test/hotspot/jtreg/runtime/cds/appcds/cacheObject/HeapFragmentationTest.java

+ * @summary Relocate CDS archived regions to the top of the G1 heap
+ * @bug 8214455
+ * @requires vm.cds.archived.java.heap
+ * @requires (sun.arch.data.model == "64"  & os.maxMemory > 4g)


additional space before the &

Fixed. Thanks for the review.

openjdk · 2021-04-06T09:42:38Z

@iklam This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8214455: Relocate CDS archived regions to the top of the G1 heap

Reviewed-by: tschatzl, ccheung

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 1 new commit pushed to the master branch:

22b20f8: 8264424: Support OopStorage bulk allocation

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

calvinccheung · 2021-04-07T05:34:08Z

test/hotspot/jtreg/runtime/cds/appcds/cacheObject/HeapFragmentationTest.java

+        // First, make sure the test runs without CDS
+        TestCommon.runWithoutCDS(TestCommon.concat(execArgs, runTimeHeapSize, mainClass, BUFF_SIZE))
+          .assertNormalExit(successOutput);
+
+        // Run with CDS. The archived heap regions should be relocated to avoid fragmentation.
+        TestCommon.run(TestCommon.concat(execArgs, runTimeHeapSize, mainClass,  BUFF_SIZE))
+          .assertNormalExit(successOutput);


The assertNormalExit line should be indented four spaces relative to the previous line.

calvinccheung

Looks good. Just one minor nit in the test case.

…-top

iklam · 2021-04-08T00:39:13Z

Thanks @tschatzl and @calvinccheung for the review.
/integrate

openjdk · 2021-04-08T00:42:02Z

@iklam Since your change was applied there have been 4 commits pushed to the master branch:

88eb291: 8264809: test-lib fails to build due to some warnings in ASN1Formatter and jfr
a863ab6: 8264551: Unexpected warning when jpackage creates an exe
6e2b82a: 8264731: Introduce InstanceKlass::method_at_itable_or_null()
22b20f8: 8264424: Support OopStorage bulk allocation

Your commit was automatically rebased without conflicts.

Pushed as commit 78d1164.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8214455: Relocate CDS archived regions to the top of the G1 heap

52fd224

openjdk bot added the hotspot-runtime hotspot-runtime-dev@openjdk.org label Apr 5, 2021

Added -Xlog:cds=debug in case this test fails again

def7db7

openjdk bot added the hotspot-gc hotspot-gc-dev@openjdk.org label Apr 5, 2021

iklam marked this pull request as ready for review April 5, 2021 21:56

openjdk bot added the rfr Pull request is ready for review label Apr 5, 2021

tschatzl approved these changes Apr 6, 2021

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Apr 6, 2021

@tschatzl review -- fix typos and whitespace

bedd676

calvinccheung reviewed Apr 7, 2021

View reviewed changes

calvinccheung approved these changes Apr 7, 2021

View reviewed changes

ioilam added 2 commits April 7, 2021 10:56

fixed indentation (review comment by @calvinccheung)

dd3ce27

Merge branch 'master' into 8214455-relocated-archived-regions-to-heap…

43ee968

…-top

openjdk bot closed this Apr 8, 2021

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Apr 8, 2021

8214455: Relocate CDS archived regions to the top of the G1 heap #3349

8214455: Relocate CDS archived regions to the top of the G1 heap #3349

iklam commented Apr 5, 2021 •

edited by openjdk bot

bridgekeeper bot commented Apr 5, 2021

openjdk bot commented Apr 5, 2021

iklam commented Apr 5, 2021

openjdk bot commented Apr 5, 2021

mlbridge bot commented Apr 5, 2021 •

edited

tschatzl left a comment

tschatzl Apr 6, 2021

iklam Apr 6, 2021

tschatzl Apr 6, 2021

iklam Apr 6, 2021

openjdk bot commented Apr 6, 2021 •

edited

calvinccheung Apr 7, 2021

calvinccheung left a comment

iklam commented Apr 8, 2021

openjdk bot commented Apr 8, 2021

8214455: Relocate CDS archived regions to the top of the G1 heap #3349

8214455: Relocate CDS archived regions to the top of the G1 heap #3349

Conversation

iklam commented Apr 5, 2021 • edited by openjdk bot

Progress

Issue

Reviewers

Reviewing

bridgekeeper bot commented Apr 5, 2021

openjdk bot commented Apr 5, 2021

iklam commented Apr 5, 2021

openjdk bot commented Apr 5, 2021

mlbridge bot commented Apr 5, 2021 • edited

Webrevs

tschatzl left a comment

Choose a reason for hiding this comment

tschatzl Apr 6, 2021

Choose a reason for hiding this comment

iklam Apr 6, 2021

Choose a reason for hiding this comment

tschatzl Apr 6, 2021

Choose a reason for hiding this comment

iklam Apr 6, 2021

Choose a reason for hiding this comment

openjdk bot commented Apr 6, 2021 • edited

calvinccheung Apr 7, 2021

Choose a reason for hiding this comment

calvinccheung left a comment

Choose a reason for hiding this comment

iklam commented Apr 8, 2021

openjdk bot commented Apr 8, 2021

iklam commented Apr 5, 2021 •

edited by openjdk bot

mlbridge bot commented Apr 5, 2021 •

edited

openjdk bot commented Apr 6, 2021 •

edited