8293291: Simplify relocation of native pointers in archive heap #10296

iklam · 2022-09-15T20:49:24Z

Some objects in the archive heap contain native pointers. E.g., archived java.lang.Class objects contain a pointer to its Klass*.

At runtime, if the archived metadata are mapped at an alternative address, all of the Klass* pointers must be relocated. Instead of doing this in an ad-hoc way, this PR uses a bitmap, FileMapRegion::ptrmap_view(), to track the position of all the native pointers.

This PR is done in preparation for supporting new types of archived heap objects that have native pointers. E.g., java.lang.invoke.ResolvedMethodName has a pointer to a Method*.

Notes for reviewers:

At dump time, the native pointers are remembered in HeapShared::mark_native_pointers().
At runtime, the native pointers are patched in ArchiveHeapLoader::patch_native_pointers().
The ad-hoc relocation code has been removed from javaClasses.cpp and systemDictionaryShared.cpp
Many of the changes are for renaming from "oopmaps" to "bitmaps": the old code handled only a single type of pointers (oop references). The new code handles two types of bitmaps ("oopmap" is for oop references, "ptrmap" is for native pointers).

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8293291: Simplify relocation of native pointers in archive heap

Reviewers

Calvin Cheung (@calvinccheung - Reviewer) ⚠️ Review applies to 810289f7
Coleen Phillimore (@coleenp - Reviewer) ⚠️ Review applies to 810289f7

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/10296/head:pull/10296
$ git checkout pull/10296

Update a local copy of the PR:
$ git checkout pull/10296
$ git pull https://git.openjdk.org/jdk pull/10296/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 10296

View PR using the GUI difftool:
$ git pr show -t 10296

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/10296.diff

bridgekeeper · 2022-09-15T20:50:01Z

👋 Welcome back iklam! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2022-09-15T20:52:44Z

@iklam The following label will be automatically applied to this pull request:

hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2022-09-15T21:14:21Z

Webrevs

calvinccheung

I've a few minor comments. Thanks.

calvinccheung · 2022-09-16T22:38:15Z

src/hotspot/share/include/cds.h

@@ -55,8 +55,12 @@ typedef struct CDSFileMapRegion {
                              // - for heap regions, the base address is the compressed oop encoding base
  size_t  _used;              // Number of bytes actually used by this region (excluding padding bytes added
                              // for alignment purposed.
-  size_t  _oopmap_offset;     // Bitmap for relocating embedded oops (offset from SharedBaseAddress).
+  size_t  _oopmap_offset;     // Bitmap for relocating oop fields in archived heap objects.
+                              // (The base address is the botom of the BM region)


typo: botom -> bottom

calvinccheung · 2022-09-16T22:38:23Z

src/hotspot/share/include/cds.h

  size_t  _oopmap_size_in_bits;
+  size_t  _ptrmap_offset;     // Bitmap for relocating native pointer fields in archived heap objects.
+                              // (The base address is the botom of the BM region).


typo: botom -> bottom

calvinccheung · 2022-09-16T22:39:29Z

src/hotspot/share/cds/heapShared.cpp

+    }
+  }
+
+  log_info(cds, heap)("calculate_ptrmap: marked %d non-null native pointers our of "


typo: our -> out

calvinccheung · 2022-09-16T22:41:55Z

src/hotspot/share/cds/filemap.cpp

+      int oopmap_idx = i * 2;
+      int ptrmap_idx = i * 2 + 1;
+      space_at(region_idx)->init_bitmaps(bitmaps->at(i*2), bitmaps->at(i*2 + 1));


Please add some comment on oopmap_idx and ptrmap_idx.

Line 1693 could be:
space_at(region_idx)->init_bitmaps(bitmaps->at(oopmap_idx), bitmaps->at(ptrmap_idx));

calvinccheung · 2022-09-16T22:44:14Z

src/hotspot/share/cds/metaspaceShared.cpp

  for (int i=0; i<regions->length(); i++) {
-    ResourceBitMap oopmap = HeapShared::calculate_oopmap(regions->at(i));


Pre-existing: line 908.
While you're touching this code, please add a space before and after the '=' and '<'.

calvinccheung

Updates look good.

openjdk · 2022-10-13T04:43:09Z

@iklam This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8293291: Simplify relocation of native pointers in archive heap

Reviewed-by: ccheung, coleenp

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

…elocation

…ap is not supported there

coleenp

This makes sense and does seem cleaner.

…simplify-archive-heap-native-ptr-relocation

iklam · 2022-10-19T04:56:22Z

Thanks @calvinccheung and @coleenp for the review. I have merged with latest repo (after CPU sync) and retested with tiers 1-4. All passed.
/integrate

openjdk · 2022-10-19T04:58:08Z

Going to push as commit 3f4964f.

openjdk · 2022-10-19T04:58:26Z

@iklam Pushed as commit 3f4964f.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8293291: Simplify relocation of native pointers in archive heap

a112adf

openjdk bot added the hotspot-runtime hotspot-runtime-dev@openjdk.org label Sep 15, 2022

clean up

4eee596

iklam marked this pull request as ready for review September 15, 2022 21:07

openjdk bot added the rfr Pull request is ready for review label Sep 15, 2022

calvinccheung reviewed Sep 16, 2022

View reviewed changes

@calvinccheung comments

24b905b

calvinccheung approved these changes Oct 13, 2022

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Oct 13, 2022

iklam added 5 commits October 13, 2022 10:50

Merge branch 'master' into 8293291-simplify-archive-heap-native-ptr-r…

5df9757

…elocation

Merge branch 'master' into 8293291-simplify-archive-heap-native-ptr-r…

d2e969b

…elocation

Merge branch 'master' into 8293291-simplify-archive-heap-native-ptr-r…

dc5d591

…elocation

cleaned up code; fixed test that failed on windows because archive he…

54f78f5

…ap is not supported there

Fixed windows test failure

810289f

calvinccheung approved these changes Oct 18, 2022

View reviewed changes

coleenp approved these changes Oct 19, 2022

View reviewed changes

Merge branch 'master' of https://github.com/openjdk/jdk into 8293291-…

a39ca65

…simplify-archive-heap-native-ptr-relocation

openjdk bot added the integrated Pull request has been integrated label Oct 19, 2022

openjdk bot closed this Oct 19, 2022

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Oct 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8293291: Simplify relocation of native pointers in archive heap #10296

8293291: Simplify relocation of native pointers in archive heap #10296

iklam commented Sep 15, 2022 •

edited by openjdk bot

Loading

bridgekeeper bot commented Sep 15, 2022

openjdk bot commented Sep 15, 2022

mlbridge bot commented Sep 15, 2022 •

edited

Loading

calvinccheung left a comment

calvinccheung Sep 16, 2022

calvinccheung Sep 16, 2022

calvinccheung Sep 16, 2022

calvinccheung Sep 16, 2022

calvinccheung Sep 16, 2022

calvinccheung left a comment

openjdk bot commented Oct 13, 2022 •

edited

Loading

coleenp left a comment

iklam commented Oct 19, 2022

openjdk bot commented Oct 19, 2022

openjdk bot commented Oct 19, 2022

		for (int i=0; i<regions->length(); i++) {
		ResourceBitMap oopmap = HeapShared::calculate_oopmap(regions->at(i));

8293291: Simplify relocation of native pointers in archive heap #10296

8293291: Simplify relocation of native pointers in archive heap #10296

Conversation

iklam commented Sep 15, 2022 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

bridgekeeper bot commented Sep 15, 2022

openjdk bot commented Sep 15, 2022

mlbridge bot commented Sep 15, 2022 • edited Loading

Webrevs

calvinccheung left a comment

Choose a reason for hiding this comment

calvinccheung Sep 16, 2022

Choose a reason for hiding this comment

calvinccheung Sep 16, 2022

Choose a reason for hiding this comment

calvinccheung Sep 16, 2022

Choose a reason for hiding this comment

calvinccheung Sep 16, 2022

Choose a reason for hiding this comment

calvinccheung Sep 16, 2022

Choose a reason for hiding this comment

calvinccheung left a comment

Choose a reason for hiding this comment

openjdk bot commented Oct 13, 2022 • edited Loading

coleenp left a comment

Choose a reason for hiding this comment

iklam commented Oct 19, 2022

openjdk bot commented Oct 19, 2022

openjdk bot commented Oct 19, 2022

iklam commented Sep 15, 2022 •

edited by openjdk bot

Loading

mlbridge bot commented Sep 15, 2022 •

edited

Loading

openjdk bot commented Oct 13, 2022 •

edited

Loading