Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8253600: G1: Fully support pinned regions for full gc #824

Closed

Conversation

@tschatzl
Copy link
Contributor

@tschatzl tschatzl commented Oct 23, 2020

Hi all,

can I get reviews for this change that implements "proper" support for pinned regions in the G1 full collector?

By proper I mean that at the end of gc, pinned regions contain the correct TAMS and bitmap markings under the TAMS so that dead objects within them are supported?

Currently all (pinned) regions have their TAMS set to bottom() and their bitmap above TAMS cleared (at least logically :) ). This works as long objects within these regions can't be dead as it is the case now:

  • humongous regions are either live or fully reclaimed.
  • all other pinned regions are archive regions at the moment that are always treated as fully live (and do not contain dead objects).

This change is a requirement for fixing JDK-8253081 as some earlier change made it possible to have dead objects within open archive regions. It also enables supporting removal of gclocker use for g1, i.e. using region pinning.

Based on the PR#808 (#808).

Testing: tier1-8, testing with prototype for region pinning, testing with prototype for JDK-8253081.
Performance testing: no regressions

Some comments for questions that might come up during review:

  • how does this work with the bitmaps now:
    • at start of full gc the next bitmap is cleared
    • full gc marks the next bitmap
    • for all pinned regions, keep TAMS and top() (*), otherwise set TAMS to bottom
    • swap bitmaps
    • clear next bitmap for next marking

(*) this means that from a usage POV pinned regions are considered full. This is inaccurate, but sufficient: full gc clears all remembered sets anyway, so we do not need that information for gc efficiency purposes anyway to evacuate later. The next marking before old gen evacuation will update it to the correct values anyway. G1 does not support allocation into "holes" in pinned regions that can be open archive only at this time too, so there is no need to be more exact.

  • use of a region attribute table for phase 2+ only: compared to before we need fast access to information whether a given reference goes into a pinned region (as opposed to an archive region) wrt to adjusting that pointer to avoid doing work for these references.

Phase 1 marking could have used this information for the do-we-need-to-preserve-the-mark check too: however this would have required g1 to add an extra another pass over all regions to update that. This seemed slower than just checking this information "more slowly" for the objects that need mark preservation. Tests showed that this is the case for <0.00% (yeah, these references that need mark preservation are rounding errors in cases it matters) of overall references, so I did not add that pass.
(Additionally g1 full gc is a last-ditch effort, and while marking takes a significant time, it does not completely dominate it).

I.e. the second clause in the condition of this hunk is intentionally slower than could be:

@@ -52,7 +52,9 @@ inline bool G1FullGCMarker::mark_object(oop obj) {
  // Marked by us, preserve if needed.
  markWord mark = obj->mark();
  if (obj->mark_must_be_preserved(mark) &&
      // It is not necessary to preserve marks for objects in pinned regions because
      // we do not change their headers (i.e. forward them).
      !G1CollectedHeap::heap()->heap_region_containing(obj)->is_pinned()) {
    preserved_stack()->push(obj, mark);
  }
  • there is no code yet that checks for empty pinned regions yet. Only JDK-8253081 introduces that because still all contents of all archive regions are live forever.

Also please note that the 51b297b change is from the #808 change.

Thanks,
Thomas


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8253600: G1: Fully support pinned regions for full gc

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/824/head:pull/824
$ git checkout pull/824

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Oct 23, 2020

👋 Welcome back tschatzl! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Oct 23, 2020

@tschatzl The following label will be automatically applied to this pull request:

  • hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot label Oct 23, 2020
@tschatzl tschatzl marked this pull request as ready for review Oct 23, 2020
@openjdk openjdk bot added the rfr label Oct 23, 2020
@mlbridge
Copy link

@mlbridge mlbridge bot commented Oct 23, 2020

Webrevs

Copy link
Contributor

@kstefanj kstefanj left a comment

Nice change Thomas, mostly just small comments.

src/hotspot/share/gc/g1/g1FullGCAdjustTask.cpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/g1/g1FullGCAdjustTask.cpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/g1/g1FullGCMarker.hpp Show resolved Hide resolved
src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/g1/heapRegion.hpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/g1/g1FullCollector.hpp Outdated Show resolved Hide resolved
Also remove _archive_allocator_map et al as the new attribute table
implements the same functionality also suggested by sjohanss in
private.
@openjdk
Copy link

@openjdk openjdk bot commented Oct 29, 2020

@tschatzl this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout 8253600-full-gc-pinned-region-support
git fetch https://git.openjdk.java.net/jdk master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

@openjdk openjdk bot removed the merge-conflict label Oct 29, 2020
@tschatzl
Copy link
Contributor Author

@tschatzl tschatzl commented Oct 29, 2020

The new change (apart from the merge) should fix all the concerns @kstefanj mentioned.

In addition to that I looked into completely moving the G1ArchiveAllocator::_archive_region_map into the G1FullCollector::_region_attr_table as they serve the same purpose.

Other than in full gc, G1ArchiveAllocator::_archive_region_map is only used in non-perf critical code (object dumping, asserts), so the replacements should be more than fast enough.

The only drawback is that this adds a new method in CollectedHeap as a start for CDS archive support for other collectors which is thought of. More is needed, but that's the minimum to replace functionality previously provided by G1ArchiveAllocator::_archive_region_map at this time.

Its default implementation simply returns false if asked whether a given object on the heap is an archive object.

virtual bool is_archived_object(oop object) const { return false; }

Copy link
Contributor

@kstefanj kstefanj left a comment

Thanks Thomas for addressing my concerns around the archive-map and the new region-attr overlap. I think this looks much better.

Just a few additional comments.

src/hotspot/share/gc/g1/g1FullCollector.cpp Show resolved Hide resolved
src/hotspot/share/gc/g1/g1FullGCMarker.hpp Show resolved Hide resolved
src/hotspot/share/gc/g1/g1FullGCMarker.inline.hpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/shared/collectedHeap.hpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/g1/g1FullCollector.hpp Outdated Show resolved Hide resolved
@tschatzl
Copy link
Contributor Author

@tschatzl tschatzl commented Nov 3, 2020

  • I'll file a CR for that investigation. In my recent measurements of a different full-iterate-all-regions closure we are talking about something < 0.0x ms.
  • added an is_in_pinned method, and since now we've got three not-completely-trivial methods added a .inline.hpp file. I did not see need for moving the completely trivial getters returning just a member variable there, but tell me if I should move them too.
  • moved "implementation" of CollectedHeap::is_archived_object to .cpp file

Copy link
Contributor

@kstefanj kstefanj left a comment

This looks great, thanks.

@openjdk
Copy link

@openjdk openjdk bot commented Nov 3, 2020

@tschatzl This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8253600: G1: Fully support pinned regions for full gc

Reviewed-by: sjohanss, ayang

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 7 new commits pushed to the master branch:

  • 97d6e4a: 8256046: Shenandoah: Mix-in NULL_PTR in non-strong ShLRBNode's type
  • a1d4b9f: 8256009: Remove src/hotspot/share/adlc/Test/i486.ad
  • 3455fa9: 8256050: JVM crashes with -XX:+PrintDeoptimizationDetails
  • e6df13e: 8256054: C2: Floating-point min/max operations on vectors intermittently produce wrong results for NaN values
  • 52805f5: 8256048: Incomplete gitignore setting for netbeans project
  • 9d07259: 8255598: [PPC64] assert(Universe::heap()->is_in(result)) failed: object not in heap
  • e281b13: 8255011: [TESTBUG] compiler/codecache/stress/UnexpectedDeoptimizationAllTest.java timed out

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Nov 3, 2020
Copy link
Member

@albertnetymk albertnetymk left a comment

Thank you for the code walk-through. Some further questions/comments.

src/hotspot/share/gc/g1/g1FullGCHeapRegionAttr.hpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/g1/g1FullCollector.hpp Show resolved Hide resolved
Copy link
Member

@albertnetymk albertnetymk left a comment

G1FullGCHeapRegionAttr::is_pinned and HeapRegion::is_pinned share the same name, but have different meanings. I wonder if it's possible to rename the former into sth more precise. Maybe G1FullGCHeapRegionAttr::should_not_be_relocated, or the version without negation G1FullGCHeapRegionAttr::should_relocate?

Copy link
Member

@albertnetymk albertnetymk left a comment

Thanks to Thomas' offline explanation, my previous comment is incorrect, due to my erroneous understanding of the enum in HeapRegionType.

Thank you for new updates. This PR is good to go.

@tschatzl
Copy link
Contributor Author

@tschatzl tschatzl commented Nov 10, 2020

Thanks @kstefanj @albertnetymk for your reviews.

/integrate

@openjdk openjdk bot closed this Nov 10, 2020
@openjdk openjdk bot added integrated and removed ready rfr labels Nov 10, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Nov 10, 2020

@tschatzl Since your change was applied there have been 7 commits pushed to the master branch:

  • 97d6e4a: 8256046: Shenandoah: Mix-in NULL_PTR in non-strong ShLRBNode's type
  • a1d4b9f: 8256009: Remove src/hotspot/share/adlc/Test/i486.ad
  • 3455fa9: 8256050: JVM crashes with -XX:+PrintDeoptimizationDetails
  • e6df13e: 8256054: C2: Floating-point min/max operations on vectors intermittently produce wrong results for NaN values
  • 52805f5: 8256048: Incomplete gitignore setting for netbeans project
  • 9d07259: 8255598: [PPC64] assert(Universe::heap()->is_in(result)) failed: object not in heap
  • e281b13: 8255011: [TESTBUG] compiler/codecache/stress/UnexpectedDeoptimizationAllTest.java timed out

Your commit was automatically rebased without conflicts.

Pushed as commit 6555996.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

openjdk-notifier bot referenced this issue Nov 10, 2020
@tschatzl tschatzl deleted the 8253600-full-gc-pinned-region-support branch Nov 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants