8331048: G1: Prune rebuild candidates based on G1HeapWastePercent early #18970

walulyai · 2024-04-26T10:07:55Z

Hi,

Please review this change that moves the G1CollectionSetChooser::build out of the cleanup pause into the remark pause. This prunes the rebuild candidates based on G1HeapWastePercent, thereby reducing the rebuild time in cases where pruning happens.

Testing: Tier 1 - 5

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8331048: G1: Prune rebuild candidates based on G1HeapWastePercent early (Enhancement - P4)

Reviewers

Albert Mingkun Yang (@albertnetymk - Reviewer)
Thomas Schatzl (@tschatzl - Reviewer) ⚠️ Review applies to a42cfbd1

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/18970/head:pull/18970
$ git checkout pull/18970

Update a local copy of the PR:
$ git checkout pull/18970
$ git pull https://git.openjdk.org/jdk.git pull/18970/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 18970

View PR using the GUI difftool:
$ git pr show -t 18970

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/18970.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2024-04-26T10:08:46Z

👋 Welcome back iwalulya! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2024-04-26T10:10:38Z

@walulyai This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8331048: G1: Prune rebuild candidates based on G1HeapWastePercent early

Reviewed-by: ayang, tschatzl

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

openjdk · 2024-04-26T10:11:25Z

@walulyai The following label will be automatically applied to this pull request:

hotspot-gc

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2024-04-26T11:04:56Z

Webrevs

tschatzl · 2024-04-26T11:09:19Z

src/hotspot/share/gc/g1/g1ConcurrentMark.cpp

+    if (_needs_remembered_set_rebuild) {
+      // Prune rebuild candidates based on G1HeapWastePercent.
+      // Improves rebuild time in addition to remembered set memory usage.
+      G1CollectionSetChooser::build(_g1h->workers(), _g1h->num_regions(), _g1h->policy()->candidates());
+    }
+


I would prefer to put this block into above block so that it is part of the "Select For Rebuild and Reclaim Empty Regions" measurement.

tschatzl · 2024-04-26T11:53:22Z

src/hotspot/share/gc/g1/g1ConcurrentMark.cpp

+    if (_needs_remembered_set_rebuild) {
+      // Prune rebuild candidates based on G1HeapWastePercent.
+      // Improves rebuild time in addition to remembered set memory usage.
+      G1CollectionSetChooser::build(_g1h->workers(), _g1h->num_regions(), _g1h->policy()->candidates());


The build method sorts according to gc efficiency and prunes.

GC efficiency can not be properly determined at this point and is entirely based on occupancy (remsets are empty); so basically what this does is sort and prune by live objects.

The whole gc-efficiency calculation can then be removed.

Actually I am not sure making the efficiency only based on free space is problematic, however we shouldn't pretend we care then.

An alternative I tried long time ago has been counting incoming references during marking as a substitute for the remset size metric. However in reality, the remembered set is a metric based on cards, so it will potentially be hard to assign cost for remembered set processing based on incoming references.

tschatzl · 2024-05-02T14:12:37Z

src/hotspot/share/gc/g1/g1CollectionSetCandidates.cpp

@@ -95,6 +95,21 @@ void G1CollectionCandidateList::verify() {
 #endif

 int G1CollectionCandidateList::compare(G1CollectionSetCandidateInfo* ci1, G1CollectionSetCandidateInfo* ci2) {


I would prefer if the old comparison function would be renamed as well similarly to the other, e.g. compare_by_efficiencies.

tschatzl · 2024-05-02T14:13:18Z

src/hotspot/share/gc/g1/g1CollectionSetCandidates.cpp

+  if (reclaimable1 == reclaimable2) return 0;
+  if (reclaimable1 > reclaimable2) return -1;
+  return 1;


Please use the same style for this if-elseif-else cascade than for the other one.

tschatzl · 2024-05-02T14:17:55Z

src/hotspot/share/gc/g1/g1CollectionSetCandidates.cpp

+    HeapRegion* hr = (*iter)->_r;
+    (*iter)->_gc_efficiency = hr->calc_gc_efficiency();
+  }
+  _marking_regions.sort_by_efficiency();


It is a bit surprising that this method also sorts by efficiency. I would almost say that the sorting is more important than the calculations.
I.e. something like sort_by_gc_efficiency seems better.

tschatzl

Thanks for the changes. Lgtm.

albertnetymk

Now that pruning occurs in remark, I wonder if those pruned regions should/can switch to remset-untracked instead.

albertnetymk · 2024-05-03T08:44:39Z

src/hotspot/share/gc/g1/g1CollectionSetCandidates.cpp

@@ -199,7 +225,7 @@ void G1CollectionSetCandidates::set_candidates_from_marking(G1CollectionSetCandi
  verify();
 }

-void G1CollectionSetCandidates::sort_by_efficiency() {
+void G1CollectionSetCandidates::sort_retained_by_efficiency() {


Why is this renaming? Checking its callers, I feel the postcondition is that both kinds of candidates are sorted -- marking-candidates can be skipped seems an impl detail.

Let me revert this!

albertnetymk · 2024-05-03T08:45:30Z

src/hotspot/share/gc/g1/g1CollectionSetCandidates.hpp

@@ -129,7 +129,9 @@ class G1CollectionCandidateList : public CHeapObj<mtGC> {
  // Comparison function to order regions in decreasing GC efficiency order. This
  // will cause regions with a lot of live objects and large remembered sets to end
  // up at the end of the list.
-  static int compare(G1CollectionSetCandidateInfo* ci1, G1CollectionSetCandidateInfo* ci2);
+  static int compare_gc_efficiency(G1CollectionSetCandidateInfo* ci1, G1CollectionSetCandidateInfo* ci2);


Preexisting: I wonder if these can be static functions in cpp file, i.e. not in hpp at all.

Possible, but then they end up in different .cpp files, not sure if that is cleaner

walulyai · 2024-05-03T08:59:44Z

Now that pruning occurs in remark, I wonder if those pruned regions should/can switch to remset-untracked instead.

iiuc, the rem_set()->clear() does that

walulyai · 2024-05-03T12:34:58Z

Thanks @albertnetymk and @tschatzl for your reviews!

/integrate

openjdk · 2024-05-03T12:36:00Z

Going to push as commit ce73fec.
Since your change was applied there have been 7 commits pushed to the master branch:

58ef9e4: 8331402: G1: Remove is_active() calls in G1HRPrinter logging
3c77dad: 8331507: JFR: Improve example usage in -XX:StartFlightRecording:help
8ed3190: 8331401: G1: Make G1HRPrinter AllStatic
1f6d38f: 8294978: Convert 5 test/jdk/jdk/jfr tests from ASM library to Classfile API
c60474b: 8323707: Adjust Classfile API's type arg model to better represent the embodied type
f78fa05: 8331636: [BACKOUT] Build failure after 8330076
f665e07: 8331540: [BACKOUT] NMT: add/make a mandatory MEMFLAGS argument to family of os::reserve/commit/uncommit memory API

Your commit was automatically rebased without conflicts.

openjdk · 2024-05-03T12:36:07Z

@walulyai Pushed as commit ce73fec.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

walulyai added 8 commits March 15, 2024 12:54

save

2d14e65

clean up

9316196

save

1e50c67

Merge branch 'master' into GroupOldRegions_Remsets

e6ae6e4

cleanup

8002fab

remove Fixme

cb24e9c

Merge branch 'master' into PruneBeforeRebuild

0fed55f

remove methods

a33095c

openjdk bot added the hotspot-gc hotspot-gc-dev@openjdk.org label Apr 26, 2024

walulyai marked this pull request as ready for review April 26, 2024 10:59

openjdk bot added the rfr Pull request is ready for review label Apr 26, 2024

albertnetymk approved these changes Apr 26, 2024

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Apr 26, 2024

tschatzl suggested changes Apr 26, 2024

View reviewed changes

Thomas Review

e67fe51

walulyai requested a review from albertnetymk May 2, 2024 11:17

tschatzl suggested changes May 2, 2024

View reviewed changes

Refactor cleanup

a95fd31

tschatzl approved these changes May 3, 2024

View reviewed changes

Merge remote-tracking branch 'upstream/master' into PruneBeforeRebuild

a42cfbd

albertnetymk reviewed May 3, 2024

View reviewed changes

walulyai added 2 commits May 3, 2024 11:13

revert method name

e52ce7c

Merge remote-tracking branch 'upstream/master' into PruneBeforeRebuild

ddc3df6

albertnetymk approved these changes May 3, 2024

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label May 3, 2024

openjdk bot closed this May 3, 2024

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels May 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8331048: G1: Prune rebuild candidates based on G1HeapWastePercent early #18970

8331048: G1: Prune rebuild candidates based on G1HeapWastePercent early #18970

walulyai commented Apr 26, 2024 •

edited by openjdk bot

Loading

bridgekeeper bot commented Apr 26, 2024

openjdk bot commented Apr 26, 2024 •

edited

Loading

openjdk bot commented Apr 26, 2024

mlbridge bot commented Apr 26, 2024 •

edited

Loading

tschatzl Apr 26, 2024

tschatzl Apr 26, 2024

tschatzl May 2, 2024

tschatzl May 2, 2024

tschatzl May 2, 2024

tschatzl left a comment

albertnetymk left a comment

albertnetymk May 3, 2024

walulyai May 3, 2024

albertnetymk May 3, 2024

walulyai May 3, 2024

walulyai commented May 3, 2024

walulyai commented May 3, 2024

openjdk bot commented May 3, 2024

openjdk bot commented May 3, 2024

		@@ -95,6 +95,21 @@ void G1CollectionCandidateList::verify() {
		#endif

		int G1CollectionCandidateList::compare(G1CollectionSetCandidateInfo* ci1, G1CollectionSetCandidateInfo* ci2) {

8331048: G1: Prune rebuild candidates based on G1HeapWastePercent early #18970

8331048: G1: Prune rebuild candidates based on G1HeapWastePercent early #18970

Conversation

walulyai commented Apr 26, 2024 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

Webrev

bridgekeeper bot commented Apr 26, 2024

openjdk bot commented Apr 26, 2024 • edited Loading

openjdk bot commented Apr 26, 2024

mlbridge bot commented Apr 26, 2024 • edited Loading

Webrevs

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tschatzl left a comment

Choose a reason for hiding this comment

albertnetymk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

walulyai commented May 3, 2024

walulyai commented May 3, 2024

openjdk bot commented May 3, 2024

openjdk bot commented May 3, 2024

walulyai commented Apr 26, 2024 •

edited by openjdk bot

Loading

openjdk bot commented Apr 26, 2024 •

edited

Loading

mlbridge bot commented Apr 26, 2024 •

edited

Loading