8315044: GenShen: Verifier detects clean card should be dirty #314

kdnilsen · 2023-08-28T23:35:46Z

When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list. This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty. This patch causes the marking to be performed.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed (1 review required, with at least 1 Committer)

Issue

JDK-8315044: GenShen: Verifier detects clean card should be dirty (Bug - P2)

Reviewers

Y. Srinivas Ramakrishna (@ysramakrishna - Committer) ⚠️ Review applies to 2618ff3c
William Kemper (@earthling-amzn - Committer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/shenandoah.git pull/314/head:pull/314
$ git checkout pull/314

Update a local copy of the PR:
$ git checkout pull/314
$ git pull https://git.openjdk.org/shenandoah.git pull/314/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 314

View PR using the GUI difftool:
$ git pr show -t 314

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/shenandoah/pull/314.diff

Webrev

Link to Webrev Comment

kdnilsen · 2023-08-28T23:36:29Z

I'm going to leave this in draft mode until I see and analyze results of integration testing.

bridgekeeper · 2023-08-28T23:36:31Z

👋 Welcome back kdnilsen! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

mlbridge · 2023-08-29T17:30:01Z

Webrevs

…es-remembered-set

ysramakrishna

changes look good; minor suggestion on some potential refactor/consolidation with existing card marking code.

ysramakrishna · 2023-09-01T00:34:58Z

src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp

+      T* addr = reinterpret_cast<T*>(java_lang_ref_Reference::discovered_addr_raw(reference));
+      if (heap->is_in_old(addr) && heap->is_in_young(discovered_head)) {
+        heap->mark_card_as_dirty(addr);


Can you instead call the existing card_mark_barrier method?
At that time, could also look at the existing uses of that method and refactor them to use the ShenandoahCardBarrier test like you did above, and elide the generational check in the body of the method itself, replacing it with an assertion (if you want, that the heap is generational).

(PS: I was also thinking that in the fullness of time it might be worth doing an audit of the uses of cas and set_oop and making sure they would call the card-marking code instead of exposing the functionality at the call-sites like done here. Callers that want to avoid card-marking would need to call the respective _raw versions then. We should makr sure to conform to naming conventions for those methods that are consistent with those used in other parts of jvm/gc code. This can be done separately and might affect legacy Shenandoah code as well. Just leaving this comment here so as not to lose the thought, but I doubt we want to do that locally here in this PR.)

Made this change and am testing result.

I agree that audit/refactoring in the future would be a good idea.

I've made your suggested changes and am sending through the test pipeline again.

I agree that it would be good to eventually audit/refactor to have more consistent method names and conventions.

ysramakrishna

One more change, I think? See comment in-line.

May be there is an issue with it though that I am not seeing which perhaps you might have tried already, but doesn't work?

Rest looks good.

ysramakrishna · 2023-09-01T21:21:40Z

src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp

  if (UseCompressedOops) {
-    *reinterpret_cast<narrowOop*>(_pending_list_tail) = CompressedOops::encode(Universe::swap_reference_pending_list(_pending_list));
+    *reinterpret_cast<narrowOop*>(_pending_list_tail) = CompressedOops::encode(former_head_of_global_list);
  } else {
-    *reinterpret_cast<oop*>(_pending_list_tail) = Universe::swap_reference_pending_list(_pending_list);
+    *reinterpret_cast<oop*>(_pending_list_tail) = former_head_of_global_list;
+  }
+  ShenandoahHeap* heap = ShenandoahHeap::heap();
+  if (ShenandoahCardBarrier) {
+    if (heap->is_in_old(_pending_list_tail) && heap->is_in_young(former_head_of_global_list)) {
+      heap->mark_card_as_dirty(_pending_list_tail);
+    }


Can we just replace this section with set_oop_field(_pending_list_tail, former_head_of_global_list) ?

Have made this improvement, and am retesting.

openjdk · 2023-09-01T21:23:48Z

@kdnilsen This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8315044: GenShen: Verifier detects clean card should be dirty

Reviewed-by: ysr, wkemper

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

…es-remembered-set

kdnilsen · 2023-09-06T15:40:34Z

/integrate

openjdk · 2023-09-06T15:43:13Z

Going to push as commit c4d0a24.

openjdk · 2023-09-06T15:43:22Z

@kdnilsen Pushed as commit c4d0a24.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Mark card as dirty if discovered reference list has interesting pointer

e4cd228

kdnilsen marked this pull request as draft August 28, 2023 23:35

openjdk bot changed the title ~~JDK-8315044: GenShen: Verifier detects clean card should be dirt~~ 8315044: GenShen: Verifier detects clean card should be dirty Aug 28, 2023

Fix whitespace

dc95476

kdnilsen marked this pull request as ready for review August 29, 2023 17:24

openjdk bot added the rfr Pull request is ready for review label Aug 29, 2023

kdnilsen added 4 commits August 29, 2023 20:24

Also update card table when moving discovered References to global list

8b898a5

Merge remote-tracking branch 'origin/master' into ref-processor-updat…

597a6e0

…es-remembered-set

Merge remote-tracking branch 'origin/master' into ref-processor-updat…

5e801c9

…es-remembered-set

Replace is_generational with ShenandoahCardBarrier test

f375d57

ysramakrishna reviewed Sep 1, 2023

View reviewed changes

Use card_mark_barrier() method and ShenandoahCardBarrier tests

2618ff3

ysramakrishna approved these changes Sep 1, 2023

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Sep 1, 2023

kdnilsen added 2 commits September 5, 2023 18:59

Better abstraction from reviewer feedback

79349d3

Merge remote-tracking branch 'origin/master' into ref-processor-updat…

e9db121

…es-remembered-set

earthling-amzn approved these changes Sep 6, 2023

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Sep 6, 2023

openjdk bot closed this Sep 6, 2023

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Sep 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8315044: GenShen: Verifier detects clean card should be dirty #314

8315044: GenShen: Verifier detects clean card should be dirty #314

kdnilsen commented Aug 28, 2023 •

edited by openjdk bot

Loading

kdnilsen commented Aug 28, 2023

bridgekeeper bot commented Aug 28, 2023

mlbridge bot commented Aug 29, 2023 •

edited

Loading

ysramakrishna left a comment

ysramakrishna Sep 1, 2023

kdnilsen Sep 1, 2023

kdnilsen Sep 1, 2023

ysramakrishna left a comment

ysramakrishna Sep 1, 2023

kdnilsen Sep 5, 2023

openjdk bot commented Sep 1, 2023 •

edited

Loading

kdnilsen commented Sep 6, 2023

openjdk bot commented Sep 6, 2023

openjdk bot commented Sep 6, 2023

8315044: GenShen: Verifier detects clean card should be dirty #314

8315044: GenShen: Verifier detects clean card should be dirty #314

Conversation

kdnilsen commented Aug 28, 2023 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

Webrev

kdnilsen commented Aug 28, 2023

bridgekeeper bot commented Aug 28, 2023

mlbridge bot commented Aug 29, 2023 • edited Loading

Webrevs

ysramakrishna left a comment

Choose a reason for hiding this comment

ysramakrishna Sep 1, 2023

Choose a reason for hiding this comment

kdnilsen Sep 1, 2023

Choose a reason for hiding this comment

kdnilsen Sep 1, 2023

Choose a reason for hiding this comment

ysramakrishna left a comment

Choose a reason for hiding this comment

ysramakrishna Sep 1, 2023

Choose a reason for hiding this comment

kdnilsen Sep 5, 2023

Choose a reason for hiding this comment

openjdk bot commented Sep 1, 2023 • edited Loading

kdnilsen commented Sep 6, 2023

openjdk bot commented Sep 6, 2023

openjdk bot commented Sep 6, 2023

kdnilsen commented Aug 28, 2023 •

edited by openjdk bot

Loading

mlbridge bot commented Aug 29, 2023 •

edited

Loading

openjdk bot commented Sep 1, 2023 •

edited

Loading