Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8254028: G1 incorrectly updates scan_top for collection set regions during preparation of evacuation #557

Closed

Conversation

tschatzl
Copy link
Contributor

@tschatzl tschatzl commented Oct 8, 2020

Hi all,

can I have reviews for this change that makes values of G1RemSetScanState::_scan_top for regions in the initial collection set consistent with ones in optional collection set?
So currently G1RemSetScanState::_scan_top is top() for regions in the initial collection set although they will never be scanned as enforced when dropping the remsets onto the card table. For the optional collection set, G1 sets scan_top manually to NULL when selecting the next few optional regions (using G1RemSet::exclude_region_from_scan()).

When debugging this discrepancy recently posed some slight surprise for me, so I would like to change this. Also there is some risk that future code that relies on that property will be surprised.

Testing: tier1-4; lots of kitchensink runs with JDK-8254164.

Thanks,
Thomas


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Testing

Linux x64 Windows x64 macOS x64
Build / test ✔️ (0/0 passed) ✔️ (0/0 passed) ✔️ (0/0 passed)

Issue

  • JDK-8254028: G1 incorrectly updates scan_top for collection set regions during preparation of evacuation

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/557/head:pull/557
$ git checkout pull/557

@bridgekeeper
Copy link

bridgekeeper bot commented Oct 8, 2020

👋 Welcome back tschatzl! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented Oct 8, 2020

@tschatzl The following label will be automatically applied to this pull request:

  • hotspot-gc

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot-gc hotspot-gc-dev@openjdk.org label Oct 8, 2020
@tschatzl tschatzl marked this pull request as ready for review October 8, 2020 13:18
@openjdk openjdk bot added the rfr Pull request is ready for review label Oct 8, 2020
@mlbridge
Copy link

mlbridge bot commented Oct 8, 2020

Webrevs

Copy link
Contributor

@kstefanj kstefanj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not to fond of the comment saying this is not really needed. After the fix for JDK-8252752 prepare_region_for_scan() don't do anything with regions in the collection-set so I would prefer if we instead filtered those regions at the call-site.

In G1PrepareRegionsClosure::do_heap_region(HeapRegion* hr) we could add:

      // Pepare regions not in the collection set for scanning.
      if (!hr->in_collection_set())  {
        _g1h->rem_set()->prepare_region_for_scan(hr);
      }

This way we don't need to check for collection-set regions in the assert either. What do you think?

@mlbridge
Copy link

mlbridge bot commented Oct 9, 2020

Mailing list message from Thomas Schatzl on hotspot-gc-dev:

Hi,

On 08.10.20 20:48, Stefan Johansson wrote:

On Thu, 8 Oct 2020 08:52:20 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

Hi all,

can I have reviews for this change that makes values of G1RemSetScanState::_scan_top for regions in the initial
collection set consistent with ones in optional collection set?
So currently G1RemSetScanState::_scan_top is top() for regions in the initial collection set although they will never
be scanned as enforced when dropping the remsets onto the card table. For the optional collection set, G1 sets scan_top
manually to NULL when selecting the next few optional regions (using G1RemSet::exclude_region_from_scan()). When
debugging this discrepancy recently posed some slight surprise for me, so I would like to change this. Also there is
some risk that future code that relies on that property will be surprised. Testing: tier1-4; lots of kitchensink runs
with JDK-8254164.
Thanks,
Thomas

I'm not to fond of the comment saying this is not really needed. After the fix for
[JDK-8252752](https://bugs.openjdk.java.net/browse/JDK-8252752) `prepare_region_for_scan()` don't do anything with
regions in the collection-set so I would prefer if we instead filtered those regions at the call-site.

In `G1PrepareRegionsClosure::do_heap_region(HeapRegion* hr)` we could add:
++
// Pepare regions not in the collection set for scanning.
if (!hr->in_collection_set()) {
_g1h->rem_set()->prepare_region_for_scan(hr);
}

This way we don't need to check for collection-set regions in the assert either. What do you think?

The comment can be fixed :) Would something like

// Only non-collection set old regions should be scanned.

be better?

I considered changing the caller as suggested, but this adds additional
(unspoken) requirements on it:

- the caller needs to know that the initialization of the corresponding
scan_top field is NULL (or something to that effect).

- and that this is the only preparation work performed in that method.
(I.e. there might be future things to do)

This seems unnecessary. What do you think?

Thanks,
Thomas

@kstefanj
Copy link
Contributor

kstefanj commented Oct 9, 2020

I considered changing the caller as suggested, but this adds additional
(unspoken) requirements on it:

But it also makes the code clearer showing that we don't do any thing for collection-set regions in this call.

  • the caller needs to know that the initialization of the corresponding
    scan_top field is NULL (or something to that effect).
  • and that this is the only preparation work performed in that method.
    (I.e. there might be future things to do)

But I do agree that it's nice to assert that the collection-set regions are in the correct state. And if we want to prepare for future preparation that also do stuff for the collection-set regions I think we should go back to the old if-else like this:

if (r->in_collection_set()) {
  // Regions in the collection-set should not be scanned.
  assert(_scan_state->scan_top(hrm_index) == NULL,
         "scan_top of region %u is unexpectedly " PTR_FORMAT,
         hrm_index, p2i(_scan_state->scan_top(hrm_index)));
} else if(r->is_old_or_humongous_or_archive()) {
  // Prepare all other regions for scanning.
  _scan_state->set_scan_top(hrm_index, r->top());
} else {
  assert(r->is_free(), "Region %u should be free but is %s",
         hrm_index, r->get_type_str());
} 

I think this satisfies both our wishes, do you agree?

@openjdk
Copy link

openjdk bot commented Oct 10, 2020

@tschatzl This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8254028: G1 incorrectly updates scan_top for collection set regions during preparation of evacuation

Reviewed-by: kbarrett

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 34 new commits pushed to the master branch:

  • 7ec9c8e: 8233214: Remove runtime code not needed with CMS removed
  • 536b35b: 8254319: Shenandoah: Interpreter native-LRB needs to activate during HAS_FORWARDED
  • be26972: 8253379: [windows] Several jpackage tests failed with error code 1638
  • 52e45a3: 8229186: Improve error messages for TestStringIntrinsics failures
  • 6d2c1a6: 8254292: Update JMH devkit to 1.26
  • 2bbf8a2: 8245543: Cgroups: Incorrect detection logic on some systems (still reproducible)
  • aaa0a2a: 8254297: Zero and Minimal VMs are broken with undeclared identifier 'DerivedPointerTable' after JDK-8253180
  • 7e80c98: 8254261: fix javadocs in jdk.test.lib.Utils
  • d4b5dfd: 8253857: Shenandoah: Bugs in ShenandoahEvacOOMHandler related code
  • e9c1905: 8253740: [PPC64] Minor interpreter cleanup
  • ... and 24 more: https://git.openjdk.java.net/jdk/compare/894ec76c11dd627383c0c345ef22e3f7745e3292...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Oct 10, 2020
@tschatzl
Copy link
Contributor Author

/integrate

@mlbridge
Copy link

mlbridge bot commented Oct 12, 2020

Mailing list message from Thomas Schatzl on hotspot-gc-dev:

Hi,

On 10.10.20 00:29, Thomas Schatzl wrote:

Hi all,

can I have reviews for this change that makes values of G1RemSetScanState::_scan_top for regions in the initial
collection set consistent with ones in optional collection set?
So currently G1RemSetScanState::_scan_top is top() for regions in the initial collection set although they will never
be scanned as enforced when dropping the remsets onto the card table. For the optional collection set, G1 sets scan_top
manually to NULL when selecting the next few optional regions (using G1RemSet::exclude_region_from_scan()). When
debugging this discrepancy recently posed some slight surprise for me, so I would like to change this. Also there is
some risk that future code that relies on that property will be surprised. Testing: tier1-4; lots of kitchensink runs
with JDK-8254164.
Thanks,
Thomas

Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision:

sjohanss review

after some chat with Stefan we agreed on above version.

Since this revision has been created in direct conversation with Stefan,
and Kim already approved it, I'll integrate it.

Thanks for your reviews.

Thanks,
Thomas

@openjdk openjdk bot closed this Oct 12, 2020
@openjdk openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Oct 12, 2020
@openjdk
Copy link

openjdk bot commented Oct 12, 2020

@tschatzl Since your change was applied there have been 45 commits pushed to the master branch:

  • a2bb4c6: 8254314: Shenandoah: null checks in c2 should not skip over native load barrier
  • c73a0ff: 8252105: Parallel heap inspection for ZCollectedHeap
  • 45b09a3: 8253833: mutexLocker assert_locked_or_safepoint should not access VMThread state from non-VM-thread
  • 77c7762: 8254353: Remove unused non-product flags
  • d3069ac: 8254362: x86_32 builds fail after JDK-8253180
  • 25001c5: 8254352: 3 compiler tests failed with "assert(allocates2(pc)) failed: not in CodeBuffer memory"
  • d43f141: 8254351: Minimal VM build fails with undeclared identifier 'MaxVectorSize' after JDK-8252847
  • cc52358: 8254335: logging/logStream.hpp includes memory/resourceArea.hpp but doesn't need it
  • 4b5ac3a: 8252847: Optimize primitive arrayCopy stubs using AVX-512 masked instructions
  • ec41046: 8254348: Build fails when cds is disabled after JDK-8247536
  • ... and 35 more: https://git.openjdk.java.net/jdk/compare/894ec76c11dd627383c0c345ef22e3f7745e3292...master

Your commit was automatically rebased without conflicts.

Pushed as commit bf46acf.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@tschatzl tschatzl deleted the 8254028-incorrect-scan-top-update branch October 12, 2020 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-gc hotspot-gc-dev@openjdk.org integrated Pull request has been integrated
3 participants