Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8255019: Shenandoah: Split STW and concurrent mark into separate classes #1009

Closed
wants to merge 37 commits into from

Conversation

zhengyu123
Copy link
Contributor

@zhengyu123 zhengyu123 commented Nov 2, 2020

This is the first part of refactoring, that aims to isolate three Shenandoah GC modes (concurrent, degenerated and full gc).

Shenandoah started with two GC modes, concurrent and full gc, with minimal shared code, mainly in mark phase. After introducing degenerated GC, it shared quite large portion of code with concurrent GC, with the concept that degenerated GC can simply pick up remaining work of concurrent GC in STW mode.

It was not a big problem at that time, since concurrent GC also processed roots STW. Since Shenandoah gradually moved root processing into concurrent phase, code started to diverge, that made code hard to reason and maintain.

First step, I would like to split STW and concurrent mark, so that:

  1. Code has to special case for STW and concurrent mark.
  2. STW mark does not need to rendezvous workers between root mark and the rest of mark
  3. STW mark does not need to activate SATB barrier and drain SATB buffers.
  4. STW mark does not need to remark some of roots.

The patch mainly just shuffles code. Creates a base class ShenandoahMark, and moved shared code (from current shenandoahConcurrentMark) into this base class. I did 'git mv shenandoahConcurrentMark.inline.hpp shenandoahMark.inline.hpp, but git does not seem to reflect that.

A few changes:

  1. Moved task queue set from ShenandoahConcurrentMark to ShenandoahHeap. ShenandoahMark and its subclasses are stateless. Instead, mark states are maintained in task queue, mark bitmap and SATB buffers, so that they can be created on demand.
  2. Split ShenandoahConcurrentRootScanner template to ShenandoahConcurrentRootScanner and ShenandoahSTWRootScanner
  3. Split code inside op_final_mark code into finish_mark and prepare_evacuation helper functions.
  4. Made ShenandoahMarkCompact stack allocated (as well as ShenandoahConcurrentGC and ShenandoahDegeneratedGC in upcoming refactoring)

Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8255019: Shenandoah: Split STW and concurrent mark into separate classes

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/1009/head:pull/1009
$ git checkout pull/1009

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Nov 2, 2020

👋 Welcome back zgu! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr label Nov 2, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Nov 2, 2020

@zhengyu123 The following labels will be automatically applied to this pull request:

  • hotspot-gc
  • shenandoah

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added hotspot-gc shenandoah labels Nov 2, 2020
@mlbridge
Copy link

@mlbridge mlbridge bot commented Nov 2, 2020

@openjdk
Copy link

@openjdk openjdk bot commented Nov 4, 2020

⚠️ @zhengyu123 This pull request contains merges that bring in commits not present in the target repository. Since this is not a "merge style" pull request, these changes will be squashed when this pull request in integrated. If this is your intention, then please ignore this message. If you want to preserve the commit structure, you must change the title of this pull request to Merge <project>:<branch> where <project> is the name of another project in the OpenJDK organization (for example Merge jdk:master).

Copy link
Contributor

@rkennke rkennke left a comment

Overall a great refactoring!
I wonder if we should keep marking state in a single class e.g. ShenandoahMarkingState or even just add it to ShenandoahMarkingContext, instead of scattering it over ShenandoahHeap?

Copy link
Contributor

@rkennke rkennke left a comment

Good work! I have a few questions...

@openjdk
Copy link

@openjdk openjdk bot commented Jan 4, 2021

@zhengyu123 this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout JDK-8255019-sh-mark
git fetch https://git.openjdk.java.net/jdk master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

@openjdk openjdk bot added merge-conflict and removed ready labels Jan 4, 2021
@openjdk openjdk bot added ready and removed merge-conflict labels Jan 5, 2021
Copy link
Contributor

@shipilev shipilev left a comment

First read review follows.

src/hotspot/share/gc/shenandoah/shenandoahMark.inline.hpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/shenandoah/shenandoahMarkCompact.cpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/shenandoah/shenandoahSTWMark.cpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/shenandoah/shenandoahSTWMark.cpp Outdated Show resolved Hide resolved
src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp Outdated Show resolved Hide resolved
if (point == _degenerated_mark) {
finish_mark();
}
Copy link
Contributor

@shipilev shipilev Jan 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if we don't call finish_mark, do we ever call set_concurrent_mark_in_progress(false); and mark_complete_marking_context();?

Copy link
Contributor

@shipilev shipilev Jan 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was not answered, and I don't see relevant follow-up changes. Is this a bug?

Copy link
Contributor Author

@zhengyu123 zhengyu123 Jan 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't think so.

For fall through case, STWMark sets both flags.

For degen case, if we pass _degnerated_mark point, op_final_mark calls finish_mark to set them.

@zhengyu123
Copy link
Contributor Author

@zhengyu123 zhengyu123 commented Jan 13, 2021

@shade, did I address all your concerns? Thanks.

Copy link
Contributor

@shipilev shipilev left a comment

All right, fine, let's do it.

@openjdk openjdk bot added merge-conflict and removed ready labels Jan 14, 2021
@openjdk openjdk bot added ready and removed merge-conflict labels Jan 14, 2021
@zhengyu123
Copy link
Contributor Author

@zhengyu123 zhengyu123 commented Jan 14, 2021

/integrate

@openjdk openjdk bot closed this Jan 14, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Jan 14, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Jan 14, 2021

@zhengyu123 Since your change was applied there have been 7 commits pushed to the master branch:

  • aba3431: 8258956: Memory Leak in StringCoding on ThreadLocal resultCached StringCoding.Result
  • 8554fe6: 8253866: Security Libs Terminology Refresh
  • c2a3c7e: 8259727: Remove redundant "target" arguments to methods in Links
  • 1620664: 8259723: Move Table class to formats.html package
  • 38a1201: 8258912: Remove JVM options CountJNICalls and CountJVMCalls
  • be57cf1: 8226416: MonitorUsedDeflationThreshold can cause repeated async deflation requests
  • c822eda: 8259699: Reduce char[] copying in URLEncoder.encode(String, Charset)

Your commit was automatically rebased without conflicts.

Pushed as commit da6bcf9.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@zhengyu123 zhengyu123 deleted the JDK-8255019-sh-mark branch May 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-gc integrated shenandoah
3 participants