Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8254315: Shenandoah: Concurrent weak reference processing #505

Closed
wants to merge 91 commits into from

Conversation

rkennke
Copy link
Contributor

@rkennke rkennke commented Oct 5, 2020

Until now, references (as in java.lang.ref.Reference and its subclasses WeakReference, SoftReference, PhantomReference and the non-public FinalReference - I'll collectively call them weak references for the purpose of clarity). Workloads that make heavvy use of such weak references will therefore potentially cause significant GC pauses.

There are 3 main items that contribute to pause time linear to number of references, or worse:

  • We need to scan and consider each reference on the various 'discovered' lists.
  • We need to mark through subgraph of objects that are reachable only through FinalReference. Notice that this is theoretically only bounded by the live data set size.
  • Finally, all no-longer-reachable references need to be enqueued in the 'pending list'

The problem is somewhat mitigated by pre-cleaning the discovered list: Any weak reference that we find to be strongly reachable will be removed before we go into the final-mark-pause. However, that is only a band-aid.

The solution to this is two-fold:

  1. Extend concurrent marking to also mark the 'finalizable' subgraph of the heap. This requires to extend the marking bitmap to allow for two kinds of reachability: each object can now be strongly and finalizably reachable. Whenever marking encounters a FinalReference, it will mark through the referent and switch to 'finalizably' reachability for all objects starting from the referent. When marking encounters finalizably reachable objects while marking strongly, it will 'upgrade' reachability of such objects to strongly reachable. All of this can be done concurrently. Any encounter of a Reference (or subclass) object will enqueue that object into a thread-local 'discovered' list. Except for FinalReference, marking stops there, and does not mark through the referent.
  2. Concurrent processing is performed after the final-mark pause. GC workers scan all discovered lists that have been collected by concurrent marking, and depending on reachability of the referent, either drop the Reference, or enqueue it into the global 'pending' list (from where it will be processed by Java reference handler thread). In addition to that, we must ensure that no referents become resurrected by accessing Reference.get() on it. In order to achieve this, we employ special barriers in Reference.get() intrinsics that return NULL when the referent is not reachable.

Testing: hotspot_gc_shenadoah (release+fastdebug, x86+aarch64), specjvm+specjbb without regressions, tier1, tier2, vmTestbase_vm_metaspace, vmTestbase_nsk_jvmti, with -XX:+UseShenandoahGC without regressions, specjvm with various levels of verification


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Testing

Linux x64 Linux x86 Windows x64 macOS x64
Build (5/5 running) (2/2 running) (2/2 running) (2/2 running)

Issue

  • JDK-8254315: Shenandoah: Concurrent weak reference processing

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/505/head:pull/505
$ git checkout pull/505

rkennke added 24 commits Sep 23, 2020
@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Oct 5, 2020

👋 Welcome back rkennke! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Oct 5, 2020

@rkennke this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout shenandoah-concurrent-weakrefs
git fetch https://git.openjdk.java.net/jdk master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

@openjdk openjdk bot added the merge-conflict label Oct 5, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Oct 5, 2020

@rkennke The following labels will be automatically applied to this pull request:

  • hotspot
  • shenandoah

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added hotspot shenandoah labels Oct 5, 2020
@openjdk openjdk bot added ready and removed merge-conflict labels Oct 29, 2020
@openjdk openjdk bot added merge-conflict and removed ready labels Nov 2, 2020
@openjdk openjdk bot added ready merge-conflict and removed merge-conflict ready labels Nov 2, 2020
@openjdk openjdk bot added ready and removed merge-conflict labels Nov 3, 2020
Copy link
Contributor

@shipilev shipilev left a comment

Looks fine, modulo few nits below:

@openjdk
Copy link

@openjdk openjdk bot commented Nov 3, 2020

⚠️ @rkennke This pull request contains merges that bring in commits not present in the target repository. Since this is not a "merge style" pull request, these changes will be squashed when this pull request in integrated. If this is your intention, then please ignore this message. If you want to preserve the commit structure, you must change the title of this pull request to Merge <project>:<branch> where <project> is the name of another project in the OpenJDK organization (for example Merge jdk:master).

@rkennke
Copy link
Contributor Author

@rkennke rkennke commented Nov 3, 2020

/integrate

@openjdk openjdk bot closed this Nov 3, 2020
@openjdk openjdk bot added integrated and removed ready rfr labels Nov 3, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Nov 3, 2020

@rkennke Since your change was applied there have been 7 commits pushed to the master branch:

  • 83f3cf4: 8196087: java/awt/image/DrawImage/IncorrectUnmanagedImageRotatedClip.java fails
  • 134e22a: 8255606: Enable concurrent stack processing on x86_32 platforms
  • ca216ba: 8255810: Zero: build fails without JVMTI
  • d47336b: 8248320: Provide a unique accessible name for
  • 64a9811: 8255798: Remove dead headless code in CompileJavaModules.gmk
  • 364b0fe: 8255801: Race when building ct.sym build tools
  • f389a71: 8255737: Zero: DO_UPDATE_INSTRUCTION_COUNT should only update when relevant VM flags are set

Your commit was automatically rebased without conflicts.

Pushed as commit f64a15d.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot integrated shenandoah
5 participants