8254315: Shenandoah: Concurrent weak reference processing

[rkennke]

Until now, references (as in java.lang.ref.Reference and its subclasses WeakReference, SoftReference, PhantomReference and the non-public FinalReference - I'll collectively call them weak references for the purpose of clarity). Workloads that make heavvy use of such weak references will therefore potentially cause significant GC pauses.

There are 3 main items that contribute to pause time linear to number of references, or worse:

• We need to scan and consider each reference on the various 'discovered' lists.
• We need to mark through subgraph of objects that are reachable only through FinalReference. Notice that this is theoretically only bounded by the live data set size.
• Finally, all no-longer-reachable references need to be enqueued in the 'pending list'

The problem is somewhat mitigated by pre-cleaning the discovered list: Any weak reference that we find to be strongly reachable will be removed before we go into the final-mark-pause. However, that is only a band-aid.

The solution to this is two-fold:

1. Extend concurrent marking to also mark the 'finalizable' subgraph of the heap. This requires to extend the marking bitmap to allow for two kinds of reachability: each object can now be strongly and finalizably reachable. Whenever marking encounters a FinalReference, it will mark through the referent and switch to 'finalizably' reachability for all objects starting from the referent. When marking encounters finalizably reachable objects while marking strongly, it will 'upgrade' reachability of such objects to strongly reachable. All of this can be done concurrently. Any encounter of a Reference (or subclass) object will enqueue that object into a thread-local 'discovered' list. Except for FinalReference, marking stops there, and does not mark through the referent.
2. Concurrent processing is performed after the final-mark pause. GC workers scan all discovered lists that have been collected by concurrent marking, and depending on reachability of the referent, either drop the Reference, or enqueue it into the global 'pending' list (from where it will be processed by Java reference handler thread). In addition to that, we must ensure that no referents become resurrected by accessing Reference.get() on it. In order to achieve this, we employ special barriers in Reference.get() intrinsics that return NULL when the referent is not reachable.

Testing: hotspot_gc_shenadoah (release+fastdebug, x86+aarch64), specjvm+specjbb without regressions, tier1, tier2, vmTestbase_vm_metaspace, vmTestbase_nsk_jvmti, with -XX:+UseShenandoahGC without regressions, specjvm with various levels of verification

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Testing

	Linux x64	Linux x86	Windows x64	macOS x64
Build	⏳ (5/5 running)	⏳ (2/2 running)	⏳ (2/2 running)	⏳ (2/2 running)

Issue

• JDK-8254315: Shenandoah: Concurrent weak reference processing

Reviewers

• Zhengyu Gu (@zhengyu123 - Reviewer) ⚠️ Review applies to 6418428
• Aleksey Shipilev (@shipilev - Reviewer) ⚠️ Review applies to 9165b6e

Download

$ git fetch https://git.openjdk.java.net/jdk pull/505/head:pull/505
$ git checkout pull/505

[bridgekeeper]

👋 Welcome back rkennke! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@rkennke this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout shenandoah-concurrent-weakrefs
git fetch https://git.openjdk.java.net/jdk master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

[openjdk]

@rkennke The following labels will be automatically applied to this pull request:

• hotspot
• shenandoah

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[shipilev]

Looks fine, modulo few nits below:

[openjdk]

⚠️ @rkennke This pull request contains merges that bring in commits not present in the target repository. Since this is not a "merge style" pull request, these changes will be squashed when this pull request in integrated. If this is your intention, then please ignore this message. If you want to preserve the commit structure, you must change the title of this pull request to Merge <project>:<branch> where <project> is the name of another project in the OpenJDK organization (for example Merge jdk:master).

[rkennke]

/integrate

[openjdk]

@rkennke Since your change was applied there have been 7 commits pushed to the master branch:

• 83f3cf4: 8196087: java/awt/image/DrawImage/IncorrectUnmanagedImageRotatedClip.java fails
• 134e22a: 8255606: Enable concurrent stack processing on x86_32 platforms
• ca216ba: 8255810: Zero: build fails without JVMTI
• d47336b: 8248320: Provide a unique accessible name for
• 64a9811: 8255798: Remove dead headless code in CompileJavaModules.gmk
• 364b0fe: 8255801: Race when building ct.sym build tools
• f389a71: 8255737: Zero: DO_UPDATE_INSTRUCTION_COUNT should only update when relevant VM flags are set

Your commit was automatically rebased without conflicts.

Pushed as commit f64a15d.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.