8254598: StringDedupTable should use OopStorage #3662

kimbarrett · 2021-04-23T19:48:47Z

Please review this change to the String Deduplication facility. It is being
changed to use OopStorage to hold weak references to relevant objects,
rather than bespoke data structures requiring dedicated processing phases by
supporting GCs.

(The Shenandoah update was contributed by Zhengyu Gu.)

This change significantly simplifies the interface between a given GC and
the String Deduplication facility, which should make it much easier for
other GCs to opt in. However, this change does not alter the set of GCs
providing support; currently only G1 and Shenandoah support string
deduplication. Adding support by other GCs will be in followup RFEs.

Reviewing via the diffs might not be all that useful for some parts, as
several files have been essentially completely replaced, and a number of
files have been added or eliminated. The full webrev might be a better
place to look.

The major changes are in gc/shared/stringdedup/* and in the supporting
collectors, but there are also some smaller changes in other places, most
notably classfile/{stringTable,javaClasses}.

This change is additionally labeled for review by core-libs, although there
are no source changes there. This change injects a byte field of bits into
java.lang.String, using one of the previously unused padding bytes in that
class. (There were two unused bytes, now only one.)

Testing:
mach5 tier1-2 with and without -XX:+UseStringDeduplication

Locally (linux-x64) ran all of the existing tests that use string
deduplication with both G1 and Shenandoah. Note that
TestStringDeduplicationInterned.java is disabled for shenandoah, as it
currently fails, for reasons I haven't figured out but suspect are test
related.

gc/stringdedup/ -- these used to be in gc/g1
runtime/cds/SharedStringsDedup.java
runtime/cds/appcds/cacheObject/DifferentHeapSizes.java
runtime/cds/appcds/sharedStrings/*

shenandoah-only:

gc/shenandoah/jvmti/TestHeapDump.java
gc/shenandoah/TestStringDedup.java
gc/shenandoah/TestStringDedupStress.java

Performance tested baseline, baseline + stringdedup, new with stringdedup,
finding no significant differences.

/label hotspot
/label core-libs
/contributor @kimbarrett
/contributor @zhengyu123

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8254598: StringDedupTable should use OopStorage

Reviewers

Coleen Phillimore (@coleenp - Reviewer) ⚠️ Review applies to 2df362c
Ioi Lam (@iklam - Reviewer) ⚠️ Review applies to 2df362c
Thomas Schatzl (@tschatzl - Reviewer) ⚠️ Review applies to 656e242
Albert Mingkun Yang (@albertnetymk - Committer) ⚠️ Review applies to 656e242

Contributors

Kim Barrett <kbarrett@openjdk.org>
Zhengyu Gu <zgu@openjdk.org>

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/3662/head:pull/3662
$ git checkout pull/3662

Update a local copy of the PR:
$ git checkout pull/3662
$ git pull https://git.openjdk.java.net/jdk pull/3662/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 3662

View PR using the GUI difftool:
$ git pr show -t 3662

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/3662.diff

bridgekeeper · 2021-04-23T19:49:36Z

👋 Welcome back kbarrett! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2021-04-23T20:00:36Z

@kimbarrett
The hotspot label was successfully added.

openjdk · 2021-04-23T20:01:51Z

@kimbarrett
The core-libs label was successfully added.

openjdk · 2021-04-23T20:02:51Z

@kimbarrett Syntax: /contributor (add|remove) [@user | openjdk-user | Full Name <email@address>]. For example:

/contributor add @openjdk-bot
/contributor add duke
/contributor add J. Duke <duke@openjdk.org>

openjdk · 2021-04-23T20:03:03Z

@kimbarrett Syntax: /contributor (add|remove) [@user | openjdk-user | Full Name <email@address>]. For example:

/contributor add @openjdk-bot
/contributor add duke
/contributor add J. Duke <duke@openjdk.org>

kimbarrett · 2021-04-24T06:23:12Z

/contributor add @kimbarrett
/contributor add @zhengyu123

openjdk · 2021-04-24T06:23:43Z

@kimbarrett
Contributor Kim Barrett <kbarrett@openjdk.org> successfully added.

openjdk · 2021-04-24T06:24:34Z

@kimbarrett
Contributor Zhengyu Gu <zgu@openjdk.org> successfully added.

mlbridge · 2021-04-24T19:38:22Z

Webrevs

coleenp

I looked at the runtime code, which looks fine. I didn't read the GC code.

src/hotspot/share/classfile/javaClasses.inline.hpp

src/hotspot/share/runtime/globals.hpp

src/hotspot/share/runtime/mutexLocker.cpp

src/hotspot/share/classfile/stringTable.cpp

openjdk · 2021-04-27T22:48:12Z

@kimbarrett this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout new_dedup2
git fetch https://git.openjdk.java.net/jdk master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

src/hotspot/share/gc/shared/stringdedup/stringDedupTable.cpp

zhengyu123

Just FYI:
Concurrent GC, such as Sheanndoah and ZGC (if it decides to implement string deduplication in the future), can not enqueue candidates during concurrent thread root scanning, because of potential lock rank inversion between OopStorage lock and stack watermark lock.

src/hotspot/share/classfile/stringTable.cpp

iklam

The CDS changes look reasonable to me.

tschatzl

First pass, just comment suggestions for now.

src/hotspot/share/classfile/javaClasses.hpp

src/hotspot/share/gc/shared/stringdedup/stringDedup.hpp

src/hotspot/share/classfile/vmSymbols.hpp

kimbarrett · 2021-05-07T08:21:40Z

I've made some improvements to comments, responding to Thomas's suggestions. Ive also changed some of the naming around the injected java.lang.String flags, with modification now using test-and-set semantics (and named accordingly), i.e. returning true if already set rather than if changed.

I've not yet dealt with the merge conflict in Shenandoah, but it looks like it should be fairly easy. Just waiting for more comments before dealing with merging updates.

tschatzl

Lgtm.

src/hotspot/share/gc/shared/stringdedup/stringDedup.hpp

albertnetymk

Just some minor comments.

src/hotspot/share/gc/shared/stringdedup/stringDedupProcessor.cpp

src/hotspot/share/gc/shared/stringdedup/stringDedupTable.cpp

src/hotspot/share/memory/universe.cpp

src/hotspot/share/gc/shared/stringdedup/stringDedup.cpp

kimbarrett · 2021-05-13T22:47:34Z

Following up on an off-line discussion with @albertnetymk , I've done a little refactoring of Requests::add. I also made a few other small cleanups, noticed while dealing with @albertnetymk comments. I still haven't dealt with the accumulated merge conflicts. I'll be doing that next.

openjdk · 2021-05-14T04:28:11Z

@kimbarrett This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8254598: StringDedupTable should use OopStorage

Co-authored-by: Kim Barrett <kbarrett@openjdk.org>
Co-authored-by: Zhengyu Gu <zgu@openjdk.org>
Reviewed-by: coleenp, iklam, tschatzl, ayang

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

kimbarrett · 2021-05-14T05:06:12Z

The "merge from master" commit (ccb9951) doesn't build with Shenandoah. I've asked Zhengyu to take a look.

zhengyu123

The "merge from master" commit (ccb9951) doesn't build with Shenandoah. I've asked Zhengyu to take a look.

Just missing a parameter:

index ddaa66ccc14..93a067fa22d 100644
--- a/src/hotspot/share/gc/shenandoah/shenandoahSTWMark.cpp
+++ b/src/hotspot/share/gc/shenandoah/shenandoahSTWMark.cpp
@@ -57,7 +57,7 @@ ShenandoahInitMarkRootsClosure::ShenandoahInitMarkRootsClosure(ShenandoahObjToSc
 
 template <class T>
 void ShenandoahInitMarkRootsClosure::do_oop_work(T* p) {
-  ShenandoahMark::mark_through_ref<T, NO_DEDUP>(p, _queue, _mark_context, false);
+  ShenandoahMark::mark_through_ref<T, NO_DEDUP>(p, _queue, _mark_context, NULL, false);
 }```

zhengyu123 · 2021-04-28T20:51:08Z

src/hotspot/share/gc/shared/stringdedup/stringDedupTable.cpp

+  size_t _number_of_buckets;
+  size_t _bucket_index;
+  size_t _shrink_index;
+  bool _grow_only;


Indentation

Not sure what this indentation comment is referring to.

kimbarrett · 2021-05-14T18:38:17Z

Thanks @zhengyu123 for the shenandoah merge fix.

Thanks @iklam , @coleenp , @tschatzl , @albertnetymk for reviews.

/integrate

openjdk · 2021-05-14T18:39:22Z

@kimbarrett Pushed as commit be0a655.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

openjdk bot added the hotspot hotspot-dev@openjdk.org label Apr 23, 2021

openjdk bot added the core-libs core-libs-dev@openjdk.org label Apr 23, 2021

kimbarrett force-pushed the new_dedup2 branch 2 times, most recently from 8a98add to bc96315 Compare April 24, 2021 19:30

New string deduplication

2df362c

kimbarrett force-pushed the new_dedup2 branch from bc96315 to 2df362c Compare April 24, 2021 19:32

openjdk bot added the rfr Pull request is ready for review label Apr 24, 2021

coleenp approved these changes Apr 27, 2021

View reviewed changes

src/hotspot/share/classfile/javaClasses.inline.hpp Outdated Show resolved Hide resolved

src/hotspot/share/runtime/globals.hpp Show resolved Hide resolved

src/hotspot/share/runtime/mutexLocker.cpp Show resolved Hide resolved

src/hotspot/share/classfile/stringTable.cpp Show resolved Hide resolved

openjdk bot added the merge-conflict Pull request has merge conflict with target branch label Apr 27, 2021

iklam suggested changes Apr 28, 2021

View reviewed changes

src/hotspot/share/gc/shared/stringdedup/stringDedupTable.cpp Show resolved Hide resolved

zhengyu123 reviewed Apr 28, 2021

View reviewed changes

src/hotspot/share/classfile/stringTable.cpp Outdated Show resolved Hide resolved

fix some typos in comments

2a9cb05

iklam approved these changes May 3, 2021

View reviewed changes

tschatzl reviewed May 4, 2021

View reviewed changes

kimbarrett commented May 6, 2021

View reviewed changes

src/hotspot/share/classfile/vmSymbols.hpp Outdated Show resolved Hide resolved

kimbarrett added 2 commits May 7, 2021 03:42

improve naming and comments around injected String flags

c390cbb

more comment improvements

656e242

tschatzl approved these changes May 12, 2021

View reviewed changes

src/hotspot/share/gc/shared/stringdedup/stringDedup.hpp Outdated Show resolved Hide resolved

fix typo

c7960e1

albertnetymk approved these changes May 12, 2021

View reviewed changes

kimbarrett added 2 commits May 13, 2021 14:36

refactor Requests::add

41d01b8

misc cleanups

30e7b8e

Merge branch 'master' into new_dedup2

ccb9951

openjdk bot added ready Pull request is ready to be integrated and removed merge-conflict Pull request has merge conflict with target branch labels May 14, 2021

zhengyu123 reviewed May 14, 2021

View reviewed changes

kimbarrett added 2 commits May 14, 2021 14:31

fix shenandoah merge

897a1cb

Merge branch 'master' into new_dedup2

a592537

openjdk bot closed this May 14, 2021

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels May 14, 2021

kimbarrett deleted the new_dedup2 branch May 16, 2021 19:49

8254598: StringDedupTable should use OopStorage #3662

8254598: StringDedupTable should use OopStorage #3662

kimbarrett commented Apr 23, 2021 •

edited by openjdk bot

bridgekeeper bot commented Apr 23, 2021

openjdk bot commented Apr 23, 2021

openjdk bot commented Apr 23, 2021

openjdk bot commented Apr 23, 2021

openjdk bot commented Apr 23, 2021

kimbarrett commented Apr 24, 2021

openjdk bot commented Apr 24, 2021

openjdk bot commented Apr 24, 2021

mlbridge bot commented Apr 24, 2021 •

edited

coleenp left a comment

openjdk bot commented Apr 27, 2021

zhengyu123 left a comment

iklam left a comment

tschatzl left a comment

kimbarrett commented May 7, 2021

tschatzl left a comment

albertnetymk left a comment

kimbarrett commented May 13, 2021

openjdk bot commented May 14, 2021 •

edited

kimbarrett commented May 14, 2021

zhengyu123 left a comment

zhengyu123 Apr 28, 2021

kimbarrett May 14, 2021

kimbarrett commented May 14, 2021

openjdk bot commented May 14, 2021

8254598: StringDedupTable should use OopStorage #3662

8254598: StringDedupTable should use OopStorage #3662

Conversation

kimbarrett commented Apr 23, 2021 • edited by openjdk bot

Progress

Issue

Reviewers

Contributors

Reviewing

bridgekeeper bot commented Apr 23, 2021

openjdk bot commented Apr 23, 2021

openjdk bot commented Apr 23, 2021

openjdk bot commented Apr 23, 2021

openjdk bot commented Apr 23, 2021

kimbarrett commented Apr 24, 2021

openjdk bot commented Apr 24, 2021

openjdk bot commented Apr 24, 2021

mlbridge bot commented Apr 24, 2021 • edited

Webrevs

coleenp left a comment

Choose a reason for hiding this comment

openjdk bot commented Apr 27, 2021

zhengyu123 left a comment

Choose a reason for hiding this comment

iklam left a comment

Choose a reason for hiding this comment

tschatzl left a comment

Choose a reason for hiding this comment

kimbarrett commented May 7, 2021

tschatzl left a comment

Choose a reason for hiding this comment

albertnetymk left a comment

Choose a reason for hiding this comment

kimbarrett commented May 13, 2021

openjdk bot commented May 14, 2021 • edited

kimbarrett commented May 14, 2021

zhengyu123 left a comment

Choose a reason for hiding this comment

zhengyu123 Apr 28, 2021

Choose a reason for hiding this comment

kimbarrett May 14, 2021

Choose a reason for hiding this comment

kimbarrett commented May 14, 2021

openjdk bot commented May 14, 2021

kimbarrett commented Apr 23, 2021 •

edited by openjdk bot

mlbridge bot commented Apr 24, 2021 •

edited

openjdk bot commented May 14, 2021 •

edited