8253179: Replace LinkedList Impl in net.http.Http2Connection #431

c-cleary · 2020-09-30T10:22:11Z

This patch replaces a LinkedList data structure used in the net.http.Http2Connection class with an ArrayList. This issue relates to JDK-8246048: Replace LinkedList with ArrayLists in java.net.

Some justifications for this change are as follows:

Sequential Access Times for ArrayLists are improved due to locality of reference (i.e ArrayList elements stored in same memory neighborhood)
Get(index) operations are O(1) time complexity for ArrayLists as opposed to worst-case O(N-1) for LinkedLists
While insertion operations can be expensive (O(N) in the worst case), these operations appear to be infrequent/non-existent in this case.

Additional justifications or challenges to those listed are welcome! The general idea is that ArrayLists out-perform LinkedLists in this scenario.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8253179: Replace LinkedList Impl in net.http.Http2Connection

Reviewers

Daniel Fuchs (@dfuch - Reviewer)
Pavel Rappo (@pavelrappo - Reviewer)
Chris Hegarty (@ChrisHegarty - Reviewer)
Aleksey Shipilev (@shipilev - Reviewer)

Download

$ git fetch https://git.openjdk.java.net/jdk pull/431/head:pull/431
$ git checkout pull/431

bridgekeeper · 2020-09-30T10:24:21Z

👋 Welcome back ccleary! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2020-09-30T10:25:48Z

@ccleary-oracle The following label will be automatically applied to this pull request:

net

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2020-10-01T07:50:44Z

Webrevs

01: Full - Incremental (fc3f574)
00: Full (9ecb85e)

shipilev · 2020-10-01T08:22:42Z

src/java.net.http/share/classes/jdk/internal/net/http/Http2Connection.java

@@ -700,7 +700,7 @@ void shutdown(Throwable t) {
        Throwable initialCause = this.cause;
        if (initialCause == null) this.cause = t;
        client2.deleteConnection(this);
-        List<Stream<?>> c = new LinkedList<>(streams.values());
+        List<Stream<?>> c = new ArrayList<>(streams.values());


Why can't we dispense with this copy completely, and instead just iterate streams.values()?

Good point Aleksey. I guess that the original intent was to avoid ConcurrentModificationException. However I see that streams is a ConcurrentHashMap now. So maybe we could dispense of the copy:

private final Map<Integer,Stream<?>> streams = new ConcurrentHashMap<>();

Could be worth changing the type from Map<Integer,Stream<?>> to ConcurrentMap<Integer,Stream<?>> in the declaration above if we decide to get rid of the copy though.

Taking a look at this presently

Although Aleksey's proposal does not change the behavior, it raises a question (to be explored separately): can a stream be added to that map while Http2Connection.shutdown is in progress? If this is the case, then the method is racy.

Even if that happened , it should not matter as the connection will be (or is) closed anyway. I don't believe that's the kind of scenario that the copy was trying to prevent. Closing a stream will definitely remove it from the map though, so my guess that is where the CME would happen if no precaution was taken.

c-cleary · 2020-10-02T08:42:40Z

src/java.net.http/share/classes/jdk/internal/net/http/Http2Connection.java

@@ -700,8 +700,7 @@ void shutdown(Throwable t) {
        Throwable initialCause = this.cause;
        if (initialCause == null) this.cause = t;
        client2.deleteConnection(this);
-        List<Stream<?>> c = new ArrayList<>(streams.values());
-        for (Stream<?> s : c) {
+        for (Stream<?> s : streams.values()) {
            try {


To address some of the feedback given, the streams are now directly addressed instead of making a copy. ConcurrentMap has also been used to initialise streams instead of Map. Seeking feedback/opinions on this if possible

dfuch

Looks good to me Conor.
best regards

-- daniel

openjdk · 2020-10-06T09:57:09Z

@ccleary-oracle This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for more details.

After integration, the commit message for the final commit will be:

8253179: Replace LinkedList Impl in net.http.Http2Connection

Reviewed-by: dfuchs, prappo, chegar, shade

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 80 new commits pushed to the master branch:

54b340b: 8254081: java/security/cert/PolicyNode/GetPolicyQualifiers.java fails due to an expired certificate
51fdb4c: 8254075: Shenandoah: Remove ShenandoahCodeRootsStyle diagnostic flag and related test
77921b9: 8254080: fix for JDK-8204256 causes jlink test failures
57493c1: 8253694: Remove Thread::muxAcquire() from ThreadCrashProtection()
d2b1dc6: 8254054: Pre-submit testing using GitHub Actions should not use the deprecated set-env command
a34f48b: 8253832: CharsetDecoder : decode() mentioning CoderMalfunctionError behavior not as per spec
f397b60: 8251123: doclint warnings about missing javadoc tags and comments
c9d1dcc: 8253902: G1: Starting a new marking cycle before the conc mark thread fully completed causes assertion failure
9199783: 8253565: PPC64: Fix duplicate if condition in vm_version_ppc.cpp
1728547: 8254010: GrowableArrayView::print fails to compile
... and 70 more: https://git.openjdk.java.net/jdk/compare/04775f11fe637029b6040b494bb495939c6d3fcf...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@dfuch, @pavelrappo, @ChrisHegarty, @shipilev) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

pavelrappo

This looks good.

c-cleary · 2020-10-06T15:46:25Z

/integrate

openjdk · 2020-10-06T15:47:28Z

@ccleary-oracle
Your change (at version fc3f574) is now ready to be sponsored by a Committer.

shipilev · 2020-10-06T17:04:01Z

I can sponsor this, but it is not clear what testing was done on this. At least tier1?

c-cleary · 2020-10-06T21:39:05Z

I can sponsor this, but it is not clear what testing was done on this. At least tier1?

tier1, tier2, jdk_net, test/jdk/java/net/httpclient tests were conducted with all passing.

shipilev · 2020-10-07T06:20:57Z

/sponsor

openjdk · 2020-10-07T06:22:10Z

@shipilev @ccleary-oracle Since your change was applied there have been 86 commits pushed to the master branch:

3973073: 8253977: More memory leaks in client-libs on macOS
2a0389a: 8254095: remove jdk.test.lib.Utils::distro() method
ac772cd: 8253750: use build-stable default seed for Utils.RANDOM_GENERATOR
6712f8c: 8254084: Remove TemplateTable::pd_initialize
82fe023: 8254061: Missing space in flag description
bd50ccd: 8253735: Cleanup SearchIndexItem API
54b340b: 8254081: java/security/cert/PolicyNode/GetPolicyQualifiers.java fails due to an expired certificate
51fdb4c: 8254075: Shenandoah: Remove ShenandoahCodeRootsStyle diagnostic flag and related test
77921b9: 8254080: fix for JDK-8204256 causes jlink test failures
57493c1: 8253694: Remove Thread::muxAcquire() from ThreadCrashProtection()
... and 76 more: https://git.openjdk.java.net/jdk/compare/04775f11fe637029b6040b494bb495939c6d3fcf...master

Your commit was automatically rebased without conflicts.

Pushed as commit 703b345.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8253179: Replace LinkedList Impl in net.http.Http2Connection

9ecb85e

openjdk bot added the net label Sep 30, 2020

c-cleary marked this pull request as ready for review Oct 1, 2020

openjdk bot added the rfr label Oct 1, 2020

shipilev reviewed Oct 1, 2020

View changes

8253179: Removed ArrayList copy of streams

fc3f574

c-cleary reviewed Oct 2, 2020

View changes

c-cleary changed the title ~~8253179: Replace LinkedList Impl in net.http.Http2Connection~~ 8253179: WIP Replace LinkedList Impl in net.http.Http2Connection Oct 2, 2020

c-cleary changed the title ~~8253179: WIP Replace LinkedList Impl in net.http.Http2Connection~~ [WIP] 8253179: Replace LinkedList Impl in net.http.Http2Connection Oct 2, 2020

openjdk bot removed the rfr label Oct 2, 2020

c-cleary changed the title ~~[WIP] 8253179: Replace LinkedList Impl in net.http.Http2Connection~~ 8253179: Replace LinkedList Impl in net.http.Http2Connection Oct 5, 2020

openjdk bot added the rfr label Oct 5, 2020

dfuch approved these changes Oct 6, 2020

View changes

openjdk bot added the ready label Oct 6, 2020

pavelrappo approved these changes Oct 6, 2020

View changes

ChrisHegarty approved these changes Oct 6, 2020

View changes

openjdk bot added the sponsor label Oct 6, 2020

shipilev approved these changes Oct 6, 2020

View changes

openjdk bot closed this Oct 7, 2020

openjdk bot added integrated and removed sponsor labels Oct 7, 2020

openjdk bot removed ready rfr labels Oct 7, 2020

8253179: Replace LinkedList Impl in net.http.Http2Connection #431

8253179: Replace LinkedList Impl in net.http.Http2Connection #431

c-cleary commented Sep 30, 2020 •

edited by openjdk[bot] bot

bridgekeeper bot commented Sep 30, 2020

openjdk bot commented Sep 30, 2020

mlbridge bot commented Oct 1, 2020 •

edited

shipilev Oct 1, 2020

dfuch Oct 1, 2020

c-cleary Oct 1, 2020

pavelrappo Oct 1, 2020 •

edited

dfuch Oct 1, 2020

c-cleary Oct 2, 2020

dfuch left a comment

openjdk bot commented Oct 6, 2020 •

edited

pavelrappo left a comment

c-cleary commented Oct 6, 2020

openjdk bot commented Oct 6, 2020

shipilev commented Oct 6, 2020

c-cleary commented Oct 6, 2020

shipilev commented Oct 7, 2020

openjdk bot commented Oct 7, 2020

8253179: Replace LinkedList Impl in net.http.Http2Connection #431

8253179: Replace LinkedList Impl in net.http.Http2Connection #431

Conversation

c-cleary commented Sep 30, 2020 • edited by openjdk[bot] bot

Progress

Issue

Reviewers

Download

bridgekeeper bot commented Sep 30, 2020

openjdk bot commented Sep 30, 2020

mlbridge bot commented Oct 1, 2020 • edited

Webrevs

shipilev Oct 1, 2020

Choose a reason for hiding this comment

dfuch Oct 1, 2020

Choose a reason for hiding this comment

c-cleary Oct 1, 2020

Choose a reason for hiding this comment

pavelrappo Oct 1, 2020 • edited

Choose a reason for hiding this comment

dfuch Oct 1, 2020

Choose a reason for hiding this comment

c-cleary Oct 2, 2020

Choose a reason for hiding this comment

dfuch left a comment

openjdk bot commented Oct 6, 2020 • edited

pavelrappo left a comment

c-cleary commented Oct 6, 2020

openjdk bot commented Oct 6, 2020

shipilev commented Oct 6, 2020

c-cleary commented Oct 6, 2020

shipilev commented Oct 7, 2020

openjdk bot commented Oct 7, 2020

c-cleary commented Sep 30, 2020 •

edited by openjdk[bot] bot

mlbridge bot commented Oct 1, 2020 •

edited

pavelrappo Oct 1, 2020 •

edited

openjdk bot commented Oct 6, 2020 •

edited