8268294: Reusing HttpClient in a WebSocket.Listener hangs. #4506

Michael-Mc-Mahon · 2021-06-16T07:56:11Z

Hi,

This fixes a problem where the listener methods of a WebSocket client were being invoked by the Selector manager thread, which is problematic, because if the implementation of any of these methods tries to do any blocking work, this impacts other http activity, and if the blocking work is a http client call, then a hang can result. The fix makes the HttpClient's executor available to WebSocketImpl and that is used to offload the listener invocations.

The fix also adds a more comprehensive test framework for WebSockets (in WebSocketServer). Up to now we just had a limited server side in the tests that can only do the handshake. This change adds an API and implementation for server's to receive websocket messages and send replies back to clients. To implement this, the server hooks into WebSocket's Frame, MessageEncoder and MessageDecoder classes. Therefore, the implementations of these classes had to be modified to allow for the encoding of server generated messages and the decoding of client generated messages. The only difference between client and server in this respect relates to payload masking which is only done for messages sent from client to server.

There's a couple of warts that I wasn't sure what to do with. 1) There is already a copy of the Frame implementation class in the test hierarchy. I presume this is used by other tests, but that implementation is not used by this change. 2) The WebSocketServer is based on the existing DummyWebSocketServer class which is used by other tests. I can't see any easy way to refactor/combine these implementations.

Thanks,

Michael.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8268294: Reusing HttpClient in a WebSocket.Listener hangs.

Reviewers

Daniel Fuchs (@dfuch - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/4506/head:pull/4506
$ git checkout pull/4506

Update a local copy of the PR:
$ git checkout pull/4506
$ git pull https://git.openjdk.java.net/jdk pull/4506/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 4506

View PR using the GUI difftool:
$ git pr show -t 4506

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/4506.diff

bridgekeeper · 2021-06-16T07:57:16Z

👋 Welcome back michaelm! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2021-06-16T08:00:41Z

@Michael-Mc-Mahon The following label will be automatically applied to this pull request:

net

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2021-06-16T08:05:09Z

Webrevs

dfuch

Great fix and great test Michael.
I don't specially like the blind cast to HttpClientFacade - maybe we can revisit that later.
I have only one comment - about the use of Thread.sleep(3_000) in the test.

dfuch · 2021-06-16T09:07:49Z

...tpclient/websocket/java.net.http/jdk/internal/net/http/websocket/WebSocketAndHttpClient.java

+        WebSocketTest wsTest = new WebSocketTest(httpClient, args[0]);
+        HttpTest httpTest = new HttpTest(httpClient, args[1]);
+
+        AtomicReference<String> result = new AtomicReference<>("failed");


Would it be possible to use a CountDownLatch - or a CompletableFuture instead of an atomic reference in order to replace the Thread.sleep(3_000) at line 44 with latch.await() or cf.join()?
The test would then fail in jtreg timeout without the fix, but would not have to wait for an arbitrary magic 3s delay if the fix is present (which might not be enough delay on slow machines with debug builds etc...)

Good idea on the CF or latch. As regards the blind cast, I suppose I could test for it and if the type is different, leave the executor reference as null, which reverts current behavior, but I can't imagine a scenario where that would actually happen. If it did, would we want to revert the behavior, or fail visibly with CCE?

I agree - I can't see how it could be something else than HttpClientFacade with the current implementation. Maybe just let's leave that up for later cleanup. What I had in mind was adding a static helper method somewhere on a public class in jdk.internal.net.http that would have access to the package private APIs - something like:

public static Executor getClientExecutor(HttpClient client) { ... and here we could do some switch depending on the concrete subclass of HttpClient we're dealing with ... }

But I don't see an obvious home for such a method - so maybe we should leave this for a later cleanup.

…ooks odd

openjdk · 2021-06-16T11:24:32Z

@Michael-Mc-Mahon This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8268294: Reusing HttpClient in a WebSocket.Listener hangs.

Reviewed-by: dfuchs

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 2 new commits pushed to the master branch:

72b3b0a: 8267752: KVHashtable doesn't deallocate entries
af13fe1: 8267870: Remove unnecessary char_converter during class loading

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

Michael-Mc-Mahon · 2021-06-17T07:13:03Z

/integrate

openjdk · 2021-06-17T07:14:12Z

Going to push as commit 2d088fa.
Since your change was applied there have been 21 commits pushed to the master branch:

e844610: 8268927: Windows: link error: unresolved external symbol "int __cdecl convert_to_unicode(char const *,wchar_t * *)"
3637e50: Merge
9f7851b: 8260194: Update the documentation for -Xcheck:jni
ee03bc6: 8268863: ProblemList serviceability/sa/TestJmapCoreMetaspace.java on linux-x64 with ZGC
f0f2178: 8268909: ProblemList jdk/jfr/api/consumer/streaming/TestLatestEvent.java on win-x64
54f5ffe: 8259338: Add expiry exception for identrustdstx3 alias to VerifyCACerts.java test
2c7e47e: 8268774: Residual logging output written to STDOUT, not STDERR
8ea0606: 8268714: [macos-aarch64] 7 java/net/httpclient/websocket tests failed
38dbe05: 8268901: JDK-8268768 missed removing two files
9ad19f7: 8256934: C2: assert(C->live_nodes() <= C->max_node_limit()) failed: Live Node limit exceeded limit
... and 11 more: https://git.openjdk.java.net/jdk/compare/3ad658677b1ad2ea15d75ec7f1fe71e37f3913b9...master

Your commit was automatically rebased without conflicts.

openjdk · 2021-06-17T07:14:32Z

@Michael-Mc-Mahon Pushed as commit 2d088fa.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Michael-Mc-Mahon added 2 commits June 15, 2021 16:37

fix implementation plus some extra websocket test work

6be92f5

add comments to test

4bf897f

openjdk bot added the rfr Pull request is ready for review label Jun 16, 2021

openjdk bot added the net net-dev@openjdk.org label Jun 16, 2021

dfuch reviewed Jun 16, 2021

View reviewed changes

Michael-Mc-Mahon added 2 commits June 16, 2021 12:09

changed AtomicReference to CompletableFuture

e898a9e

fixed small error in previous update. Doesn't affect test, but just l…

4a8cb7e

…ooks odd

dfuch approved these changes Jun 16, 2021

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Jun 16, 2021

Merge branch 'master' into websocket

8f279da

dfuch approved these changes Jun 16, 2021

View reviewed changes

openjdk bot closed this Jun 17, 2021

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jun 17, 2021

Michael-Mc-Mahon deleted the websocket branch June 17, 2021 07:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8268294: Reusing HttpClient in a WebSocket.Listener hangs. #4506

8268294: Reusing HttpClient in a WebSocket.Listener hangs. #4506

Uh oh!

Michael-Mc-Mahon commented Jun 16, 2021 •

edited by openjdk bot

Loading

Uh oh!

bridgekeeper bot commented Jun 16, 2021

Uh oh!

openjdk bot commented Jun 16, 2021

Uh oh!

mlbridge bot commented Jun 16, 2021 •

edited

Loading

Uh oh!

dfuch left a comment

Uh oh!

dfuch Jun 16, 2021

Uh oh!

Michael-Mc-Mahon Jun 16, 2021

Uh oh!

dfuch Jun 16, 2021

Uh oh!

openjdk bot commented Jun 16, 2021 •

edited

Loading

Uh oh!

Michael-Mc-Mahon commented Jun 17, 2021

Uh oh!

openjdk bot commented Jun 17, 2021

Uh oh!

openjdk bot commented Jun 17, 2021

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

8268294: Reusing HttpClient in a WebSocket.Listener hangs. #4506

8268294: Reusing HttpClient in a WebSocket.Listener hangs. #4506

Uh oh!

Conversation

Michael-Mc-Mahon commented Jun 16, 2021 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Reviewing

Uh oh!

bridgekeeper bot commented Jun 16, 2021

Uh oh!

openjdk bot commented Jun 16, 2021

Uh oh!

mlbridge bot commented Jun 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

dfuch left a comment

Choose a reason for hiding this comment

Uh oh!

dfuch Jun 16, 2021

Choose a reason for hiding this comment

Uh oh!

Michael-Mc-Mahon Jun 16, 2021

Choose a reason for hiding this comment

Uh oh!

dfuch Jun 16, 2021

Choose a reason for hiding this comment

Uh oh!

openjdk bot commented Jun 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Michael-Mc-Mahon commented Jun 17, 2021

Uh oh!

openjdk bot commented Jun 17, 2021

Uh oh!

openjdk bot commented Jun 17, 2021

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Michael-Mc-Mahon commented Jun 16, 2021 •

edited by openjdk bot

Loading

mlbridge bot commented Jun 16, 2021 •

edited

Loading

openjdk bot commented Jun 16, 2021 •

edited

Loading