8268714: [macos-aarch64] 7 java/net/httpclient/websocket tests failed

[dfuch]

Hi,

Please find below a test-only change to fix some intermittent failures observed with the httpclient/websocket tests:
these tests intermittently and randomly fail with ENOMEM ("No buffer space available").

Some machines in our CI seem to allow a higher level of concurrency while being (maybe) configured with lower system resources (such as available buffer space for the TCP stack).

Some of the httpclient/websocket tests attempt to fill the sockets buffers in order to assert some conditions when the buffers are full and writing is paused. When the test process terminates, this leaves behind TCP sockets in the TIME_WAIT state that still hold system buffer resources in case retransmission is needed. When several such tests are run this ends up causing random "No buffer space available" errors on other tests (including these tests themselves) running concurrently or shortly after on the same machine.

This change implements a few tricks to alleviate the situation:

• configure the tests with smaller send buffers on the client side and receive buffers on the server side, in order to limit how much buffer space is consumed by the test.
• when the not-reading server is closed, and before the accepted socket is closed, read all available data off the socket buffer in order to free up the buffer space that the test has consumed before closing the socket.
• in some tests that create a large number of HttpClients, limit the number of clients created in shared client mode, and add a call to System.gc() and a small pause to give time for gc to collect the old clients which are no longer referenced.

With these changes, I have run the HttpClient tests 200 times on the problematic machines without observing any failures (where previously there was at least a couple of failures per 50 runs). I also ran tier1 once, and tier2 twice and the results came clean.

I am therefore claiming success (even if it might prove temporary ;-) )

If these failures come back to haunt the CI again after this fix, a further remediation policy could be to put the httpclient/websocket directory in exclusive test execution mode (in TEST.root) - this seems to work too - but cleaning up garbage in the tests themselves seems preferable.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8268714: [macos-aarch64] 7 java/net/httpclient/websocket tests failed

Reviewers

• Chris Hegarty (@ChrisHegarty - Reviewer)
• Michael McMahon (@Michael-Mc-Mahon - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk17 pull/79/head:pull/79
$ git checkout pull/79

Update a local copy of the PR:
$ git checkout pull/79
$ git pull https://git.openjdk.java.net/jdk17 pull/79/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 79

View PR using the GUI difftool:
$ git pr show -t 79

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk17/pull/79.diff

[bridgekeeper]

👋 Welcome back dfuchs! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@dfuch The following label will be automatically applied to this pull request:

• net

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 00: Full (d2b63d52)

[ChrisHegarty]

Looks good to me. ( a pragmatic approach to the issue at hand - resolve intermittently failing tests )

[openjdk]

@dfuch This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8268714: [macos-aarch64] 7 java/net/httpclient/websocket tests failed

Reviewed-by: chegar, michaelm

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[Michael-Mc-Mahon]

LGTM

[dfuch]

/integrate

[openjdk]

Going to push as commit 8ea0606.
Since your change was applied there has been 1 commit pushed to the master branch:

• 38dbe05: 8268901: JDK-8268768 missed removing two files

Your commit was automatically rebased without conflicts.

[openjdk]

@dfuch Pushed as commit 8ea0606.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.