Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8265362 java/net/Socket/UdpSocket.java fails with "java.net.BindException: Address already in use" (macos-aarch64) #4103

Closed
wants to merge 3 commits into from

Conversation

msheppar
Copy link

@msheppar msheppar commented May 18, 2021

The test java/net/Socket/UdpSocket.java has been seen to fail with a BindException, in the testMaxSockets test, on a regular basis on macOS-aarch64 platform. testMaxSockets tests the maximum number of UDP Sockets that may be created as defined by a system property sun.net.maxDatagramSockets. It invokes the Socket constructor Socket(InetAddress host, int port, boolean stream) with stream set to false to create a UDP Socket. This instantiation is a compound operation, consisting of the creation of a socket, the explicit binding of wildcard address and ephemeral port, and a connect to the socket end point specified in the constructor parameters. Analysis has shown that during the test that the OS intermittently allocates the same ephemeral port multiple times during the bind system call, such that two separate sockets end up bound to the same port. Then on the connect invocation a BindException is thrown for the second socket. This has been determined to be a transient condition during heavy loads and where a significant number of ephemeral ports are being allocated.

As this is deemed an OS issues, and in order to increase test stability, it has been found that for the BindException condition a retry of the Socket creation mitigates the issues and tests the max creation property.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8265362: java/net/Socket/UdpSocket.java fails with "java.net.BindException: Address already in use" (macos-aarch64)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/4103/head:pull/4103
$ git checkout pull/4103

Update a local copy of the PR:
$ git checkout pull/4103
$ git pull https://git.openjdk.java.net/jdk pull/4103/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 4103

View PR using the GUI difftool:
$ git pr show -t 4103

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/4103.diff

…if a BindException thrown during the testMaxSockets test
@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented May 18, 2021

👋 Welcome back msheppar! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented May 18, 2021

@msheppar The following label will be automatically applied to this pull request:

  • net

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the net label May 18, 2021
@msheppar msheppar changed the title JDK-8265362: additions to execute a retry of UDP Socket construction … JDK-8265362: java/net/Socket/UdpSocket.java fails with "java.net.BindException: Address already in use" (macos-aarch64) May 18, 2021
@msheppar msheppar changed the title JDK-8265362: java/net/Socket/UdpSocket.java fails with "java.net.BindException: Address already in use" (macos-aarch64) JDK-8265362 java/net/Socket/UdpSocket.java fails with "java.net.BindException: Address already in use" (macos-aarch64) May 18, 2021
throw soEx;
}
}
return newUdpSocket;
Copy link
Contributor

@AlanBateman AlanBateman May 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this test in advance of JEP 353 as we didn't have much coverage for this deprecated constructor. No objection to the retry if it helps. Is the catching of SocketException a left over from testing? I assume the patch can be simplified down to:

try {
   return new Socket(InetAddress.getLoopbackAddress(), 8000, false);
} catch (BindException e) {
    System.out.println(...);
   return new Socket(InetAddress.getLoopbackAddress(), 8000, false);
}

Copy link
Author

@msheppar msheppar May 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, thanks for that ... updated as requested

Copy link
Contributor

@AlanBateman AlanBateman May 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, and if you want to keep it consistent with the existing code then you could rename "Socket newUdpSocket" and "biEx", or just change it to "return new Socket(...)".

Copy link
Contributor

@AlanBateman AlanBateman May 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW: Is one retry enough? There is at least one other replace where we've had to retry to workaround a macOS bug and one retry was enough there too.

Copy link
Author

@msheppar msheppar May 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have submitted a significant number of MACH5 job runs with repeat mode over the past week - approx 50 x 50, and observed the retry to have been triggered about 5 times, this gives some level of assurance that test stability will exist. But CI builds will exercise this a lot more.

Copy link
Author

@msheppar msheppar May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated as requested newUdpSocket --> s, and biEx --> unexpected

…xception: Address already in use" (macos-aarch64)

update as per AB comment
removal of bugid from ProblemList
@openjdk openjdk bot added the rfr label May 21, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented May 21, 2021

Webrevs

dfuch
dfuch approved these changes May 21, 2021
Copy link
Member

@dfuch dfuch left a comment

Thanks for taking this on Mark! The proposed changes look good to me.

@openjdk
Copy link

@openjdk openjdk bot commented May 21, 2021

@msheppar This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8265362: java/net/Socket/UdpSocket.java fails with "java.net.BindException: Address already in use" (macos-aarch64)

Reviewed-by: dfuchs, alanb

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 76 new commits pushed to the master branch:

  • f5562f1: 8258535: jvm.ClassReader should set the accessor to the corresponding record component
  • d8e6e28: 8267544: (test) rmi test NonLocalSkeleton fails if network has multiple adapters with the same address
  • f453166: 8267578: Remove unnecessary preview checks
  • 49f622c: 8208747: [a11y] [macos] In Optionpane Demo, inside ComponentDialog Example, unable to navigate to all items, with VO on
  • 54520fb: 8267580: The method JavacParser#peekToken is wrong when the first parameter is not zero
  • 3113910: 8267553: Extra JavaThread assignment in ClassLoader::create_class_path_entry()
  • 4d26f22: 8264304: Create implementation for NSAccessibilityToolbar protocol peer
  • 6288a99: 8267404: vmTestbase/vm/mlvm/anonloader/stress/oome/metaspace/Test.java failed with OutOfMemoryError
  • 71e2fa2: 8267531: [x86] Assembler::andb(Address,Register) encoding is incorrect
  • 4023646: 8266528: Optimize C2 VerifyIterativeGVN execution time
  • ... and 66 more: https://git.openjdk.java.net/jdk/compare/e6705c0e4b548a83197c3ea70bdef25ec65d4c00...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label May 21, 2021
…xception: Address already in use" (macos-aarch64)

update as per request by AB
private Socket newUdpSocket() throws IOException {
return new Socket(InetAddress.getLoopbackAddress(), 8000, false);
Socket s = null;
Copy link
Contributor

@zoulasc zoulasc May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @zoulasc, thanks for making a comment in an OpenJDK project!

All comments and discussions in the OpenJDK Community must be made available under the OpenJDK Terms of Use. If you already are an OpenJDK Author, Committer or Reviewer, please click here to open a new issue so that we can record that fact. Please Use "Add GitHub user zoulasc for the summary.

If you are not an OpenJDK Author, Committer or Reviewer, simply check the box below to accept the OpenJDK Terms of Use for your comments.

Your comment will be automatically restored once you have accepted the OpenJDK Terms of Use.

try {
s = new Socket(InetAddress.getLoopbackAddress(), 8000, false);
} catch (BindException unexpected) {
System.out.println("BindException caught retry Socket creation");
Copy link
Contributor

@zoulasc zoulasc May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @zoulasc, thanks for making a comment in an OpenJDK project!

All comments and discussions in the OpenJDK Community must be made available under the OpenJDK Terms of Use. If you already are an OpenJDK Author, Committer or Reviewer, please click here to open a new issue so that we can record that fact. Please Use "Add GitHub user zoulasc for the summary.

If you are not an OpenJDK Author, Committer or Reviewer, simply check the box below to accept the OpenJDK Terms of Use for your comments.

Your comment will be automatically restored once you have accepted the OpenJDK Terms of Use.


try {
s = new Socket(InetAddress.getLoopbackAddress(), 8000, false);
} catch (BindException unexpected) {
Copy link
Contributor

@zoulasc zoulasc May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @zoulasc, thanks for making a comment in an OpenJDK project!

All comments and discussions in the OpenJDK Community must be made available under the OpenJDK Terms of Use. If you already are an OpenJDK Author, Committer or Reviewer, please click here to open a new issue so that we can record that fact. Please Use "Add GitHub user zoulasc for the summary.

If you are not an OpenJDK Author, Committer or Reviewer, simply check the box below to accept the OpenJDK Terms of Use for your comments.

Your comment will be automatically restored once you have accepted the OpenJDK Terms of Use.

Copy link
Author

@msheppar msheppar May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used stdout for convenience as it aligns with the two other output statements, so that I see immediately the retry output as follows:

----------System.out:(12/349)----------
[TestNG] Running:
java/net/Socket/UdpSocket.java

BindException caught retry Socket creation
test UdpSocket.testMaxSockets(): success
test UdpSocket.testSendReceive(): success

===============================================

Copy link
Author

@msheppar msheppar May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could have used return directly in multiple places ... but my style preference is a single exit point from a method

@mlbridge
Copy link

@mlbridge mlbridge bot commented May 24, 2021

Mailing list message from Christos Zoulas on net-dev:

On May 24, 2021, at 10:39 AM, Mark Sheppard <msheppar at openjdk.java.net> wrote:

I could have used return directly in multiple places ... but my style preference is a single exit point from a method

My preference is the opposite :-) I like the "early returns" coding style because I don't need to "keep state" while
reviewing code; specially long methods (my brain is not as good as it used to be). In this case there is also over
initialization of "s", (it does not need to be set to null), and because of the java flow control analysis deficiencies
"s" cannot be declared to be "final". So getting rid of the variable makes a lot of sense to me (fewer lines of code
and less complexity).

Best,

christos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/net-dev/attachments/20210524/75a4925d/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <https://mail.openjdk.java.net/pipermail/net-dev/attachments/20210524/75a4925d/signature.asc>

1 similar comment
@mlbridge
Copy link

@mlbridge mlbridge bot commented May 24, 2021

Mailing list message from Christos Zoulas on net-dev:

On May 24, 2021, at 10:39 AM, Mark Sheppard <msheppar at openjdk.java.net> wrote:

I could have used return directly in multiple places ... but my style preference is a single exit point from a method

My preference is the opposite :-) I like the "early returns" coding style because I don't need to "keep state" while
reviewing code; specially long methods (my brain is not as good as it used to be). In this case there is also over
initialization of "s", (it does not need to be set to null), and because of the java flow control analysis deficiencies
"s" cannot be declared to be "final". So getting rid of the variable makes a lot of sense to me (fewer lines of code
and less complexity).

Best,

christos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/net-dev/attachments/20210524/75a4925d/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <https://mail.openjdk.java.net/pipermail/net-dev/attachments/20210524/75a4925d/signature.asc>

@msheppar
Copy link
Author

@msheppar msheppar commented May 24, 2021

/integrate

@openjdk openjdk bot closed this May 24, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels May 24, 2021
@openjdk
Copy link

@openjdk openjdk bot commented May 24, 2021

@msheppar Since your change was applied there have been 80 commits pushed to the master branch:

  • 640a2af: 8267184: Add -Djava.security.manager=allow to tests calling System.setSecurityManager
  • f04db5f: 8224158: assertion related to NPE at DynamicCallSiteDesc::withArgs should be reworded
  • 838a007: 8267584: The java.security.krb5.realm system property only needs to be defined once
  • f2d880c: 8266400: importkeystore fails to a password less pkcs12 keystore
  • f5562f1: 8258535: jvm.ClassReader should set the accessor to the corresponding record component
  • d8e6e28: 8267544: (test) rmi test NonLocalSkeleton fails if network has multiple adapters with the same address
  • f453166: 8267578: Remove unnecessary preview checks
  • 49f622c: 8208747: [a11y] [macos] In Optionpane Demo, inside ComponentDialog Example, unable to navigate to all items, with VO on
  • 54520fb: 8267580: The method JavacParser#peekToken is wrong when the first parameter is not zero
  • 3113910: 8267553: Extra JavaThread assignment in ClassLoader::create_class_path_entry()
  • ... and 70 more: https://git.openjdk.java.net/jdk/compare/e6705c0e4b548a83197c3ea70bdef25ec65d4c00...master

Your commit was automatically rebased without conflicts.

Pushed as commit bb085f6.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated net
4 participants