8269657: Test java/nio/channels/DatagramChannel/Loopback.java failed: Unexpected message #19314

sendaoYan · 2024-05-20T15:39:52Z

Hi all,
The test test/jdk/java/nio/channels/DatagramChannel/Loopback.java intermittent fail, the failure probability is about 2/1000.
This PR mark add some entropy(System.nanoTime) to the message content, and if the payloads doesn't match then ignore it, because payloads probably receiving some stray packet sent by another process. Only if the port unmatch and the content match we consider it a failure.
The change has been verified locally, I have runed the test after the this patch 20k times and all passed. Only change the testcase, no risk.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8269657: Test java/nio/channels/DatagramChannel/Loopback.java failed: Unexpected message (Bug - P3)

Reviewers

Daniel Fuchs (@dfuch - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/19314/head:pull/19314
$ git checkout pull/19314

Update a local copy of the PR:
$ git checkout pull/19314
$ git pull https://git.openjdk.org/jdk.git pull/19314/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 19314

View PR using the GUI difftool:
$ git pr show -t 19314

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/19314.diff

Webrev

Link to Webrev Comment

…rmittent Signed-off-by: sendaoYan <yansendao.ysd@alibaba-inc.com>

bridgekeeper · 2024-05-20T15:40:15Z

👋 Welcome back syan! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2024-05-20T15:40:37Z

@sendaoYan This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8269657: Test java/nio/channels/DatagramChannel/Loopback.java failed: Unexpected message

Reviewed-by: dfuchs

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 435 new commits pushed to the master branch:

3b3a19e: 8335314: Problem list compiler/uncommontrap/DeoptReallocFailure.java
d457609: 8319947: Recursive lightweight locking: s390x implementation
c47a0e0: 8334147: Shenandoah: Avoid taking lock for disabled free set logging
308a812: 8334645: Un-problemlist vmTestbase/nsk/sysdict/vm/stress/chain/chain007/chain007.java
b4df380: 8334763: --enable-asan: assert(_thread->is_in_live_stack((address)this)) failed: not on stack?
cd46c87: 8334843: RISC-V: Fix wraparound checking for r_array_index in lookup_secondary_supers_table_slow_path
4e8cbf8: 8335134: Test com/sun/jdi/BreakpointOnClassPrepare.java timeout
3b1ca98: 8334895: OpenJDK fails to configure on linux aarch64 when CDS is disabled after JDK-8331942
c35e58a: 8309634: Resolve CONSTANT_MethodRef at CDS dump time
243bae7: 8304693: Remove -XX:-UseVtableBasedCHA
... and 425 more: https://git.openjdk.org/jdk/compare/86eb5d9f3be30ff9df1318f18ab73c7129c978f6...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@dfuch) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

openjdk · 2024-05-20T15:40:56Z

@sendaoYan The following label will be automatically applied to this pull request:

nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2024-05-20T15:44:09Z

Webrevs

AlanBateman · 2024-05-20T16:53:11Z

No objection to add the keyword. From a quick look at the logs then it hints that it may be a kernel issue rather than a JDK issue, e.g. here one of the failures in the logs you attached:

UNSPEC socket
join ff02:0:0:0:0:0:0:a @ veth87b01b2
set outgoing multicast interface to veth87b01b2
IP_MULTICAST_LOOP enabled
send /[0:0:0:0:0:0:0:0]:50287 -> /[ff02:0:0:0:0:0:0:a]:50287
received java.nio.HeapByteBuffer[pos=5 lim=100 cap=100] from /[fe80:0:0:0:8cf3:1bff:fe85:f8e6%5]:50287
IP_MULTICAST_LOOP disabled
send /[0:0:0:0:0:0:0:0]:50287 -> /[ff02:0:0:0:0:0:0:a]:50287
selected 1
received java.nio.HeapByteBuffer[pos=5 lim=5 cap=100] from /[fe80:0:0:0:8cf3:1bff:fe85:f8e6%5]:50287
STDERR:
java.lang.RuntimeException: Unexpected message

I assume veth87b01b2 is a virtual interface. When IP_MULTICAST_LOOP is enabled the 5-byte multicast datagram is looped as expected. When IP_MULTICAST_LOOP is disabled then it appears to be looped again. So this may require digging into the kernel to better understand this. It would be interesting to know if it's always IPv6 where the failures happen.

sendaoYan · 2024-05-21T01:18:07Z

No objection to add the keyword. From a quick look at the logs then it hints that it may be a kernel issue rather than a JDK issue, e.g. here one of the failures in the logs you attached:
UNSPEC socket
join ff02:0:0:0:0:0:0:a @ veth87b01b2
set outgoing multicast interface to veth87b01b2
IP_MULTICAST_LOOP enabled
send /[0:0:0:0:0:0:0:0]:50287 -> /[ff02:0:0:0:0:0:0:a]:50287
received java.nio.HeapByteBuffer[pos=5 lim=100 cap=100] from /[fe80:0:0:0:8cf3:1bff:fe85:f8e6%5]:50287
IP_MULTICAST_LOOP disabled
send /[0:0:0:0:0:0:0:0]:50287 -> /[ff02:0:0:0:0:0:0:a]:50287
selected 1
received java.nio.HeapByteBuffer[pos=5 lim=5 cap=100] from /[fe80:0:0:0:8cf3:1bff:fe85:f8e6%5]:50287
STDERR:
java.lang.RuntimeException: Unexpected message
I assume veth87b01b2 is a virtual interface. When IP_MULTICAST_LOOP is enabled the 5-byte multicast datagram is looped as expected. When IP_MULTICAST_LOOP is disabled then it appears to be looped again. So this may require digging into the kernel to better understand this. It would be interesting to know if it's always IPv6 where the failures happen.

I reporduced this intermittent failure on a ECS, so veth87b01b2 is a virtual interface. I will close IPv6 and reproduce this failure. After I close IPv6, and this failure can not repoeduce, then maybe the failure alway happend in IPv6.

sendaoYan · 2024-05-28T09:55:37Z

The GHA test runner report two failures:

macos-x64 build fail, compiler clang14 crash when compile ad_x86.cpp. It's unrelated to this PR.
ObjectMonitorUsage.java fails which has been recorded in JDK-8332923. It's unrelated to this PR.

bridgekeeper · 2024-06-25T12:31:13Z

@sendaoYan This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

sendaoYan · 2024-06-25T12:34:43Z

Hi, can anyone review this PR.

test/jdk/java/nio/channels/DatagramChannel/Loopback.java

…we receive whose payload doesn't match what we expect

…2535

openjdk · 2024-06-26T02:51:16Z

⚠️ @sendaoYan This pull request contains merges that bring in commits not present in the target repository. Since this is not a "merge style" pull request, these changes will be squashed when this pull request in integrated. If this is your intention, then please ignore this message. If you want to preserve the commit structure, you must change the title of this pull request to Merge <project>:<branch> where <project> is the name of another project in the OpenJDK organization (for example Merge jdk:master).

test/jdk/java/nio/channels/DatagramChannel/Loopback.java

dfuch · 2024-06-26T14:43:01Z

Please give me some time to test the proposed changes.

sendaoYan · 2024-06-26T14:57:46Z

Please give me some time to test the proposed changes.

Okey.

test/jdk/java/nio/channels/DatagramChannel/Loopback.java

RealCLanger · 2024-06-27T04:39:10Z

Hi, can you maybe refrain from adding the intermittent keyword to the test? In our test setup at SAP this would mean that the test is not run any longer.

sendaoYan · 2024-06-27T06:43:53Z

Hi, can you maybe refrain from adding the intermittent keyword to the test? In our test setup at SAP this would mean that the test is not run any longer.

@dfuch I run this test after the this patch 20k times and all passed, maybe the @key intermittent tag not longer needed

dfuch · 2024-06-27T07:18:39Z

@dfuch I run this test after the this patch 20k times and all passed, maybe the @key intermittent tag not longer needed

Fine to remove the intermitent label then. We will need to change the title of the PR and the JBS issue to reflect what this change is about though.

sendaoYan · 2024-06-27T07:23:46Z

/issue JDK-8332535

dfuch · 2024-06-27T10:43:46Z

/issue JDK-8332535

openjdk · 2024-06-27T10:44:13Z

@dfuch Only the author (@sendaoYan) is allowed to issue the /issue command.

dfuch · 2024-06-27T10:46:18Z

Actually - maybe we should use JDK-8269657 for this change, since it fixes the test failure. You can grab it from @Michael-Mc-Mahon, I don't think he will mind :-)
Otherwise we'll just close it as a duplicate of JDK-8332535 - but it is strange to see an issue closed as a duplicate of its subtask.

FWIW - I have launched some repeated tests in our CI - so far this looks promising :-)

sendaoYan · 2024-06-27T11:50:18Z

Actually - maybe we should use JDK-8269657 for this change, since it fixes the test failure. You can grab it from @Michael-Mc-Mahon, I don't think he will mind :-) Otherwise we'll just close it as a duplicate of JDK-8332535 - but it is strange to see an issue closed as a duplicate of its subtask.

FWIW - I have launched some repeated tests in our CI - so far this looks promising :-)

Okey.
/issue JDK-8269657

openjdk · 2024-06-27T11:51:43Z

@sendaoYan
Adding additional issue to issue list: 8269657: java/nio/channels/DatagramChannel/Loopback.java failed: Unexpected message.

sendaoYan · 2024-06-27T11:52:50Z

/issue remove JDK-8332535

openjdk · 2024-06-27T11:52:58Z

@sendaoYan The issue 8332535 was not found in the list of additional solved issues. The list currently contains these issues: 8269657

test/jdk/java/nio/channels/DatagramChannel/Loopback.java

sendaoYan · 2024-06-28T09:12:22Z

/issue JDK-8269657

openjdk · 2024-06-28T09:13:38Z

@sendaoYan This issue is referenced in the PR title - it will now be updated.

sendaoYan · 2024-06-28T09:14:10Z

Thanks for the patient guidance and review.
/integrate

openjdk · 2024-06-28T09:15:22Z

@sendaoYan
Your change (at version fb5bdc2) is now ready to be sponsored by a Committer.

dfuch · 2024-06-28T09:25:09Z

Hi @sendaoYan - thanks for integrating all the feedback. I can sponsor this change - unless you already have a sponsor?

sendaoYan · 2024-06-28T09:28:24Z

Hi @sendaoYan - thanks for integrating all the feedbcak. I can sponsor this change - unless you already have a sponsor?

No, I need your sponsor, thanks.

dfuch · 2024-06-28T09:37:09Z

/sponsor

openjdk · 2024-06-28T09:38:19Z

Going to push as commit c798316.
Since your change was applied there have been 437 commits pushed to the master branch:

99d2bbf: 8334433: jshell.exe runs an executable test.exe on startup
6f4ddc2: 8335142: compiler/c1/TestTraceLinearScanLevel.java occasionally times out with -Xcomp
3b3a19e: 8335314: Problem list compiler/uncommontrap/DeoptReallocFailure.java
d457609: 8319947: Recursive lightweight locking: s390x implementation
c47a0e0: 8334147: Shenandoah: Avoid taking lock for disabled free set logging
308a812: 8334645: Un-problemlist vmTestbase/nsk/sysdict/vm/stress/chain/chain007/chain007.java
b4df380: 8334763: --enable-asan: assert(_thread->is_in_live_stack((address)this)) failed: not on stack?
cd46c87: 8334843: RISC-V: Fix wraparound checking for r_array_index in lookup_secondary_supers_table_slow_path
4e8cbf8: 8335134: Test com/sun/jdi/BreakpointOnClassPrepare.java timeout
3b1ca98: 8334895: OpenJDK fails to configure on linux aarch64 when CDS is disabled after JDK-8331942
... and 427 more: https://git.openjdk.org/jdk/compare/86eb5d9f3be30ff9df1318f18ab73c7129c978f6...master

Your commit was automatically rebased without conflicts.

openjdk · 2024-06-28T09:38:25Z

@dfuch @sendaoYan Pushed as commit c798316.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

sendaoYan · 2024-06-28T09:45:03Z

Thanks for the sponsor.

8332535: Mark java/nio/channels/DatagramChannel/Loopback.java as inte…

9879aab

…rmittent Signed-off-by: sendaoYan <yansendao.ysd@alibaba-inc.com>

openjdk bot added the rfr Pull request is ready for review label May 20, 2024

openjdk bot added the nio nio-dev@openjdk.org label May 20, 2024

Merge branch 'openjdk:master' into jbs8332535

3f4d450

dfuch reviewed Jun 25, 2024

View reviewed changes

test/jdk/java/nio/channels/DatagramChannel/Loopback.java Outdated Show resolved Hide resolved

sendaoYan added 2 commits June 26, 2024 10:48

add some entropy to the message content and discard any message that …

a7e552f

…we receive whose payload doesn't match what we expect

Merge branch 'jbs8332535' of github.com:sendaoYan/jdk-ysd into jbs833…

7bce58c

…2535

dfuch reviewed Jun 26, 2024

View reviewed changes

test/jdk/java/nio/channels/DatagramChannel/Loopback.java Outdated Show resolved Hide resolved

compare ByteBuffer with mismatch()

8cca8b5

dfuch reviewed Jun 26, 2024

View reviewed changes

test/jdk/java/nio/channels/DatagramChannel/Loopback.java Show resolved Hide resolved

add Thread.sleep(3000) in while loop

23277ef

dfuch reviewed Jun 26, 2024

View reviewed changes

test/jdk/java/nio/channels/DatagramChannel/Loopback.java Outdated Show resolved Hide resolved

delete Thread.sleep and InterruptedException

7b06dcf

openjdk bot changed the title ~~8332535: Mark java/nio/channels/DatagramChannel/Loopback.java as intermittent~~ 8332535: Discard any message not expect in Loopback.java Jun 27, 2024

sendaoYan changed the title ~~8332535: Discard any message not expected in Loopback.java~~ 8269657: Test Loopback.java failed: Unexpected message Jun 27, 2024

dfuch reviewed Jun 28, 2024

View reviewed changes

test/jdk/java/nio/channels/DatagramChannel/Loopback.java Outdated Show resolved Hide resolved

delete bud id 8332535

fb5bdc2

dfuch approved these changes Jun 28, 2024

View reviewed changes

openjdk bot changed the title ~~8269657: Test Loopback.java failed: Unexpected message~~ 8269657: Test java/nio/channels/DatagramChannel/Loopback.java failed: Unexpected message Jun 28, 2024

openjdk bot added the ready Pull request is ready to be integrated label Jun 28, 2024

openjdk bot added the sponsor Pull request is ready to be sponsored label Jun 28, 2024

openjdk bot added the integrated Pull request has been integrated label Jun 28, 2024

openjdk bot closed this Jun 28, 2024

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review sponsor Pull request is ready to be sponsored labels Jun 28, 2024

sendaoYan deleted the jbs8332535 branch June 28, 2024 09:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8269657: Test java/nio/channels/DatagramChannel/Loopback.java failed: Unexpected message #19314

8269657: Test java/nio/channels/DatagramChannel/Loopback.java failed: Unexpected message #19314

sendaoYan commented May 20, 2024 •

edited by openjdk bot

Loading

bridgekeeper bot commented May 20, 2024

openjdk bot commented May 20, 2024 •

edited

Loading

openjdk bot commented May 20, 2024

mlbridge bot commented May 20, 2024 •

edited

Loading

AlanBateman commented May 20, 2024

sendaoYan commented May 21, 2024 •

edited

Loading

sendaoYan commented May 28, 2024 •

edited

Loading

bridgekeeper bot commented Jun 25, 2024

sendaoYan commented Jun 25, 2024

openjdk bot commented Jun 26, 2024

dfuch commented Jun 26, 2024

sendaoYan commented Jun 26, 2024

RealCLanger commented Jun 27, 2024

sendaoYan commented Jun 27, 2024 •

edited

Loading

dfuch commented Jun 27, 2024 •

edited

Loading

sendaoYan commented Jun 27, 2024

dfuch commented Jun 27, 2024

openjdk bot commented Jun 27, 2024

dfuch commented Jun 27, 2024 •

edited

Loading

sendaoYan commented Jun 27, 2024

openjdk bot commented Jun 27, 2024

sendaoYan commented Jun 27, 2024

openjdk bot commented Jun 27, 2024

sendaoYan commented Jun 28, 2024

openjdk bot commented Jun 28, 2024

sendaoYan commented Jun 28, 2024

openjdk bot commented Jun 28, 2024

dfuch commented Jun 28, 2024 •

edited

Loading

sendaoYan commented Jun 28, 2024

dfuch commented Jun 28, 2024

openjdk bot commented Jun 28, 2024

openjdk bot commented Jun 28, 2024

sendaoYan commented Jun 28, 2024

8269657: Test java/nio/channels/DatagramChannel/Loopback.java failed: Unexpected message #19314

8269657: Test java/nio/channels/DatagramChannel/Loopback.java failed: Unexpected message #19314

Conversation

sendaoYan commented May 20, 2024 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

Webrev

bridgekeeper bot commented May 20, 2024

openjdk bot commented May 20, 2024 • edited Loading

openjdk bot commented May 20, 2024

mlbridge bot commented May 20, 2024 • edited Loading

Webrevs

AlanBateman commented May 20, 2024

sendaoYan commented May 21, 2024 • edited Loading

sendaoYan commented May 28, 2024 • edited Loading

bridgekeeper bot commented Jun 25, 2024

sendaoYan commented Jun 25, 2024

openjdk bot commented Jun 26, 2024

dfuch commented Jun 26, 2024

sendaoYan commented Jun 26, 2024

RealCLanger commented Jun 27, 2024

sendaoYan commented Jun 27, 2024 • edited Loading

dfuch commented Jun 27, 2024 • edited Loading

sendaoYan commented Jun 27, 2024

dfuch commented Jun 27, 2024

openjdk bot commented Jun 27, 2024

dfuch commented Jun 27, 2024 • edited Loading

sendaoYan commented Jun 27, 2024

openjdk bot commented Jun 27, 2024

sendaoYan commented Jun 27, 2024

openjdk bot commented Jun 27, 2024

sendaoYan commented Jun 28, 2024

openjdk bot commented Jun 28, 2024

sendaoYan commented Jun 28, 2024

openjdk bot commented Jun 28, 2024

dfuch commented Jun 28, 2024 • edited Loading

sendaoYan commented Jun 28, 2024

dfuch commented Jun 28, 2024

openjdk bot commented Jun 28, 2024

openjdk bot commented Jun 28, 2024

sendaoYan commented Jun 28, 2024

sendaoYan commented May 20, 2024 •

edited by openjdk bot

Loading

openjdk bot commented May 20, 2024 •

edited

Loading

mlbridge bot commented May 20, 2024 •

edited

Loading

sendaoYan commented May 21, 2024 •

edited

Loading

sendaoYan commented May 28, 2024 •

edited

Loading

sendaoYan commented Jun 27, 2024 •

edited

Loading

dfuch commented Jun 27, 2024 •

edited

Loading

dfuch commented Jun 27, 2024 •

edited

Loading

dfuch commented Jun 28, 2024 •

edited

Loading