8308807: MulticastSocket cannot join IPv4 multicast group when IPv6 is enabled (aix)

[deepa181]

DatagramSocket delegates to an inner DatagramSocket object. Irrespective of whether datagramSocket is IPv4 or IPv6, we create an IPv6 datagramChannel as its's delegate. So, This can cause problems with operations like joinGroup.

On AIX, IPv6 datagramSocket can not join an IPv4 multicast group.

These failures can be fixed by making sure that the delegate created for a datagram socket has the same protocol family.

Reported Issue : JDK-8308807

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Error

 ⚠️ 8308807 is used in problem lists: [test/jdk/ProblemList.txt]

Issue

• JDK-8308807: MulticastSocket cannot join IPv4 multicast group when IPv6 is enabled (aix) (Bug - P3)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/14142/head:pull/14142
$ git checkout pull/14142

Update a local copy of the PR:
$ git checkout pull/14142
$ git pull https://git.openjdk.org/jdk.git pull/14142/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 14142

View PR using the GUI difftool:
$ git pr show -t 14142

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/14142.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back deepa181! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@deepa181 The following labels will be automatically applied to this pull request:

• net
• nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 01: Full (896bacb6)
• 00: Full (930a2038)

[AlanBateman]

I've moved the JBS issue to the right place and fix the description.

The changes proposed here impact all platforms and will require discussion and cleanup. If I understand the intent, if you create a DatagramSocket or MulticastSocket that is bound to a specific local address then you want to use that to determine if the underlying socket is IPv4 or IPv6. That might be okay but multicasting normally means binding to the wildcard address so it doesn't help - I assume it only works for the test because new InetSocketAddress(port).getAddress() returns an Inet4Address so you get an IPv4 socket. It should only create a dual IPv4/IPv6 socket for this case so this is why it will break other platforms. In any case, can you look at the isXXX methods defined by sun.nio.ch.Net to see how it is handled on other platforms.

[deepa181]

I've moved the JBS issue to the right place and fix the description.

The changes proposed here impact all platforms and will require discussion and cleanup. If I understand the intent, if you create a DatagramSocket or MulticastSocket that is bound to a specific local address then you want to use that to determine if the underlying socket is IPv4 or IPv6. That might be okay but multicasting normally means binding to the wildcard address so it doesn't help. In any case, can you look at the isXXX methods defined by sun.nio.ch.Net to see how it is handled on other platforms.

Thanks @AlanBateman for fix JBS issue description. I will look at the isXXX methods defined by sun.nio.ch.Net.

[msheppar]

at a cursory glance, and without a deep dive to follow any associated call flows into native code, these changes look like the are not aligned with the semantics espoused in the systems newtwork properties, especially that of the first:

IPv4 / IPv6

java.net.preferIPv4Stack (default: false)
If IPv6 is available on the operating system the underlying native socket will be, by default, an IPv6 socket which lets applications connect to, and accept connections from, both IPv4 and IPv6 hosts. However, in the case an application would rather use IPv4 only sockets, then this property can be set to true. The implication is that it will not be possible for the application to communicate with IPv6 only hosts.

java.net.preferIPv6Addresses (default: false)
When dealing with a host which has both IPv4 and IPv6 addresses, and if IPv6 is available on the operating system, the default behavior is to prefer using IPv4 addresses over IPv6 ones. This is to ensure backward compatibility: for example, for applications that depend on the representation of an IPv4 address (e.g. 192.168.1.1). This property can be set to true to change that preference and use IPv6 addresses over IPv4 ones where possible, or system to preserve the order of the addresses as returned by the system-wide resolver 

[deepa181]

Yes, I agree with @AlanBateman and @msheppar . Even when we try to have an IPv4 multicast socket join an IPv4 multicast group, we still fail because the delegate that is created for an IPv4 multicast socket is IPv4/IPv6 (dual stack) in nature. AIX does not allow dual stack (IPv4/IPv6) sockets to join IPv4 multicast groups.

[AlanBateman]

Yes, I agree with @AlanBateman and @msheppar . Even when we try to have an IPv4 multicast socket join an IPv4 multicast group, we still fail because the delegate that is created for an IPv4 multicast socket is IPv4/IPv6 (dual stack) in nature. AIX does not allow dual stack (IPv4/IPv6) sockets to join IPv4 multicast groups.

The "Platform dependencies" section in the java.nio.channels.MulticastChannel (implemented by DatagramChannel) makes it clear that it is implementation specific as to whether a channel to an IPv6 socket can join an IPv4 multicast group. The recommendation is to specify the protocol family when creating the channel. Legacy DatagramSocket/MulticastSocket do not define a constructor that allows the protocol family be specified at xxxSocket create time. That issue has been there since JDK 1.4 when IPv6 support was added. I'm curious why this might be coming up with AIX now.

[deepa181]

Here, We are looking at all of the test failures mentioned in the JBS issue right now and trying to fix them . So, @AlanBateman Do you have any suggestion to fix these tests? I would be happy to look into that.

Thanks.

[AlanBateman]

@MBaesken @RealCLanger I wonder if you might have a bit of time to comment on this issue. If I read Deepa's comment correctly, she is saying that IPv6 UDP sockets on AIX cannot join IPv4 multicast groups. The spec doesn't require this but it problematic for MulticastSocket (as it has been since Java 4 when IPv6 was added). The part that I'm wondering about is why this issue might be coming up now. As I understand, SAP have supporting/maintaining/testing on AIX for a long time. Maybe your systems don't have IPv6 enabled so you never run into this?

[RealCLanger]

@MBaesken @RealCLanger I wonder if you might have a bit of time to comment on this issue. If I read Deepa's comment correctly, she is saying that IPv6 UDP sockets on AIX cannot join IPv4 multicast groups. The spec doesn't require this but it problematic for MulticastSocket (as it has been since Java 4 when IPv6 was added). The part that I'm wondering about is why this issue might be coming up now. As I understand, SAP have supporting/maintaining/testing on AIX for a long time. Maybe your systems don't have IPv6 enabled so you never run into this?

Well, in that area we always had deviations on AIX, so definitely this rings a bell. However, since AIX was not the most important platform for us and that particular IPv6/IPv4 multicast feature didn't seem to be mission critical and wouldn't fail TCK tests, we rather resorted to excluding some tests internally, I believe. But it would be good to really resolve this compatibility/spec/documentation issue, if possible.

[AlanBateman]

Well, in that area we always had deviations on AIX, so definitely this rings a bell. However, since AIX was not the most important platform for us and that particular IPv6/IPv4 multicast feature didn't seem to be mission critical and wouldn't fail TCK tests, we rather resorted to excluding some tests internally, I believe. But it would be good to really resolve this compatibility/spec/documentation issue, if possible.

Thanks. So yes, if some/all of the tests for multicasting were excluded then it helps explain why it didn't come up. The spec doesn't require that a DatgaramSocket or DatagramChannel to an IPv6 socket be capable of joining an IPv4 multicast group. So the tests in several areas might need updates as I don't think we've had a platform to date where it wasn't possible.

[deepa181]

Well, in that area we always had deviations on AIX, so definitely this rings a bell. However, since AIX was not the most important platform for us and that particular IPv6/IPv4 multicast feature didn't seem to be mission critical and wouldn't fail TCK tests, we rather resorted to excluding some tests internally, I believe. But it would be good to really resolve this compatibility/spec/documentation issue, if possible.

Thanks. So yes, if some/all of the tests for multicasting were excluded then it helps explain why it didn't come up. The spec doesn't require that a DatgaramSocket or DatagramChannel to an IPv6 socket be capable of joining an IPv4 multicast group. So the tests in several areas might need updates as I don't think we've had a platform to date where it wasn't possible.

@AlanBateman , is it possible if there is a central place where the change could me made for fixing all of the multicast test cases?

[AlanBateman]

@AlanBateman , is it possible if there is a central place where the change could me made for fixing all of the multicast test cases?

Your starting point is probably to exclude all of the tests that are failing. This will be tests for DatagramSocket/MulticastSocket, DatagramChannel and the tests for the JMX agent that use the discovery protocol. Once they are excluded then you can start working through the issues.

For the JMX agent, then I would expect that if if the system property com.sun.management.jdp.address is configured then the protocol family is known so it should be possible to create the DatagramChannel to use that protocol family. It might do so already, but you can check that.

For the networking and NIO tests then you probably should look at jdk.test.lib.NetworkConfiguration. That has a number of AIX specific comments that may be out of date. After that I think you will need to work through the tests one by one. Between NetworkConfiguration and jdk.test.lib.net.IPSupport, I would expect you have all the infrastructure needed for the tests to not attempt to join IPv4 multicast groups when IPSupport.hasIPv6() is true.

[pushkarnk]

Yes, I agree with @AlanBateman and @msheppar . Even when we try to have an IPv4 multicast socket join an IPv4 multicast group, we still fail because the delegate that is created for an IPv4 multicast socket is IPv4/IPv6 (dual stack) in nature. AIX does not allow dual stack (IPv4/IPv6) sockets to join IPv4 multicast groups.

The "Platform dependencies" section in the java.nio.channels.MulticastChannel (implemented by DatagramChannel) makes it clear that it is implementation specific as to whether a channel to an IPv6 socket can join an IPv4 multicast group. The recommendation is to specify the protocol family when creating the channel. Legacy DatagramSocket/MulticastSocket do not define a constructor that allows the protocol family be specified at xxxSocket create time. That issue has been there since JDK 1.4 when IPv6 support was added. I'm curious why this might be coming up with AIX now.

IIUC what @deepa181 is saying, it looks like the "IPv4 socket being unable to join an IPv4 multicast group" problems surfaced when we moved away from PlainDatagramSocketImpl in JDK 17? Probably because the delegate created does not factor in the ProtocolFamily and though we started with attempting to have an "IPv4 multicast socket join an IPv4 group", we end up trying to have an "IPv4/IPv6 dual-stack socket (the delegate) join an IPv4 group" which AIX doesn't permit.

[dfuch]

IIUC what @deepa181 is saying, it looks like the "IPv4 socket being unable to join an IPv4 multicast group" problems surfaced when we moved away from PlainDatagramSocketImpl in JDK 17? Probably because the delegate created does not factor in the ProtocolFamily and though we started with attempting to have an "IPv4 multicast socket join an IPv4 group", we end up trying to have an "IPv4/IPv6 dual-stack socket (the delegate) join an IPv4 group" which AIX doesn't permit.

IIRC It has never been possible to create a non-dual MulticastSocket / DatagramSocket without setting the property -Djava.net.preferIPv4Stack=true. The possibility to select a protocol family independently of this property is only available with DatagramChannel.

[AlanBateman]

IIRC It has never been possible to create a non-dual MulticastSocket / DatagramSocket without setting the property -Djava.net.preferIPv4Stack=true. The possibility to select a protocol family independently of this property is only available with DatagramChannel.

Right now, Net.canIPv6SocketJoinIPv4Group() returns false on AIX so it will never attempt to join an IPv4 multicast group when the socket is IPv6. The old DatagamSocketImpl didn't have an eager check, which makes me wonder what AIX supports, it's not clear!

[deepa181]

Hi @AlanBateman ,

I ran the Jtreg tests on jdk17 and observed the following tests passed unexpectedly when I set the VM_OPTIONS to -Djdk.net.usePlainDatagramSocketImpl=true

1. java/net/MulticastSocket/B6427403.java
2. java/net/MulticastSocket/NoLoopbackPackets.java
3. java/net/MulticastSocket/SetLoopbackMode.java
4. java/net/MulticastSocket/Test.java

I am curious about how setting the jdk.net.usePlainDatagramSocketImpl property impacts their behavior. I would like to understand why they pass with this configuration and whether it has any implications.

Could you please share your insights on this matter? I appreciate your assistance in understanding this behavior and ensuring the correctness of the tests.

Thank you.

[AlanBateman]

I am curious about how setting the jdk.net.usePlainDatagramSocketImpl property impacts their behavior. I would like to understand why they pass with this configuration and whether it has any implications.

Could you please share your insights on this matter? I appreciate your assistance in understanding this behavior and ensuring the correctness of the tests.

I think this issue needs an authoritative summary on what AIX supports and doesn't support. The observations reported conflict with some of the comments so I don't think I can say definitely if there are code or test changes required.

For a long time, Net.canJoin6WithIPv4Group() has returned "false" on AIX so there is no attempt to join IPv4 multicast groups with a dual IPv4/IPv6 socket. DatagramChannel has supported multicasting since JDK 7 (2011) so if there were issues or test failures on AIX then surely they would have been noticed before now, or maybe all the test were excluded, or maybe the testing was done with IPv6 disabled.

The old DatagramSocketImpl implementation didn't have a setting/configuration to say if a dual IPv4/IPv6 socket could join an IPv4 multicast group, it just attempted it. Your results suggests that it works, which is surprising. Does AIX have the equivalent of strace to trace system calls? It might be useful to confirm that the socket is actually IPv6 and that it joining IPv4 multicast groups.

[bridgekeeper]

@deepa181 This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[TOatGithub]

@AlanBateman , is it possible if there is a central place where the change could me made for fixing all of the multicast test cases?
...
For the networking and NIO tests then you probably should look at jdk.test.lib.NetworkConfiguration. That has a number of AIX specific comments that may be out of date. ...

Created https://bugs.openjdk.org/browse/JDK-8315651 to address jdk.test.lib.NetworkConfiguration.

[MBaesken]

With this PR added to our build/test queue we noticed a number of additional test failures cross platform .
Examples
java/net/MulticastSocket/JoinLeave.java
java/net/MulticastSocket/MulticastAddresses.java

[openjdk]

@deepa181 Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See OpenJDK Developers’ Guide for more information.

[AlanBateman]

I see this PR has been re-based but it has the same changes as before. This PR lacks an authoritative statement on what AIX supports and doesn't support, I don't think it's possible to make any progress on this without that.

[dfuch]

I concur with Alan that the changes proposed here are not appropriate for DatagramSocket/MulticastSocket.

[bridgekeeper]

@deepa181 This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[bridgekeeper]

@deepa181 This pull request has been inactive for more than 8 weeks and will now be automatically closed. If you would like to continue working on this pull request in the future, feel free to reopen it! This can be done using the /open pull request command.