Implement IPv6 support for distributed XGBoost

[ikriv]

Currently distributed XGBoost does not support IPv6, this is explicitly asserted in rabit/include/rabit/internal/socket.h
Our internal network is IPv6 only, which means we can't run XGBoost out of the box.

IPv6 is 20+ years old, and IPv4 is being phased out more and more as the time progresses. I think it is very important to enable IPv6 in XGBoost for more successful adoption.

[trivialfis]

Hi @ikriv , thank you for raising the issue. Would you like to work on this?

[wuhy08]

@trivialfis I am also interested in this issue, could you point where we should start?

[trivialfis]

I will take a deeper look into this.

[ikriv]

Hi @ikriv , thank you for raising the issue. Would you like to work on this?

Hi. It's a possibility, but I can't fully commit right now :) That is, if someone else volunteers, I would be very grateful.

[trivialfis]

I will try to implement it after 1.6.

Note to myself:

• Check NCCL support.
• Change socket wrapper to support IPV6.
• Implement it in Python tracker.
• Possible change in Scala tracker.

[ikriv]

Thank you so much, @trivialfis! I think Python tracker is already good (at least it does not complain loudly), it's the socket wrapper that insists on IPv4

[ikriv]

Thanks for including this in 2.0 roadmap!
Q: is there an approximate ETA for 2.0 implementation (sorry if it's announced somewhere, I honestly searched :) )

[trivialfis]

Not yet. I will update this thread.

[trivialfis]

Hi, can anyone here help with providing guidance on testing IPv6 on a public cloud like AWS or GCP? I'm not an expert on network administrating and got lost in those cloud documents.

The draft PR is here: #8225 .

[trivialfis]

I tested the code using GCP VM instances. Feel free to try it if you are interested in this.

[zhangleiray007]

Q: I noticed the ipv6 support is on the 2.0 Roadmap. is there any plan or date for ipv6 support on pyspark :)

[trivialfis]

Would be great if there people with expertise can help work on this. It takes some effort to setup a test environment in the CI and we don't have any admin.

[CrazyBinXXX]

Is spark ipv6 supported now? If not I'd like to contribute.