Can't initialize H2OContext on IPv6-only machine

[tomdz]

On a machine without IPv4 it simply crashes, even in local-only mode (master = local[*]):

scala> val h2oContext = new org.apache.spark.h2o.H2OContext(sc).start()
15/12/09 11:53:33 WARN H2OContext: Increasing 'spark.locality.wait' to value 30000
[Exit 255]

[tomdz]

This can also be reproduced on a dual stack machine by running Spark with -Djava.net.preferIPv6Addresses=true

[mmalohlava]

Thanks @tomdz! We have to improve handling of IPv6 stack in H2O itself.

[tomdz]

Any workaround I can use in the meantime ?

[mmalohlava]

So far, i was not able reproduce it, i have dual stack machine, passing:

--conf spark.driver.extraJavaOptions="-Djava.net.preferIPv6Addresses=true"\
--conf spark.executor.extraJavaOptions="-Djava.net.preferIPv6Addresses=true"

I tried to pass different master={local | local[*] | local-cluster[....] }, but creation of H2O context was still successful (but i can see that even spark is using IPV4).

However, i know that inside H2O there are few places expecting IPv4.

[tomkraljevic]

Yes, IPv4 is a requirement, and will likely be for the foreseeable future.
Interestingly, I have yet to see an enterprise customer without IPv4.

Tom

On Dec 9, 2015, at 1:30 PM, Michal Malohlava notifications@github.com wrote:

So far, i was not able reproduce it, i have dual stack machine, passing:

--conf spark.driver.extraJavaOptions="-Djava.net.preferIPv6Addresses=true"
--conf spark.executor.extraJavaOptions="-Djava.net.preferIPv6Addresses=true"
I tried to pass different master={local | local[*] | local-cluster[....] }, but creation of H2O context was still successful (but i can see that even spark is using IPV4).

However, i know that inside H2O there are few places expecting IPv4.

—
Reply to this email directly or view it on GitHub.

[tomdz]

We (Facebook) are very heavily invested in IPv6 (see e.g. http://www.internetsociety.org/deploy360/wp-content/uploads/2014/04/WorldIPv6Congress-IPv6_LH-v2.pdf) and software that we run internally is generally required to play nice with IPv6-only hosts these days (which in Java is not all that difficult, fwiw).

[tomdz]

@mmalohlava I tried this with Spark 1.5.1 running on JDK 1.8.0_60-b27 on Centos, fwiw. Maybe adding java.net.preferIPv4Stack=false also helps ?

[mmalohlava]

@tomdz so far no luck - java7/java8 with preferIPv4Stack=false uses IPv4. Need to try that on dedicated linux box

[mmalohlava]

Should be fixed now...

[mmalohlava]

HEXDEV-593 https://0xdata.atlassian.net/browse/HEXDEV-593