No data node found when publish hosts are private IPs

[antoine-galataud]

Hi,

I use elasticsearch-hadoop version 2.1.2.
I have an Elasticsearch 1.6 cluster running in a VPC on AWS EC2 with the following network settings:

network.bind_host: "0.0.0.0"
network.publish_host: _non_loopback:ipv4_

Each node has a public IP, and is answering well from outside of the VPC.
Now I want to access them with EsSpark from outside the VPC:

EsSpark.esRDD(sc, "*/some_type", "?q=*'", Map("es.nodes" -> "1.2.3.4", "es.nodes.discovery" -> "true"))

where 1.2.3.4 is a public IP. Turning debug mode on, I get:

DEBUG org.elasticsearch.spark.rdd.ScalaEsRDD  - Nodes discovery enabled - found [10.0.0.1:9200]
DEBUG org.elasticsearch.spark.rdd.ScalaEsRDD  - Found data nodes [10.0.0.1:9200]
DEBUG org.elasticsearch.spark.rdd.ScalaEsRDD  - Filtered discovered only nodes [1.2.3.4:9200, 10.0.0.1:9200] to data-only [10.0.0.1:9200]
Exception in thread "main" org.elasticsearch.hadoop.rest.EsHadoopTransportException: java.net.NoRouteToHostException: No route to host

Turning node discovery off:

DEBUG org.elasticsearch.spark.rdd.ScalaEsRDD  - Found data nodes [10.0.0.1:9200]
DEBUG org.elasticsearch.spark.rdd.ScalaEsRDD  - Filtered discovered only nodes [1.2.3.4:9200] to data-only []
Exception in thread "main" org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: No data nodes with HTTP-enabled available; node discovery is disabled and none of nodes specified fits the criterion [1.2.3.4:9200]

When discovery is off, it should be possible to force use of public IP address

[costin]

Use the cloud/wan feature introduced in 2.2.

[antoine-galataud]

Thank you for pointing out this documentation. I'll give a try to the es.nodes.wan.only setting.