2018-05-15 18:38:15 ERROR NetworkClient:144 - Node [127.0.0.1:9200] failed (Connection refused: con ect); selected next node [17.13.50.21:9200] 2018-05-15 18:38:19 ERROR NetworkClient:144 - Node [127.0.0.1:9200] failed (Connection refused: con ect); no other nodes left - aborting...

[LunaLuan]

Hello there, I can not connect ES-Spark from local to Elasticsearch in server. I think services need dependencces, so I connect Spark to Elasticsearch in servers. This is my code:

es_read_conf = { 
    # specify the node that we are sending data to (this should be the master)    
    "es.nodes" : '17.13.50.21:9200',
            
        # specify the port in case it is not the default port
    "es.port" : '9200',
    
    # specify the read resource in the format 'index/doc-type'
    "es.resource" : "stream-test/sample"
    }

es_rdd = sc.newAPIHadoopRDD(
    inputFormatClass="org.elasticsearch.hadoop.mr.EsInputFormat",
    keyClass="org.apache.hadoop.io.NullWritable", 
    valueClass="org.elasticsearch.hadoop.mr.LinkedMapWritable", 
    conf=es_read_conf)

And I met error:
2018-05-15 18:38:15 ERROR NetworkClient:144 - Node [127.0.0.1:9200] failed (Connection refused: con ect); selected next node [17.13.50.21:9200] 2018-05-15 18:38:19 ERROR NetworkClient:144 - Node [127.0.0.1:9200] failed (Connection refused: con ect); no other nodes left - aborting...

Can you help me? Thank you.

[hemangakbari]

It is not getting proper jar files

[LunaLuan]

Thank you. I resolved the problem. I forgot "pyspark --jars ..."

[stevemar]

closing as the OP said it's resolved.

[pradeep0306]

I had added to spark submit
libraryDependencies += "org.elasticsearch" %% "elasticsearch-spark-20" % "8.1.1"

and even in spark submit script as --jars elasticsearch-spark-20_2.11-8.1.1.jar

but I am still getting same issue.

22/04/01 01:46:44 ERROR rest.NetworkClient: Node [127.0.0.1:9200] failed (Connection refused (Connection refused)); no other nodes left - aborting...
Exception in thread "main" org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'