Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ES-Hadoop with SSL proxy #249

Closed
marcmm opened this issue Aug 20, 2014 · 1 comment
Closed

ES-Hadoop with SSL proxy #249

marcmm opened this issue Aug 20, 2014 · 1 comment

Comments

@marcmm
Copy link

marcmm commented Aug 20, 2014

I have a hadoop cluster that is on a different network than our elastic search cluster. I connect to the ES cluster through an https proxy. I'm not able to use the same type of connection when using elasticsearch-hadoop, and get a connection error : java.io.IOException:org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)
The same configuration works just fine without SSL.

Is this scenario supported?

Here's my table configuration :
CREATE EXTERNAL TABLE es_test (scenarioid string)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'es_test/scenarios',
'es.nodes' = 'server.foo.net',
'es.port' = '443',
'es.net.proxy.http.host' = 'server.foo.net',
'es.net.proxy.http.user' = 'myuser',
'es.net.proxy.http.host' = '443',
'es.net.proxy.http.pass' = 'mypw';

@costin
Copy link
Member

costin commented Aug 20, 2014

SSL is not supported out of the box. es-hadoop supports SOCKS and HTTP (NOT HTTPS) proxies, to route connections but not SSL - hence why you're getting the above exception. Your port above - 443 - is a good sign that you are using an HTTPS proxy, which is not supported.
If you are looking into adding SSL consider using a VPN proxy or some other type of proxying that handles the SSL functionality transparently for your networks.
If you've already tried that, make sure the SSL handshake is indeed transparent - many times the routing actually changes the underlying IPs and thus the two end-points cannot see each other any more.
You can easily diagnose this by trying to connect with curl/ tracert to the es network on the other end using the http proxy you mentioned.

@costin costin closed this as completed Aug 20, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants