Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL support in ElasticHQ #376

Closed
chitenderkumar opened this Issue Apr 18, 2018 · 25 comments

Comments

Projects
None yet
5 participants
@chitenderkumar
Copy link

chitenderkumar commented Apr 18, 2018

with reference to #375 it would be great if we can get the SSL support in ElasticHQ to connect to Elasticsearch endpoint which are behind the proxy servers over the SSL/TLS protocols.

kindly let me know if any information require from my end.

@royrusso royrusso added this to the 3.5.0 milestone Apr 18, 2018

@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 7, 2018

I've found a way to enable SSL using a CA pem file and have tested it. How do I go about making a pull request to merge the feature?

@royrusso

This comment has been minimized.

Copy link
Member

royrusso commented May 7, 2018

@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 7, 2018

Uhm. I don't have permission to push to this repo. Got the error below.

remote: Permission to ElasticHQ/elasticsearch-HQ.git denied to chngtrn.
fatal: unable to access 'https://github.com/ElasticHQ/elasticsearch-HQ/': The requested URL returned error: 403

@royrusso

This comment has been minimized.

Copy link
Member

royrusso commented May 7, 2018

Ah yes. Looks like the instructions don't mention the important part... you have to fork the repo first.

@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 7, 2018

Nevermind. Let me fork it and do a PR.

@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 7, 2018

Create PR 390 below.

#390

@royrusso royrusso modified the milestones: 3.5.0, 3.4.0 May 7, 2018

@royrusso

This comment has been minimized.

Copy link
Member

royrusso commented May 7, 2018

I merged the PR in to develop for now. It'll go out with the next release (this week). Thanks!

@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 7, 2018

Thanks.

@aderumeau

This comment has been minimized.

Copy link

aderumeau commented May 7, 2018

Great, this should work with elassandra (elasticsearch+cassandra).
Thanks.

@royrusso royrusso closed this May 14, 2018

@alex-rad

This comment has been minimized.

Copy link

alex-rad commented May 16, 2018

@chngtrn can you give us instruction on how to fix the SSL connection ? I tried to modify the 3 files and also to download the dev version but it didn't worked.

Thanks

@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 16, 2018

@alex-rad Make sure your dev branch has the changes made in pull request below. Then run the app with --enable-ssl and --ca-certs flags like below. Your CA file must be the signer of your ES nodes otherwise you'll get untrust errors.

python -m application --enable-ssl --ca-certs /path/to/your/ca.crt

https://github.com/ElasticHQ/elasticsearch-HQ/pull/390/files

@royrusso

This comment has been minimized.

Copy link
Member

royrusso commented May 16, 2018

These changes should now be part of the master branch. The release from yesterday, 3.4.0, should have the ssl additions merged in now also.

@alex-rad

This comment has been minimized.

Copy link

alex-rad commented May 16, 2018

@chngtrn @royrusso Thanks for the help. Please tell me, this implementation works also with Search Guard (like https://user:pass@ip:port on Elastic Search) ?

Regards

@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 16, 2018

It definitely works with Searh Guard. We use SG. You should test it with admin user and narrow it down from there. If it's SSL error you'll see it in the console output.

I would test talking to your ES cluster using curl with your CA certificate to see if it's working. Something like the command below.

curl -u admin:password --cacert /path/to/ca.crt https://localhost:9200/_cluster/settings?pretty

@alex-rad

This comment has been minimized.

Copy link

alex-rad commented May 16, 2018

Ok, I figure it out using the curl command. The certificate is generated on localhost and I tried to connect to a different server. In this case I did a port fw tunnel to connect it like it's on localhost:

[root@ip-172-31-20-169 elasticsearch-HQ-master]curl -u admin:pass --cacert /home/centos/root-ca.pem https://localhost:9200/_cluster/settings?pretty
{
"persistent" : { },
"transient" : { }
}

Then I tried to connect on ElasticHQ also using localhost, but it doesn't seem to work, using the master branch, downloaded todayy:

[root@ip-172-31-20-169 elasticsearch-HQ-master]# python3.6 -m application --enable-ssl --ca-certs /home/centos/root-ca.pem

config settings.json not found, searched /etc/elastic-hq/settings.json,~/settings.json,/root/elasticsearch-HQ-master/settings.json,/root/elasticsearch-HQ-master/elastichq/settings.json,/root/elasticsearch-HQ-master/elastichq/config/settings.json
loading config /root/elasticsearch-HQ-master/elastichq/config/logger.json
2018-05-16 16:02:19,512 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._cursor_execute:1254 	 SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1
2018-05-16 16:02:19,512 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._cursor_execute:1255 	 ()
2018-05-16 16:02:19,513 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._cursor_execute:1254 	 SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1
2018-05-16 16:02:19,513 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._cursor_execute:1255 	 ()
2018-05-16 16:02:19,514 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1151 	 PRAGMA table_info("cluster")
2018-05-16 16:02:19,514 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1154 	 ()
2018-05-16 16:02:19,517 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._begin_impl:682 	 BEGIN (implicit)
2018-05-16 16:02:19,517 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1151 	 SELECT cluster.cluster_name AS cluster_cluster_name, cluster.cluster_ip AS cluster_cluster_ip, cluster.cluster_port AS cluster_cluster_port, cluster.cluster_scheme AS cluster_cluster_scheme, cluster.cluster_version AS cluster_cluster_version, cluster.cluster_username AS cluster_cluster_username, cluster.cluster_password AS cluster_cluster_password 
FROM cluster
2018-05-16 16:02:19,518 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1154 	 ()
2018-05-16 16:02:19,523 	 INFO 	 engineio 	 server.__init__:132 	 Server initialized for eventlet.
2018-05-16 16:02:35,140 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._begin_impl:682 	 BEGIN (implicit)
2018-05-16 16:02:35,140 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1151 	 SELECT cluster.cluster_name AS cluster_cluster_name, cluster.cluster_ip AS cluster_cluster_ip, cluster.cluster_port AS cluster_cluster_port, cluster.cluster_scheme AS cluster_cluster_scheme, cluster.cluster_version AS cluster_cluster_version, cluster.cluster_username AS cluster_cluster_username, cluster.cluster_password AS cluster_cluster_password 
FROM cluster
2018-05-16 16:02:35,140 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1154 	 ()
2018-05-16 16:02:35,142 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._rollback_impl:702 	 ROLLBACK
2018-05-16 16:02:35,172 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._begin_impl:682 	 BEGIN (implicit)
2018-05-16 16:02:35,173 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1151 	 SELECT cluster.cluster_name AS cluster_cluster_name, cluster.cluster_ip AS cluster_cluster_ip, cluster.cluster_port AS cluster_cluster_port, cluster.cluster_scheme AS cluster_cluster_scheme, cluster.cluster_version AS cluster_cluster_version, cluster.cluster_username AS cluster_cluster_username, cluster.cluster_password AS cluster_cluster_password 
FROM cluster
2018-05-16 16:02:35,173 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1154 	 ()
2018-05-16 16:02:35,174 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._rollback_impl:702 	 ROLLBACK
2018-05-16 16:02:35,208 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._begin_impl:682 	 BEGIN (implicit)
2018-05-16 16:02:35,208 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1151 	 SELECT cluster.cluster_name AS cluster_cluster_name, cluster.cluster_ip AS cluster_cluster_ip, cluster.cluster_port AS cluster_cluster_port, cluster.cluster_scheme AS cluster_cluster_scheme, cluster.cluster_version AS cluster_cluster_version, cluster.cluster_username AS cluster_cluster_username, cluster.cluster_password AS cluster_cluster_password 
FROM cluster
2018-05-16 16:02:35,209 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1154 	 ()
2018-05-16 16:02:35,210 	 DEBUG 	 elastichq 	 status.get:63 	 {"name": "ElasticHQ", "installed_version": "3.4.0", "current_stable_version": "3.4.0", "tagline": "You know, for Elasticsearch", "clusters": [], "default_url": "http://localhost:9200"}
2018-05-16 16:02:35,210 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._rollback_impl:702 	 ROLLBACK
--- Logging error ---
Traceback (most recent call last):
  File "/root/elasticsearch-HQ-master/elastichq/service/ConnectionService.py", line 62, in create_connection
    timeout=REQUEST_TIMEOUT, verify=ca_certs)
  File "/usr/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
    conn.connect()
  File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 314, in connect
    cert_reqs=resolve_cert_reqs(self.cert_reqs),
  File "/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 269, in create_urllib3_context
    context.options |= options
  File "/usr/lib64/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/lib64/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/lib64/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 305 more times]
RecursionError: maximum recursion depth exceeded

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.6/logging/__init__.py", line 992, in emit
    msg = self.format(record)
  File "/usr/lib64/python3.6/logging/__init__.py", line 838, in format
    return fmt.format(record)
  File "/usr/lib64/python3.6/logging/__init__.py", line 575, in format
    record.message = record.getMessage()
  File "/usr/lib64/python3.6/logging/__init__.py", line 338, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/usr/lib/python3.6/site-packages/eventlet/greenthread.py", line 218, in main
    result = function(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 781, in process_request
    proto.__init__(conn_state, self)
  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 335, in __init__
    self.handle()
  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 368, in handle
    self.handle_one_request()
  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 442, in handle_one_request
    self.handle_one_response()
  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 539, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python3.6/site-packages/flask_socketio/__init__.py", line 43, in __call__
    start_response)
  File "/usr/lib/python3.6/site-packages/engineio/middleware.py", line 49, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib64/python3.6/site-packages/flask_restful/__init__.py", line 480, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/lib64/python3.6/site-packages/flask_restful/__init__.py", line 595, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/root/elasticsearch-HQ-master/elastichq/common/exceptions.py", line 29, in _request_wrapper
    return functor(*args, **kwargs)
  File "/root/elasticsearch-HQ-master/elastichq/api/clusters.py", line 121, in post
    enable_ssl=enable_ssl, ca_certs=ca_certs)
  File "/root/elasticsearch-HQ-master/elastichq/service/ConnectionService.py", line 116, in create_connection
    LOG.error(message, ex)
Message: 'Unable to create connection to: https://localhost:9200'
Arguments: (RecursionError('maximum recursion depth exceeded',),)
--- Logging error ---
Traceback (most recent call last):
  File "/root/elasticsearch-HQ-master/elastichq/service/ConnectionService.py", line 62, in create_connection
    timeout=REQUEST_TIMEOUT, verify=ca_certs)
  File "/usr/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
    conn.connect()
  File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 314, in connect
    cert_reqs=resolve_cert_reqs(self.cert_reqs),
  File "/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 269, in create_urllib3_context
    context.options |= options
  File "/usr/lib64/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/lib64/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/lib64/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 305 more times]
RecursionError: maximum recursion depth exceeded

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.6/logging/handlers.py", line 71, in emit
    if self.shouldRollover(record):
  File "/usr/lib64/python3.6/logging/handlers.py", line 187, in shouldRollover
    msg = "%s\n" % self.format(record)
  File "/usr/lib64/python3.6/logging/__init__.py", line 838, in format
    return fmt.format(record)
  File "/usr/lib64/python3.6/logging/__init__.py", line 575, in format
    record.message = record.getMessage()
  File "/usr/lib64/python3.6/logging/__init__.py", line 338, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/usr/lib/python3.6/site-packages/eventlet/greenthread.py", line 218, in main
    result = function(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 781, in process_request
    proto.__init__(conn_state, self)
  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 335, in __init__
    self.handle()
  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 368, in handle
    self.handle_one_request()
  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 442, in handle_one_request
    self.handle_one_response()
  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 539, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python3.6/site-packages/flask_socketio/__init__.py", line 43, in __call__
    start_response)
  File "/usr/lib/python3.6/site-packages/engineio/middleware.py", line 49, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib64/python3.6/site-packages/flask_restful/__init__.py", line 480, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/lib64/python3.6/site-packages/flask_restful/__init__.py", line 595, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/root/elasticsearch-HQ-master/elastichq/common/exceptions.py", line 29, in _request_wrapper
    return functor(*args, **kwargs)
  File "/root/elasticsearch-HQ-master/elastichq/api/clusters.py", line 121, in post
    enable_ssl=enable_ssl, ca_certs=ca_certs)
  File "/root/elasticsearch-HQ-master/elastichq/service/ConnectionService.py", line 116, in create_connection
    LOG.error(message, ex)
Message: 'Unable to create connection to: https://localhost:9200'
Arguments: (RecursionError('maximum recursion depth exceeded',),)
2018-05-16 16:02:38,663 	 ERROR 	 elastichq 	 exceptions._request_wrapper:37 	 Oops! Something bad happened.
Traceback (most recent call last):
  File "/root/elasticsearch-HQ-master/elastichq/common/exceptions.py", line 29, in _request_wrapper
    return functor(*args, **kwargs)
  File "/root/elasticsearch-HQ-master/elastichq/api/clusters.py", line 121, in post
    enable_ssl=enable_ssl, ca_certs=ca_certs)
  File "/root/elasticsearch-HQ-master/elastichq/service/ConnectionService.py", line 117, in create_connection
    raise ex
  File "/root/elasticsearch-HQ-master/elastichq/service/ConnectionService.py", line 62, in create_connection
    timeout=REQUEST_TIMEOUT, verify=ca_certs)
  File "/usr/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
    conn.connect()
  File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 314, in connect
    cert_reqs=resolve_cert_reqs(self.cert_reqs),
  File "/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 269, in create_urllib3_context
    context.options |= options
  File "/usr/lib64/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/lib64/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/lib64/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 305 more times]
RecursionError: maximum recursion depth exceeded

I'm now about to launch another instance of ES and install ElasticHQ locally to test it on localhost.

@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 16, 2018

Oh. I've ran into that recursion error too. It's the problem with python 3.6. You have to run it using 3.4 or 3.5. I used 3.4 and it's fine.

@royrusso

This comment has been minimized.

Copy link
Member

royrusso commented May 16, 2018

I'll create an issue to test the recursion error in 3.6+. That's a very strange error, as it's happening in a pretty simple requests call.

@alex-rad

This comment has been minimized.

Copy link

alex-rad commented May 16, 2018

Thanks guys. I tried on a localhost node of ES, with python3.4. Indeed, the recursion error disappeared, but still not working on SSL. I tried with Search Guard deactivated and it works fine.

root@ip-172-31-32-48 elastic-hq]# python3.4 -m application --enable-ssl --ca-certs /etc/elasticsearch/root-ca.pem 
config settings.json not found, searched /etc/elastic-hq/settings.json,~/settings.json,/etc/elastic-hq/settings.json,/etc/elastic-hq/elastichq/settings.json,/etc/elastic-hq/elastichq/config/settings.json
loading config /etc/elastic-hq/elastichq/config/logger.json
2018-05-16 17:25:59,476 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._cursor_execute:1254 	 SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1
2018-05-16 17:25:59,477 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._cursor_execute:1255 	 ()
2018-05-16 17:25:59,477 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._cursor_execute:1254 	 SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1
2018-05-16 17:25:59,477 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._cursor_execute:1255 	 ()
2018-05-16 17:25:59,478 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1151 	 PRAGMA table_info("cluster")
2018-05-16 17:25:59,478 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1154 	 ()
2018-05-16 17:25:59,481 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._begin_impl:682 	 BEGIN (implicit)
2018-05-16 17:25:59,482 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1151 	 SELECT cluster.cluster_name AS cluster_cluster_name, cluster.cluster_ip AS cluster_cluster_ip, cluster.cluster_port AS cluster_cluster_port, cluster.cluster_scheme AS cluster_cluster_scheme, cluster.cluster_version AS cluster_cluster_version, cluster.cluster_username AS cluster_cluster_username, cluster.cluster_password AS cluster_cluster_password 
FROM cluster
2018-05-16 17:25:59,482 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1154 	 ()
2018-05-16 17:25:59,494 	 INFO 	 engineio 	 server.__init__:132 	 Server initialized for eventlet.
2018-05-16 17:28:16,154 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._begin_impl:682 	 BEGIN (implicit)
2018-05-16 17:28:16,155 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1151 	 SELECT cluster.cluster_name AS cluster_cluster_name, cluster.cluster_ip AS cluster_cluster_ip, cluster.cluster_port AS cluster_cluster_port, cluster.cluster_scheme AS cluster_cluster_scheme, cluster.cluster_version AS cluster_cluster_version, cluster.cluster_username AS cluster_cluster_username, cluster.cluster_password AS cluster_cluster_password 
FROM cluster
2018-05-16 17:28:16,155 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1154 	 ()
2018-05-16 17:28:16,161 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._rollback_impl:702 	 ROLLBACK
2018-05-16 17:28:16,299 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._begin_impl:682 	 BEGIN (implicit)
2018-05-16 17:28:16,300 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1151 	 SELECT cluster.cluster_name AS cluster_cluster_name, cluster.cluster_ip AS cluster_cluster_ip, cluster.cluster_port AS cluster_cluster_port, cluster.cluster_scheme AS cluster_cluster_scheme, cluster.cluster_version AS cluster_cluster_version, cluster.cluster_username AS cluster_cluster_username, cluster.cluster_password AS cluster_cluster_password 
FROM cluster
2018-05-16 17:28:16,300 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1154 	 ()
2018-05-16 17:28:16,305 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._rollback_impl:702 	 ROLLBACK
2018-05-16 17:28:16,428 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._begin_impl:682 	 BEGIN (implicit)
2018-05-16 17:28:16,429 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1151 	 SELECT cluster.cluster_name AS cluster_cluster_name, cluster.cluster_ip AS cluster_cluster_ip, cluster.cluster_port AS cluster_cluster_port, cluster.cluster_scheme AS cluster_cluster_scheme, cluster.cluster_version AS cluster_cluster_version, cluster.cluster_username AS cluster_cluster_username, cluster.cluster_password AS cluster_cluster_password 
FROM cluster
2018-05-16 17:28:16,429 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._execute_context:1154 	 ()
2018-05-16 17:28:16,435 	 DEBUG 	 elastichq 	 status.get:63 	 {"current_stable_version": "3.4.0", "tagline": "You know, for Elasticsearch", "installed_version": "3.4.0", "default_url": "http://localhost:9200", "clusters": [{"cluster_name": "es-cluster", "cluster_ip": "localhost", "cluster_port": "9200", "cluster_scheme": "http", "cluster_connected": false, "cluster_host": "http://localhost:9200", "cluster_version": "6.2.4", "cluster_health": null, "cluster_settings": null}], "name": "ElasticHQ"}
2018-05-16 17:28:16,436 	 INFO 	 sqlalchemy.engine.base.Engine 	 base._rollback_impl:702 	 ROLLBACK
--- Logging error ---
Traceback (most recent call last):
  File "/etc/elastic-hq/elastichq/service/ConnectionService.py", line 62, in create_connection
    timeout=REQUEST_TIMEOUT, verify=ca_certs)
  File "/usr/lib/python3.4/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python3.4/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.4/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 346, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
    conn.connect()
  File "/usr/lib/python3.4/site-packages/urllib3/connection.py", line 337, in connect
    cert = self.sock.getpeercert()
  File "/usr/lib64/python3.4/ssl.py", line 656, in getpeercert
    return self._sslobj.peer_certificate(binary_form)
SystemError: error return without exception set

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.4/logging/__init__.py", line 978, in emit
    msg = self.format(record)
  File "/usr/lib64/python3.4/logging/__init__.py", line 828, in format
    return fmt.format(record)
  File "/usr/lib64/python3.4/logging/__init__.py", line 565, in format
    record.message = record.getMessage()
  File "/usr/lib64/python3.4/logging/__init__.py", line 328, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/usr/lib/python3.4/site-packages/eventlet/greenthread.py", line 218, in main
    result = function(*args, **kwargs)
  File "/usr/lib/python3.4/site-packages/eventlet/wsgi.py", line 781, in process_request
    proto.__init__(conn_state, self)
  File "/usr/lib/python3.4/site-packages/eventlet/wsgi.py", line 335, in __init__
    self.handle()
  File "/usr/lib/python3.4/site-packages/eventlet/wsgi.py", line 368, in handle
    self.handle_one_request()
  File "/usr/lib/python3.4/site-packages/eventlet/wsgi.py", line 442, in handle_one_request
    self.handle_one_response()
  File "/usr/lib/python3.4/site-packages/eventlet/wsgi.py", line 539, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/python3.4/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python3.4/site-packages/flask_socketio/__init__.py", line 43, in __call__
    start_response)
  File "/usr/lib/python3.4/site-packages/engineio/middleware.py", line 49, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python3.4/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.4/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.4/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib64/python3.4/site-packages/flask_restful/__init__.py", line 480, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/lib/python3.4/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/lib64/python3.4/site-packages/flask_restful/__init__.py", line 595, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/etc/elastic-hq/elastichq/common/exceptions.py", line 29, in _request_wrapper
    return functor(*args, **kwargs)
  File "/etc/elastic-hq/elastichq/api/clusters.py", line 121, in post
    enable_ssl=enable_ssl, ca_certs=ca_certs)
  File "/etc/elastic-hq/elastichq/service/ConnectionService.py", line 116, in create_connection
    LOG.error(message, ex)
Message: 'Unable to create connection to: https://localhost:9200'
Arguments: (SystemError('error return without exception set',),)
--- Logging error ---
Traceback (most recent call last):
  File "/etc/elastic-hq/elastichq/service/ConnectionService.py", line 62, in create_connection
    timeout=REQUEST_TIMEOUT, verify=ca_certs)
  File "/usr/lib/python3.4/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python3.4/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.4/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 346, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
    conn.connect()
  File "/usr/lib/python3.4/site-packages/urllib3/connection.py", line 337, in connect
    cert = self.sock.getpeercert()
  File "/usr/lib64/python3.4/ssl.py", line 656, in getpeercert
    return self._sslobj.peer_certificate(binary_form)
SystemError: error return without exception set

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.4/logging/handlers.py", line 71, in emit
    if self.shouldRollover(record):
  File "/usr/lib64/python3.4/logging/handlers.py", line 187, in shouldRollover
    msg = "%s\n" % self.format(record)
  File "/usr/lib64/python3.4/logging/__init__.py", line 828, in format
    return fmt.format(record)
  File "/usr/lib64/python3.4/logging/__init__.py", line 565, in format
    record.message = record.getMessage()
  File "/usr/lib64/python3.4/logging/__init__.py", line 328, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/usr/lib/python3.4/site-packages/eventlet/greenthread.py", line 218, in main
    result = function(*args, **kwargs)
  File "/usr/lib/python3.4/site-packages/eventlet/wsgi.py", line 781, in process_request
    proto.__init__(conn_state, self)
  File "/usr/lib/python3.4/site-packages/eventlet/wsgi.py", line 335, in __init__
    self.handle()
  File "/usr/lib/python3.4/site-packages/eventlet/wsgi.py", line 368, in handle
    self.handle_one_request()
  File "/usr/lib/python3.4/site-packages/eventlet/wsgi.py", line 442, in handle_one_request
    self.handle_one_response()
  File "/usr/lib/python3.4/site-packages/eventlet/wsgi.py", line 539, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/python3.4/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python3.4/site-packages/flask_socketio/__init__.py", line 43, in __call__
    start_response)
  File "/usr/lib/python3.4/site-packages/engineio/middleware.py", line 49, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python3.4/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.4/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.4/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib64/python3.4/site-packages/flask_restful/__init__.py", line 480, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/lib/python3.4/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/lib64/python3.4/site-packages/flask_restful/__init__.py", line 595, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/etc/elastic-hq/elastichq/common/exceptions.py", line 29, in _request_wrapper
    return functor(*args, **kwargs)
  File "/etc/elastic-hq/elastichq/api/clusters.py", line 121, in post
    enable_ssl=enable_ssl, ca_certs=ca_certs)
  File "/etc/elastic-hq/elastichq/service/ConnectionService.py", line 116, in create_connection
    LOG.error(message, ex)
Message: 'Unable to create connection to: https://localhost:9200'
Arguments: (SystemError('error return without exception set',),)
2018-05-16 17:28:21,647 	 ERROR 	 elastichq 	 exceptions._request_wrapper:37 	 Oops! Something bad happened.
Traceback (most recent call last):
  File "/etc/elastic-hq/elastichq/common/exceptions.py", line 29, in _request_wrapper
    return functor(*args, **kwargs)
  File "/etc/elastic-hq/elastichq/api/clusters.py", line 121, in post
    enable_ssl=enable_ssl, ca_certs=ca_certs)
  File "/etc/elastic-hq/elastichq/service/ConnectionService.py", line 117, in create_connection
    raise ex
  File "/etc/elastic-hq/elastichq/service/ConnectionService.py", line 62, in create_connection
    timeout=REQUEST_TIMEOUT, verify=ca_certs)
  File "/usr/lib/python3.4/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python3.4/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.4/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 346, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3.4/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
    conn.connect()
  File "/usr/lib/python3.4/site-packages/urllib3/connection.py", line 337, in connect
    cert = self.sock.getpeercert()
  File "/usr/lib64/python3.4/ssl.py", line 656, in getpeercert
    return self._sslobj.peer_certificate(binary_form)
SystemError: error return without exception set

[root@ip-172-31-32-48 elastic-hq]curl -u admin:pass --cacert /etc/elasticsearch/root-ca.pem https://localhost:9200/_cluster/settings?pretty
{
  "persistent" : { },
  "transient" : { }
}
@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 16, 2018

Can you paste your SG config in elasticsearch.yml and do command below on your CA and node certs?

openssl x509 -in ca.crt -text -noout
openssl x509 -in node.crt -text -noout

@alex-rad

This comment has been minimized.

Copy link

alex-rad commented May 16, 2018

Sure. I'm using the default certs generated by SG

searchguard.ssl.transport.pemcert_filepath: esnode.pem
searchguard.ssl.transport.pemkey_filepath: esnode-key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: esnode.pem
searchguard.ssl.http.pemkey_filepath: esnode-key.pem
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.allow_unsafe_democertificates: true
searchguard.allow_default_init_sgindex: true
searchguard.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["sg_all_access"]
cluster.routing.allocation.disk.threshold_enabled: false
network.host: 0.0.0.0
discovery.zen.minimum_master_nodes: 1
node.max_local_storage_nodes: 3

[root@ip-172-31-32-48 elasticsearch]# openssl x509 -in root-ca.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1 (0x1)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: DC=com, DC=example, O=Example Com Inc., OU=Example Com Inc. Root CA, CN=Example Com Inc. Root CA
        Validity
            Not Before: Apr 22 03:43:46 2018 GMT
            Not After : Apr 19 03:43:46 2028 GMT
        Subject: DC=com, DC=example, O=Example Com Inc., OU=Example Com Inc. Root CA, CN=Example Com Inc. Root CA
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:af:ee:f8:60:11:3f:98:a7:9e:1a:57:2b:47:34:
                    ab:bb:35:4a:ed:55:4c:46:88:82:66:6b:f1:55:88:
                    e9:2f:1a:99:fe:5c:53:79:2d:57:b5:93:f5:2b:95:
                    4e:c0:26:da:2d:80:e4:ff:82:b7:0e:e2:66:47:e7:
                    1d:69:6c:0b:71:e1:3d:47:1d:ea:6b:f3:19:9e:26:
                    a3:19:da:98:ce:eb:f9:af:68:b5:1a:77:a3:06:28:
                    19:2b:57:ca:55:53:42:eb:00:8d:ba:bd:76:8f:02:
                    31:5e:21:70:14:de:a4:27:7e:d3:0d:2f:e2:1e:94:
                    95:75:3c:c6:38:63:d7:17:94:23:3e:03:29:b4:60:
                    7f:7e:aa:d2:bb:f8:54:85:f8:e9:7e:f6:ac:c2:52:
                    11:32:8e:4b:1b:b0:2e:4a:2f:d5:93:95:6d:f4:a5:
                    3d:ac:a0:5c:8c:6a:b0:75:65:8f:58:8c:91:84:5b:
                    42:66:93:89:be:97:58:72:9f:32:26:c3:6a:a0:de:
                    8c:e8:6e:92:40:a3:ce:9a:6d:19:93:8f:15:0f:34:
                    d3:65:2d:4d:33:6f:d5:38:9a:2b:19:23:31:02:4d:
                    c3:3e:a3:7d:9e:77:c2:cd:df:87:52:34:45:64:fa:
                    59:f3:38:a1:e1:51:16:7c:85:46:67:38:b9:84:d4:
                    80:09
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Authority Key Identifier: 
                keyid:92:35:0C:E0:0F:1E:2B:45:F6:4D:39:F3:7B:5F:A2:E6:12:97:40:73

            X509v3 Subject Key Identifier: 
                92:35:0C:E0:0F:1E:2B:45:F6:4D:39:F3:7B:5F:A2:E6:12:97:40:73
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
    Signature Algorithm: sha256WithRSAEncryption
         68:40:7b:f0:b1:1d:f8:84:63:b6:9b:ca:95:47:d9:d0:e4:a9:
         68:e4:76:0f:c9:de:b2:48:a3:5c:4f:7e:8e:67:80:10:7d:a0:
         86:b2:4f:92:79:c4:e2:df:94:05:44:72:f7:83:6a:9f:7c:40:
         f8:b4:a4:74:44:13:46:41:28:22:2d:ab:e6:1c:60:a1:dd:8a:
         43:ba:92:aa:db:18:61:11:e4:bd:a0:19:90:cf:16:a7:17:05:
         85:a1:de:13:9e:7b:06:d9:c0:9e:8f:24:7d:59:7a:11:cc:78:
         ac:c3:42:89:59:eb:8d:97:08:d4:74:96:34:c6:79:f5:ea:ca:
         e6:d7:32:ff:33:f7:f4:3e:f0:b2:87:d1:d4:d6:61:75:8a:f9:
         ce:4a:a6:c3:0f:66:7b:25:21:b2:72:48:0a:69:dd:4e:9a:c8:
         3f:ae:be:57:62:d0:9e:c8:97:97:50:f7:26:a5:e5:fa:7a:b5:
         89:24:d4:d4:87:ac:96:0b:f8:58:1b:f7:45:0b:8c:6b:26:17:
         d7:c3:3e:99:d3:2b:54:ca:02:4e:df:66:c9:1b:83:69:da:21:
         80:c2:fb:e0:23:d1:1f:c7:31:2f:fc:a3:fe:14:6a:c9:3a:f4:
         09:02:ae:3f:05:4d:fa:64:06:bc:d1:6f:fc:4c:19:ea:65:39:
         0d:9e:a6:55

[root@ip-172-31-32-48 elasticsearch]# openssl x509 -in esnode.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1524368626614 (0x162eb7353b6)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: DC=com, DC=example, O=Example Com Inc., OU=Example Com Inc. Root CA, CN=Example Com Inc. Root CA
        Validity
            Not Before: Apr 22 03:43:47 2018 GMT
            Not After : Apr 19 03:43:47 2028 GMT
        Subject: DC=de, L=test, O=node, OU=node, CN=node-0.example.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:96:be:7f:8e:fa:bc:b0:7e:03:02:b9:dd:b8:98:
                    07:25:30:37:ee:34:0f:c8:cc:22:8b:c6:5e:6e:b0:
                    81:3f:3e:f5:26:ec:f3:df:5d:0d:78:2d:f4:21:35:
                    05:ea:3a:e6:83:f5:f8:95:33:e1:ce:d4:1c:ca:c2:
                    63:77:8f:88:3b:78:72:27:47:57:31:10:da:0d:18:
                    a1:5a:d0:5a:fd:11:79:d4:bf:cb:1f:c3:2a:1b:3c:
                    3f:0d:4e:ef:5e:68:7e:d3:f9:de:9f:f6:8a:30:f9:
                    0e:27:c5:bf:57:8a:7e:48:45:1f:e9:70:9f:2f:ef:
                    31:23:71:7a:59:69:97:a3:71:25:38:89:56:74:3d:
                    1d:83:8b:81:fd:ad:f7:bd:48:4c:91:e7:02:eb:b1:
                    50:5e:3c:1d:cb:8d:a2:f5:b8:ae:1b:64:5d:e7:fc:
                    91:a0:0d:ed:c1:37:2d:4f:80:f5:3e:3b:e1:42:cd:
                    08:a9:04:14:f2:25:64:02:8d:de:22:4d:15:d5:6c:
                    c6:b4:d4:f8:25:01:1f:39:3b:dc:3a:35:70:29:04:
                    bc:96:74:64:58:e9:d1:9d:f2:f3:02:d8:fe:0a:96:
                    19:f1:95:c8:0f:65:d8:25:2a:78:86:4d:7f:9e:4f:
                    34:fb:46:cc:ea:ef:bc:e3:62:ba:2e:3c:bc:12:87:
                    d4:bb
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Authority Key Identifier: 
                keyid:92:35:0C:E0:0F:1E:2B:45:F6:4D:39:F3:7B:5F:A2:E6:12:97:40:73
                DirName:/DC=com/DC=example/O=Example Com Inc./OU=Example Com Inc. Root CA/CN=Example Com Inc. Root CA
                serial:01

            X509v3 Subject Key Identifier: 
                AC:AF:EF:C6:66:16:35:4A:33:D8:3B:A4:C0:A8:9D:81:FB:15:50:47
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Key Usage: critical
                Digital Signature, Non Repudiation, Key Encipherment
            X509v3 Extended Key Usage: critical
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Subject Alternative Name: 
                Registered ID:1.2.3.4.5.5, DNS:node-0.example.com, DNS:localhost, IP Address:127.0.0.1
    Signature Algorithm: sha256WithRSAEncryption
         83:8a:bb:25:ec:15:f1:af:d6:12:3f:2e:4a:5d:ff:bd:d0:36:
         a8:ea:25:dc:50:72:55:f9:ec:63:8c:58:d6:ce:33:91:f4:e7:
         ff:40:38:86:79:25:0b:16:50:b3:4a:37:be:da:1e:32:d1:af:
         8a:30:ab:68:c6:6e:97:3f:67:a9:00:77:e7:a2:6a:d9:1c:a6:
         76:ed:6c:6a:e1:2a:93:ad:a0:46:72:f0:ab:ac:97:09:1e:8b:
         1a:73:2c:33:48:49:26:e4:78:ba:57:cf:8c:49:23:51:13:30:
         df:d5:1b:c6:59:3f:56:e3:ce:51:f3:88:71:c3:bb:42:4d:67:
         a4:e7:37:32:ab:5f:30:86:30:2a:21:15:f7:a6:f2:f9:ca:36:
         72:94:9d:e3:10:32:f5:dd:de:bc:d4:68:08:2f:b5:fe:c7:73:
         62:d3:06:57:f8:7d:9c:d9:17:51:24:c1:d4:97:85:a3:00:d6:
         59:1f:1e:2a:8e:07:1b:60:78:32:f1:08:71:12:67:67:ea:81:
         5a:ac:59:7f:ad:de:a1:d0:7e:2b:dc:3d:6e:ad:c6:d0:f2:ac:
         53:d1:74:93:86:86:23:06:cd:3f:ed:7b:ff:64:90:0b:50:46:
         0a:53:6f:7b:24:61:d2:0e:39:43:95:d1:61:90:eb:49:09:94:
         58:40:cc:8b

@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 16, 2018

As you can see that the CN for the demo node cert is node-0.example.com which I'm pretty sure your ES node's hostname is not named that. My only other suggestion would be to create a CA and sign a cert for your ES nodes using the hostname of the ES nodes as then CN for each node and set searchguard.ssl.transport.enforce_hostname_verification: true and remove searchguard.allow_unsafe_democertificates: true line. Our instance with SG works fine; not using the demo certs that is.

@alex-rad

This comment has been minimized.

Copy link

alex-rad commented May 16, 2018

Thanks @chngtrn. I will try with new certificates. I also tried the node-0.example.com (after I set it as a hostname, of course) and it throws the same error (but anyway, the cert is also signed for localhost)

@royrusso

This comment has been minimized.

Copy link
Member

royrusso commented May 21, 2018

Just a small nit... the parameter is "ca-certs". The example above is using "cacert", which I don't think will work.

curl -u admin:password --cacert /path/to/ca.crt https://localhost:9200/_cluster/settings?pretty

@chngtrn

This comment has been minimized.

Copy link
Contributor

chngtrn commented May 21, 2018

That's the curl command parameter. The python application paramter is --ca-certs while curl us --cacert.

$ curl -h | grep cert
--cacert FILE CA certificate to verify peer against (SSL)

@royrusso

This comment has been minimized.

Copy link
Member

royrusso commented May 21, 2018

Ah. Sorry.. you're correct. I was building out the documentation and got confused. ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.