New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chef Cookbook with TLS/SSL for Elasticsearch 6.2, Kibana, and Cerebro Available #293

Open
cmcconnell1 opened this Issue May 10, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@cmcconnell1
Copy link

cmcconnell1 commented May 10, 2018

Hello,

Thanks for your work on cerebro, we are now using it and wanted to give back.
I was recently working on getting Elasticsearch 6.2, Kibana, and Cerebro automated/deployed with TLS/SSL certs, keys, keystore, etc. for our internal infrastructure. It's not perfect by any means, but works for us and I'm wondering if it might be helpful for others?:

6.x Elasticsearch Chef Cookbook with TLS/SSL, Kibana, and Cerbro for management, etc.

Regarding the cerebro project, I didn't see a 'contrib' dir, and not sure you'd want one?
Additionally, the cookbook that I wrote is large with a lot of stuff you probably don't need (elasticsearch clusters and kibana coordinating nodes, etc.), but might be useful in getting 'something' out there for automating cerebro install/config for the latest Elasticsearch 6.x with x-pack and the new TLS requirements, etc. It's a bit tricky to get it right.

Quick Summary:
For cerebro, I based the automation on what I could find--this one was a gem:
Access To Elasticsearch With Cerebro Via SSL+LDAP

Overview of the Kibana and Cerebro components of that cookbook

Kibana (HTTPS/SSL) "Coordinating Node"

  • Recipe: td_elasticsearch::kibana
  • Installed and configured in systemd.
  • Post chef-client available at https URL--i.e.: https://kibana1.foo.com:5601
  • Redirects to kibana login/service; the kibana service terminates the TLS/SSL at the kibana app.

Cerbro (HTTPS/SSL) on the Kibana node

  • Recipe: td_elasticsearch::cerebro
  • Installed and configured in systemd.
  • Terminates TLS/SSL at the web service on port 443 and redirects to the cerebro play/java app running locally on port 9000
  • Post chef-client available at https URL--i.e.: https://kibana1.foo.com
  • Cerebro Redirect to Login page and use the following:
  • Node Address: 'kibana.foo.com:9200' (9200 is the elasticsearch port)
  • Username: use your elasticsearch credentials/account–i.e. 'kibana,' 'cmcc', etc.
  • Password: requisite password for the above elasticsearch login

Anyway, I am hoping that perhaps some of this work could be useful to other users of the cerebro project, and I could do a p/r with a ./contrib/chef dir and maybe some doc about automating it if desired. I haven't tried to decouple the cerebro recipe from the cookbook as it uses the code and files from them.

Thanks,
-cmcc

@abessifi

This comment has been minimized.

Copy link

abessifi commented Oct 12, 2018

but might be useful in getting 'something' out there for automating cerebro install/config for the latest Elasticsearch 6.x with x-pack and the new TLS requirements, etc. It's a bit tricky to get it right.

@cmcconnell1 do you mean that Cerebro is not easy to install for ES 6.x out of the box ?

@cmcconnell1

This comment has been minimized.

Copy link

cmcconnell1 commented Oct 12, 2018

Hey @abessifi (regarding my above comment five months ago)
Since ES 6.x introduced required TLS encryption with keys etc., it was a pretty big change from the previous major versions that we were running which didn't use TLS and didn't have to have CA's to run ES queries, etc. And you can't license a 6.x cluster unless you have the certs installed/configured etc. AIR at that time, the ES docs were lacking many requisite details, etc. And regarding cerebro, at that time, I didn't find any updated automated install CM (chef, etc.) cookbooks/playbooks available online that showed how to handle the requisite certs, etc. TLS is also a requirement for 6.x and onward for kibana, etc.
So, at that time of my above comment (five months ago) I don't think anyone had any working examples showing the certs requirements for a lot of these ES-stack related components. I chose to stick our certs in locked-down S3 buckets (which for us are created for each ES cluster before deploying and are available to chef nodes via IAM policy and roles). Chef and databags is not a good idea and encrypted data bags are even worse. Regarding ES stack and certs, I wasted a lot of time trying to hack our automation for 6.x ES stack to use (CSR) certs that were signed by Digi, only to find out that those would never work with ES--if I recall correctly we couldn't use our wildcard certs since they were missing the hostname, etc. That was also not in the docs. Also, at that time you could only use PEM formatted certs (i.e. PKCS12 wouldn't work && found out the hard way there too). Of course, this may have changed. YMMV. ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment