Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ELK example needs updating for modern usage #196

Closed
geerlingguy opened this issue Jan 12, 2020 · 6 comments
Closed

ELK example needs updating for modern usage #196

geerlingguy opened this issue Jan 12, 2020 · 6 comments

Comments

@geerlingguy
Copy link
Owner

Currently it's using outdated versions of some of the roles, and is using logstash-forwarder, which has been superseded (in 2016 mind you) by Filebeat... (see https://github.com/elastic/logstash-forwarder).

The example also uses Ubuntu 16.04 (but in the text it's actually testing on 12.04, yikes!), so it should be updated for 18.04.

Lots of little fixes to do to make sure it's all working well with ELK. Also considering changing the name to efk (for elasticsearch-filebeat-kibana), and maybe even using Fluentd instead of Filebeat? Or maybe just stick to Filebeat for now.

@geerlingguy
Copy link
Owner Author

I've updated my logstash and filebeat roles to the latest 7.x versions, since they were kind of broken with the latest releases. I think I'll also move the ELK example over to this book's repository, since it was kind of outcast over in the ansible-vagrant-examples repo (and is not tested automatically, shame on me!).

@geerlingguy
Copy link
Owner Author

Going to leave name as ELK for now for history and for the fact it's still using Logstash technically.

@geerlingguy
Copy link
Owner Author

geerlingguy commented Jan 13, 2020

Ugh, when switching to Filebeat I ran into a few certificate issues:

  • Had to convert the private key to pkcs8 for the logstash key file (Private key file's format pkcs8 for logstash key spujadas/elk-docker#112 (comment))
  • Got routines:OPENSSL_internal:WRONG_VERSION_NUMBER after doing that
  • Found out that the filebeat.yml file needed some updating for a version in the 6.x release series, changing from tls to ssl for the config values under logstash.
  • To fix some DNS issues for testing locally, I'm updating the example to use logs.test instead of logs for the hostname. This requires re-creating the certs (which were created with CN=logs).

@geerlingguy
Copy link
Owner Author

Got that all working after updating all roles (elasticsearch, kibana, logstash, filebeat) to use 7.x by default, and messing about with all the config files to match updates to each of those applications.

Even wrote a tweet about it: https://twitter.com/geerlingguy/status/1216806260835864577

@geerlingguy
Copy link
Owner Author

Example is updated in the repo, tests are running correctly. Next step is to get the book text up to date (had to rework a bit with filebeat!).

@geerlingguy
Copy link
Owner Author

Done! Will be in next version of the book.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant