Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ELK example needs updating for modern usage #196

Closed
geerlingguy opened this issue Jan 12, 2020 · 6 comments
Closed

ELK example needs updating for modern usage #196

geerlingguy opened this issue Jan 12, 2020 · 6 comments

Comments

@geerlingguy
Copy link
Owner

@geerlingguy geerlingguy commented Jan 12, 2020

Currently it's using outdated versions of some of the roles, and is using logstash-forwarder, which has been superseded (in 2016 mind you) by Filebeat... (see https://github.com/elastic/logstash-forwarder).

The example also uses Ubuntu 16.04 (but in the text it's actually testing on 12.04, yikes!), so it should be updated for 18.04.

Lots of little fixes to do to make sure it's all working well with ELK. Also considering changing the name to efk (for elasticsearch-filebeat-kibana), and maybe even using Fluentd instead of Filebeat? Or maybe just stick to Filebeat for now.

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

@geerlingguy geerlingguy commented Jan 12, 2020

I've updated my logstash and filebeat roles to the latest 7.x versions, since they were kind of broken with the latest releases. I think I'll also move the ELK example over to this book's repository, since it was kind of outcast over in the ansible-vagrant-examples repo (and is not tested automatically, shame on me!).

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

@geerlingguy geerlingguy commented Jan 12, 2020

Going to leave name as ELK for now for history and for the fact it's still using Logstash technically.

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

@geerlingguy geerlingguy commented Jan 13, 2020

Ugh, when switching to Filebeat I ran into a few certificate issues:

  • Had to convert the private key to pkcs8 for the logstash key file (spujadas/elk-docker#112 (comment))
  • Got routines:OPENSSL_internal:WRONG_VERSION_NUMBER after doing that
  • Found out that the filebeat.yml file needed some updating for a version in the 6.x release series, changing from tls to ssl for the config values under logstash.
  • To fix some DNS issues for testing locally, I'm updating the example to use logs.test instead of logs for the hostname. This requires re-creating the certs (which were created with CN=logs).
@geerlingguy

This comment has been minimized.

Copy link
Owner Author

@geerlingguy geerlingguy commented Jan 13, 2020

Got that all working after updating all roles (elasticsearch, kibana, logstash, filebeat) to use 7.x by default, and messing about with all the config files to match updates to each of those applications.

Even wrote a tweet about it: https://twitter.com/geerlingguy/status/1216806260835864577

geerlingguy added a commit that referenced this issue Jan 13, 2020
geerlingguy added a commit that referenced this issue Jan 13, 2020
geerlingguy added a commit that referenced this issue Jan 13, 2020
@geerlingguy

This comment has been minimized.

Copy link
Owner Author

@geerlingguy geerlingguy commented Jan 13, 2020

Example is updated in the repo, tests are running correctly. Next step is to get the book text up to date (had to rework a bit with filebeat!).

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

@geerlingguy geerlingguy commented Jan 15, 2020

Done! Will be in next version of the book.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.