Skip to content

Latest commit

 

History

History
33 lines (21 loc) · 1.58 KB

elasticsearch.rst

File metadata and controls

33 lines (21 loc) · 1.58 KB
Raw

Using Logstash and ElasticSearch to Process Eliot Logs

Note

Logstash, Elasticsearch and Kibana change frequently. These instructions might not be quite accurate.

ElasticSearch is a search and analytics engine which can be used to store Eliot logging output. The logs can then be browsed by humans using the Kibana web UI, or on the command-line using the logstash-cli tool. Automated systems can access the logs using the ElasticSearch query API. Logstash is a log processing tool that can be used to load Eliot log files into ElasticSearch. The combination of ElasticSearch, Logstash, and Kibana is sometimes referred to as ELK.

Example Logstash Configuration

Assuming each Eliot message is written out as a JSON message on its own line (which is the case for eliot.to_file() and eliot.logwriter.ThreadedFileWriter), the following Logstash configuration will load these log messages into an in-process ElasticSearch database:

logstash_standalone.conf

logstash_standalone.conf

We can then pipe JSON messages from Eliot into ElasticSearch using Logstash:

$ python examples/stdout.py | logstash web -- agent --config logstash_standalone.conf

You can then use the Kibana UI to search and browse the logs by visiting http://localhost:9292/.