Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

README.md

Centralised Logging

This is some sample code of how to do centralised logging. So you can get something as awesome as this: http://demo.kibana.org/

Tech used:

  • Logback logstash json encoder: write json format to your logs so you can handle multiline logging with ease, e.g. stack traces.
  • Logstash forwarder: Lightweight client to forward logs, so you don't have to run the Logstash client in another jvm.
  • Logstash: Logstash to receive your logs and dump them in elasticsearch.
  • ElasticSearch: Store all the things.
  • Kibana: Visualization. Query all the things.
  • Apache reverse proxy: Kibana needs direct access to elasticsearch, so you want to redirect that through a reverse proxy for security.

This assumes you're working on Ubuntu servers. Multiple senders and a single receiver. Receiver will run Kibana too.

Receiver Installation

Sender Installation

<dependency>
  <groupId>net.logstash.logback</groupId>
  <artifactId>logstash-logback-encoder</artifactId>
  <version>2.4</version>
</dependency>
  • Dependency for Lein [net.logstash.logback/logstash-logback-encoder "2.4"]
  • This will write your logs out in json format.
  • Build and install Logstash forwarder. I recommend keeping a .deb on hand.
  • Create ~/lumberjack
  • Copy the same SSL certs you generated for the receiver and 'lumberjack.conf' into this directory.
  • Adjust the paths and receiving server in 'lumberjack.conf'
  • Drop 'upstart/lumberjack.conf' in /etc/init/. Change the 'deploy' user to the user you're using.
  • sudo service lumberjack start
  • Your logs should be sent to the receiver now.

Kibana Installation

This is a bit tricky. Kibana needs direct access to ElasticSearch. Logstash includes Kibana, so you can run it with 'java -jar logstash.jar web'. Or since Kibana is just javascript, dump it in your web server and configure a reverse proxy.

I've included a basic apache config as 'apache.conf' which you can drop in /etc/apache2/sites-available and enable. This uses SSL and HTTP basic auth on everything. /elasticsearch path reverse proxies to elasticsearch running on port 9200. And /kibana loads Kibana from /var/www/kibana.

About

An example of centralising clojure/java logging with Logback, LogStash, ElasticSearch, and Kibana

Resources

License

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.