Skip to content
a system monitoring and log collector
Makefile Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
addons
beats
grafana
heka
systemd
telegraf
.dockerignore
.gitignore
ChangeLog.md
LICENSE
Makefile
README.md
docker-compose.yml

README.md

Vision

License Apache 2 GitHub version

Description

Vision is a stack for monitoring and logging. It provides :

  • Elasticsearch web interface : http://xxx:9200
  • Kibana web interface : http://xxx:9393
  • Grafana web interface : http://xxx:9999/
  • InfluxDB web interface : http://xxx:8083
  • Heka dashboard : http://xxx:4352

Installation

  • Download and install a release :

      $ curl https://github.com/nlamirault/vision/releases/download/x.y.z/vision-x.y.z-linux.tar.gz
      $ tar zxvf vision-x.y.z
      $ cd vision-x.y.z
      $ ./init.sh
    
  • Start services using Docker Compose:

      $ docker-compose up -d
    
  • Start services using SystemD:

      $ cp -r vision-*.service /lib/systemd/system/
    

Usage

Monitoring servers : Elasticsearch/Kibana/Beats

  • Install Topbeat and Packetbeat

  • Launch Elasticsearch and Kibana services

  • Loading the templates into Elasticsearch

      $ curl -XPUT 'http://localhost:9200/_template/packetbeat' \
          -d@beats/topbeat.template.json
    
      $ curl -XPUT 'http://localhost:9200/_template/packetbeat' \
          -d@beats/packetbeat.template.json
    
  • Running topbeat metrics :

      $ topbeat -c beats/topbeat.yml
    
  • Running packetbeat metrics :

      $ packetbeat -c beats/packetbeat.yml
    
  • Testing the installation:

      $ curl -XGET 'http://localhost:9200/topbeat-*/_search?pretty'
      $ curl -XGET 'http://localhost:9200/packetbeat-*/_search?pretty'
    
  • Loading Kibana dashboards:

      $ curl -L -O http://download.elastic.co/beats/dashboards/beats-dashboards-1.0.0.tar.gz
      $ tar zxvf beats-dashboards-1.0.0.tar.gz
      $ cd beats-dashboards-1.0.0
      $ ./load.sh
    
  • Then open the Kibana website (http://localhost:9393), then select Topbeat index, and open Topbeat dashboard. Do same with Packetbeat index and dashboard.

Monitoring servers : Telegraf/InfluxDB/Grafana

  • Install Telegraf

  • Launch InfluxDB and Grafana services

  • Running telegraf metrics :

      $ telegraf -config telegraf/telegraf.conf
    
  • Then open the Grafana dashboard (http://localhost:9191) and import the Vision Telegraf dashboard from (grafana/grafana-telegraf.json)

  • You could explore metrics into the InfluxDB UI on http://localhost:8083 with the query :

      SHOW MEASUREMENTS
    

Monitoring servers : Prometheus/Grafana

  • Install Prometheus and Prometheus node exporter

  • Launch services :

      $ ./prometheus -config.file=prometheus/vision.yml
      $ ./node_exporter
    
  • Check Prometheus installation on : http://localhost:9090 and http://localhost:9090/consoles/node.html

  • hen open the Grafana dashboard (http://localhost:9191) and import the Vision Prometheus dashboard from (grafana/grafana-prometheus.json)

Log analysis (Elasticsearch/Filebeat/Kibana)

  • Install Filebeat

  • Launch Elasticsearch and Kibana services

  • Loading the Index Template into Elasticsearch

      $ curl -XPUT 'http://localhost:9200/_template/packetbeat' \
          -d@beats/filebeat.template.json
    
  • Running filebeat metrics :

      $ filebeat -c beats/filebeat.yml
    

SystemD

You could use services files to launch Vision monitoring tools using SystemD.

$ sudo cp systemd/*.service /lib/systemd/system/
$ sudo systemctl enable vision-telegraf
$ sudo systemctl enable vision-topbeat
$ sudo systemctl enable vision-packetbeat

Development

  • Creates a virtual machine called vision-dev for the development environment :

      $ ./docker-machine create -d virtualbox vision-dev
      $ eval "$(./docker-machine env vision-dev)"
    
  • Check vision machine runnning :

      $ ./docker-machine ls
    
  • Launch vision :

      $ ./docker-compose up
    
  • Open your browser and navigate to the IP address associated with the vision virtual machine :

      $ ./docker-machine ip
    
  • To see which environment variables are available to the web service, run:

      $ ./docker-compose run web env
    

Deployment

With our app running locally, we can now push this exact same environment to a cloud hosting provider with Docker Machine

Set your credentials in your environment :

$ source XXXXXXX.sh

Deploy a new instance :

$ docker-machine -D create -d digitalocean \
    --digitalocean-access-token $DIGITALOCEAN_TOKEN \
    vision-prod

Now we have two Machines running, one locally and one on Digital Ocean:

$ docker-machine ls
NAME            ACTIVE     DRIVER         STATE     URL
vision-dev      *          virtualbox     Running   tcp://w.x.y.z:2376
vision-prod                digitalocean   Running   tcp://a.b.c.d:2376

Set vision-prod as the active machine and load the Docker environment :

$ ./docker-machine active vision-prod
$ eval "$(./docker-machine env vision-prod)"

Finally, let's build the application in the Cloud :

$ ./docker-compose build
$ ./docker-compose up -d

Support

Feel free to ask question or make suggestions in our [Issue Tracker][].

License

See LICENSE for the complete license.

Changelog

A changelog is available.

Contact

Nicolas Lamirault nicolas.lamirault@gmail.com

You can’t perform that action at this time.