Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Vision

License Apache 2 GitHub version

Description

Vision is a stack for monitoring and logging. It provides :

  • Elasticsearch web interface : http://xxx:9200
  • Kibana web interface : http://xxx:9393
  • Grafana web interface : http://xxx:9999/
  • InfluxDB web interface : http://xxx:8083
  • Heka dashboard : http://xxx:4352

Installation

  • Download and install a release :

      $ curl https://github.com/nlamirault/vision/releases/download/x.y.z/vision-x.y.z-linux.tar.gz
      $ tar zxvf vision-x.y.z
      $ cd vision-x.y.z
      $ ./init.sh
    
  • Start services using Docker Compose:

      $ docker-compose up -d
    
  • Start services using SystemD:

      $ cp -r vision-*.service /lib/systemd/system/
    

Usage

Monitoring servers : Elasticsearch/Kibana/Beats

  • Install Topbeat and Packetbeat

  • Launch Elasticsearch and Kibana services

  • Loading the templates into Elasticsearch

      $ curl -XPUT 'http://localhost:9200/_template/packetbeat' \
          -d@beats/topbeat.template.json
    
      $ curl -XPUT 'http://localhost:9200/_template/packetbeat' \
          -d@beats/packetbeat.template.json
    
  • Running topbeat metrics :

      $ topbeat -c beats/topbeat.yml
    
  • Running packetbeat metrics :

      $ packetbeat -c beats/packetbeat.yml
    
  • Testing the installation:

      $ curl -XGET 'http://localhost:9200/topbeat-*/_search?pretty'
      $ curl -XGET 'http://localhost:9200/packetbeat-*/_search?pretty'
    
  • Loading Kibana dashboards:

      $ curl -L -O http://download.elastic.co/beats/dashboards/beats-dashboards-1.0.0.tar.gz
      $ tar zxvf beats-dashboards-1.0.0.tar.gz
      $ cd beats-dashboards-1.0.0
      $ ./load.sh
    
  • Then open the Kibana website (http://localhost:9393), then select Topbeat index, and open Topbeat dashboard. Do same with Packetbeat index and dashboard.

Monitoring servers : Telegraf/InfluxDB/Grafana

  • Install Telegraf

  • Launch InfluxDB and Grafana services

  • Running telegraf metrics :

      $ telegraf -config telegraf/telegraf.conf
    
  • Then open the Grafana dashboard (http://localhost:9191) and import the Vision Telegraf dashboard from (grafana/grafana-telegraf.json)

  • You could explore metrics into the InfluxDB UI on http://localhost:8083 with the query :

      SHOW MEASUREMENTS
    

Monitoring servers : Prometheus/Grafana

  • Install Prometheus and Prometheus node exporter

  • Launch services :

      $ ./prometheus -config.file=prometheus/vision.yml
      $ ./node_exporter
    
  • Check Prometheus installation on : http://localhost:9090 and http://localhost:9090/consoles/node.html

  • hen open the Grafana dashboard (http://localhost:9191) and import the Vision Prometheus dashboard from (grafana/grafana-prometheus.json)

Log analysis (Elasticsearch/Filebeat/Kibana)

  • Install Filebeat

  • Launch Elasticsearch and Kibana services

  • Loading the Index Template into Elasticsearch

      $ curl -XPUT 'http://localhost:9200/_template/packetbeat' \
          -d@beats/filebeat.template.json
    
  • Running filebeat metrics :

      $ filebeat -c beats/filebeat.yml
    

SystemD

You could use services files to launch Vision monitoring tools using SystemD.

$ sudo cp systemd/*.service /lib/systemd/system/
$ sudo systemctl enable vision-telegraf
$ sudo systemctl enable vision-topbeat
$ sudo systemctl enable vision-packetbeat

Development

  • Creates a virtual machine called vision-dev for the development environment :

      $ ./docker-machine create -d virtualbox vision-dev
      $ eval "$(./docker-machine env vision-dev)"
    
  • Check vision machine runnning :

      $ ./docker-machine ls
    
  • Launch vision :

      $ ./docker-compose up
    
  • Open your browser and navigate to the IP address associated with the vision virtual machine :

      $ ./docker-machine ip
    
  • To see which environment variables are available to the web service, run:

      $ ./docker-compose run web env
    

Deployment

With our app running locally, we can now push this exact same environment to a cloud hosting provider with Docker Machine

Set your credentials in your environment :

$ source XXXXXXX.sh

Deploy a new instance :

$ docker-machine -D create -d digitalocean \
    --digitalocean-access-token $DIGITALOCEAN_TOKEN \
    vision-prod

Now we have two Machines running, one locally and one on Digital Ocean:

$ docker-machine ls
NAME            ACTIVE     DRIVER         STATE     URL
vision-dev      *          virtualbox     Running   tcp://w.x.y.z:2376
vision-prod                digitalocean   Running   tcp://a.b.c.d:2376

Set vision-prod as the active machine and load the Docker environment :

$ ./docker-machine active vision-prod
$ eval "$(./docker-machine env vision-prod)"

Finally, let's build the application in the Cloud :

$ ./docker-compose build
$ ./docker-compose up -d

Support

Feel free to ask question or make suggestions in our [Issue Tracker][].

License

See LICENSE for the complete license.

Changelog

A changelog is available.

Contact

Nicolas Lamirault nicolas.lamirault@gmail.com

About

a system monitoring and log collector

Resources

License

Packages

No packages published