Giving the Elastic Stack a try in Vagrant
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs
elastic-stack Upgrade to Elastic 6.5.0 (#19) Nov 19, 2018
.gitattributes FIX #15 Line ending issues for text files for good Jul 8, 2018
.gitignore ADD the exported OVA box to the description Mar 5, 2017
LICENSE Initial commit Mar 28, 2016
Vagrantfile CHANGE over to the official Ubuntu box, which seems to be working fine Jul 7, 2018
readme.md Upgrade to Elastic 6.5.0 (#19) Nov 19, 2018
screenshot.png V6 (#13) Jul 4, 2018

readme.md

Elastic Stack in a Box

This repository will install the Elastic Stack (Elasticsearch, Logstash, Kibana, and Beats) and optionally start a trial of commercial features. You can either start from scratch and configure everything with Vagrant and Ansible or you can download the final OVA image.

Features

  • Filebeat system, auditd, logstash, mongodb, nginx, osquery, and redis modules
  • Filebeat collecting Kibana JSON logs from /var/log/kibana/kibana.log
  • Auditbeat file_integrity module on /home/vagrant/ directory and auditd module
  • Heartbeat pinging nginx every 10s
  • Metricbeat system, docker, elasticsearch, kibana, logstash, mongodb, nginx and redis modules
  • Packetbeat sending its data via Redis + Logstash, monitoring flows, ICMP, DNS, HTTP (nginx and Kibana), Redis, and MongoDB (generate traffic with $ mongo /elastic-stack/mongodb.js)
  • The pattern for nginx is already prepared in /opt/logstash/patterns/ and you can collect /var/log/nginx/access.log with Filebeat and add a filter in Logstash with the pattern as an exercise

Vagrant and Ansible

Do a simple vagrant up by using Vagrant's Ansible provisioner. All you need is a working Vagrant installation (2.1.2+ but the latest version is always recommended), a provider (tested with the latest VirtualBox version), and 3GB of RAM.

With the Ansible playbooks in the /elastic-stack/ folder you can configure the whole system step by step. Just run them in the given order inside the Vagrant box:

> vagrant ssh
$ cd /elastic-stack/
$ ansible-playbook 1_configure-elasticsearch.yml
$ ansible-playbook 2_configure-kibana.yml
$ ansible-playbook 3_configure-logstash.yml
$ ansible-playbook 4_configure-auditbeat.yml
$ ansible-playbook 4_configure-filebeat.yml
$ ansible-playbook 4_configure-heartbeat.yml
$ ansible-playbook 4_configure-metricbeat.yml
$ ansible-playbook 4_configure-packetbeat.yml
$ ansible-playbook 5_trial-xpack.yml

Or if you are in a hurry, run all playbooks with $ /elastic-stack/all.sh at once.

OVA Image

If Vagrant and Ansible sound too complicated, there is also the final result: An OVA image, which you can import directly into VirtualBox:

  • Download the image from https://s3.eu-central-1.amazonaws.com/xeraa/public/elastic-stack.ova.
  • Load the OVA file into VirtualBox and make sure you have 3GB of RAM available for it: File -> Import Appliance... -> Select the file and start it
  • Connect to the instance with the credentials vagrant and vagrant in the VirtualBox window.
  • Or use SSH with the same credentials:
    • Windows: Use http://www.putty.org and connect to vagrant@127.0.0.1 on port 2222.
    • Mac and Linux: $ ssh vagrant@127.0.0.1 -p 2222 -o PreferredAuthentications=password

Kibana

Access Kibana at http://localhost:5601.

Test Data

You can use /opt/injector.jar to generate test data in the person index. To generate 100,000 documents in batches of 1,000 run the following command:

$ java -jar /opt/injector.jar 100000 1000

Logstash Demo

You can play around with a Logstash example by calling $ sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /elastic-stack/raffle/raffle.conf (it can take some time) and you will find the result in the raffle index.