Skip to content


Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Autopilot Pattern ELK

Autopilot Pattern implementation of ELK

This repo is a demonstration of the Elasticsearch-Logstash-Kibana (ELK) stack, designed to be self-operating according to the autopilot pattern. This application also demonstrates Triton support of Docker log drivers.

The components of this stack are:

  • Elasticsearch, to store and search our logs.
  • Logstash, to ingest logs from our containers and write them to Elasticsearch.
  • Kibana, as a web UI to search the logs we store in Elasticsearch.
  • Consul, acting a service catalog to support discovery
  • ContainerPilot, to help with service discovery and bootstrap orchestration
  • Triton, Joyent's container-native infrastructure platform.
  • Nginx, acting as a source of logs for testing.

Diagram of Triton-ELK architecture

The ELK stack components all have configuration files that expect hard-coded IP addresses for their dependencies; Kibana and Logstash need an IP for Elasticsearch and Elasticsearch nodes need the IP of one other node to bootstrap clustering. Each container has a startup script configured as a ContainerPilot preStart handler to update the config file prior to starting the main application.

Additionally, the ELK application expects certain indexes to be created in Elasticsearch. When the Kibana application starts, the preStart handler script ( will write these indexes to Elasticsearch and will send a log entry to Logstash so that the Logstash application can create its initial schema.

Getting started

  1. Get a Joyent account and add your SSH key.
  2. Install the Docker Toolbox (including docker and docker-compose) on your laptop or other environment, as well as the Joyent Triton CLI (triton replaces our old sdc-* CLI tools)
  3. Configure Docker and Docker Compose for use with Joyent:
curl -O && chmod +x

Check that everything is configured correctly by running ./ check. If it returns without an error you're all set. This script will create and _env file that includes the Triton CNS name for the Consul service.

Start the stack

You can run the entire application with just Docker Compose:

$ docker-compose -p elk up -d
Creating elk_consul_1
Creating elk_elasticsearch_master_1
Creating elk_elasticsearch_1
Creating elk_elasticsearch_data_1
Creating elk_logstash_1
Creating elk_kibana_1

$ docker-compose -p elk ps

Name                          Command           State    Ports
elk_consul_1                 /bin/start -server    Up   53/tcp, 53/udp,
                             -bootst ...                8300/tcp, 8301/tcp,
                                                        8301/udp, 8302/tcp,
                                                        8302/udp, 8400/tcp,
elk_elasticsearch_1          /bin/containerpilot   Up   9200/tcp, 9300/tcp
elk_elasticsearch_data_1     /bin/containerpilot   Up   9200/tcp, 9300/tcp
elk_elasticsearch_master_1   /bin/containerpilot   Up   9200/tcp, 9300/tcp
elk_kibana_1                 /bin/containerpilot   Up>5601/tcp
elk_logstash_1               /bin/containerpilot   Up>12201/tcp,

Within a few moments all components of the application will be registered in the Consul discovery service and will have found the other components they need. We can add new nodes to Elasticsearch just by running docker-compose -p scale <node type>=<number of nodes>.

Run the test rig

The test script included with this repo can open the Consul and Kibana web consoles on systems that support the open command (ex. OS X):

./ show
Waiting for Consul...
Opening Consul console... Refresh the page to watch services register.
Waiting for Kibana to register as healthy...
Opening Kibana console.

This repo also includes a Docker Compose file for starting Nginx containers that are configured to use either the syslog or gelf log driver.

$ ./ -p elk test syslog
Starting Nginx log source...
Pulling nginx_syslog (autopilotpattern/nginx-elk-demo:latest)...
latest: Pulling from autopilotpattern/nginx-elk-demo
Creating elk_nginx_syslog_1
Waiting for Nginx to register as healthy...

Opening web page.

HTTP requests that we send to Nginx will be logged and be visible in Kibana.

Screenshot of Kibana


Elasticsearch-Logstash-Kibana (ELK) stack demonstrating the autopilot pattern







No packages published