Added support for Kubernetes

.gitignore

@@ -1,3 +1,4 @@
+.idea
 release/incubator-*
 release/apache-*
 .sass-cache

README.md

@@ -1,5 +1,5 @@
 SensSoft Docker Containers and Website
 --------------------------------------
 
-Specific build instructions located in docker and site folders.
+Specific build instructions located in docker, kubernetes, release and site folders.
 

docker/CLUSTER.md

@@ -0,0 +1,68 @@
+How to Build SensSoft Docker Containers
+---------------------------------------
+
+1. Install [``Docker``](http://docker.com) on your machine. Require Docker 1.7 and above.
+
+1. Install ``docker-compose`` in an virtual environment. 
+   Full instructions can be found [``here``](https://docs.docker.com/compose/install/).
+
+   ```bash
+   $ python3 -m venv env
+   $ source env/bin/activate
+   $ pip install -r requirements.txt
+   ```
+
+1. Before launching the Docker containers, ensure your ``vm_max_map_count`` 
+   kernel setting is set to at least 262144.
+   Visit [``Running Elasticsearch in Production mode``](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/docker.html#docker-cli-run-prod-mode) for OS specific instructions.
+
+   ```bash
+   # Example for Linux systems
+   $ sysctl -w vm.max_map_count=262144
+   ```
+
+1. Start Elasticsearch cluster:
+
+    ```bash
+    $ docker-compose -f docker-compose.cluster.yaml up -d --scale elasticsearch=3 elasticsearch
+    $ docker-compose -f docker-compose.cluster.yaml up -d loadbalancer
+    ```
+
+    The loadbalancer node exposes port 9200 on localhost and is the only node 
+    that has HTTP enabled. Services such as Kibana and Logstash connect to the 
+    loadbalancer node directly. Loadbalancer accepts requests from Kibana and Logstash 
+    and balances them across the elasticsearch worker nodes. The elasticsearch 
+    worker nodes communicate to each other and the loadbalancer via TCP on port 9300. 
+
+    <aside class="warning">
+    Starting an elasticsearch cluster is not recommended on a single server. This
+    is just for demonstration purposes only. Please refer to our [Kubernetes] guide to 
+    deploy an Elasticsearch cluster.
+    </aside>
+
+1. Confirm cluster state:
+   ```bash
+   $ curl -XGET http://localhost:9200/_cluster/health?pretty
+    {
+     "cluster_name" : "SensSoft",
+     "status" : "green",
+     "timed_out" : false,
+     "number_of_nodes" : 4,
+     "number_of_data_nodes" : 3,
+     "active_primary_shards" : 0,
+     "active_shards" : 0,
+     "relocating_shards" : 0,
+     "initializing_shards" : 0,
+     "unassigned_shards" : 0,
+     "delayed_unassigned_shards" : 0,
+     "number_of_pending_tasks" : 0,
+     "number_of_in_flight_fetch" : 0,
+     "task_max_waiting_in_queue_millis" : 0,
+     "active_shards_percent_as_number" : 100.0
+   }
+   ```
+   Confirm that the `number_of_nodes` is 4 and `number_of_data_nodes` is 3.
+
+1. Follow remaining instructions in README.md, starting at #6.
+
+© Copyright 2016 The Charles Stark Draper Laboratory, Inc. All rights reserved.

docker/README.md

@@ -1,7 +1,7 @@
 How to Build SensSoft Docker Containers
 ---------------------------------------
 
-1. Install [``Docker``](http://docker.com) on your machine.
+1. Install [``Docker``](http://docker.com) on your machine. Requires Docker 1.7 and above.
 
 1. Install ``docker-compose`` in an virtual environment. 
    Full instructions can be found [``here``](https://docs.docker.com/compose/install/).
@@ -21,28 +21,22 @@ How to Build SensSoft Docker Containers
    $ sysctl -w vm.max_map_count=262144
    ```
 
-1. Start Elasticsearch cluster:
-
-    ```bash
-    $ docker-compose up -d --scale elasticsearch=3 elasticsearch loadbalancer
-    ```
-
-    The loadbalancer node exposes port 9200 on localhost and is the only node 
-    that has HTTP enabled. Services such as Kibana and Logstash connect to the 
-    loadbalancer node directly. Loadbalancer accepts requests from Kibana and Logstash 
-    and balances them across the elasticsearch worker nodes. The elasticsearch 
-    worker nodes communicate to each other and the loadbalancer via TCP on port 9300. 
-
-
-1. Confirm cluster state:
+1. Start Elasticsearch. Give Elasticsearch about 2 minutes to start before confirming
+   its state. 
+
+   ```bash
+   $ docker-compose up -d loadbalancer
+   ```
+
+1. Confirm state:
    ```bash
    $ curl -XGET http://localhost:9200/_cluster/health?pretty
     {
      "cluster_name" : "SensSoft",
      "status" : "green",
      "timed_out" : false,
-     "number_of_nodes" : 4,
-     "number_of_data_nodes" : 3,
+     "number_of_nodes" : 1,
+     "number_of_data_nodes" : 1,
      "active_primary_shards" : 0,
      "active_shards" : 0,
      "relocating_shards" : 0,
@@ -55,7 +49,6 @@ How to Build SensSoft Docker Containers
      "active_shards_percent_as_number" : 100.0
    }
    ```
-   Confirm that the `number_of_nodes` is 4 and `number_of_data_nodes` is 3.
 
 1. Launch logging server. Give Logstash about 2 minutes to start before confirming 
    its state.
@@ -105,8 +98,13 @@ How to Build SensSoft Docker Containers
 
    ![alt text][dashboard]
 
+1. To Launch Tap and Distill
+   ```bash
+   $ docker-compose up -d distill tap
+   ```
+
 1. To stop all containers.
-    ```sh
+    ```bash
     $ docker-compose stop
     ```
 
@@ -123,15 +121,9 @@ Having Issues?
    $ docker-compose logs > err.dump 
    ```
 
-Todo
----- 
-- [ ] TAP docker deployment instructions.
-- [ ] Distill docker deployment instructions. 
-- [ ] Apache SensSoft Docker + Kubernetes instructions.
-
-[configure_index]: ./images/configure_index.png "Configure Kibana index"
-[confirmation]: ./images/confirmation.png "Confirm index pattern conflicts"
-[dashboard]: ./images/dashboard.png "Apache Senssoft Dashboard"
-[management]: ./images/management.png "Kibana management console"
+[configure_index]: ./docs/images/configure_index.png "Configure Kibana index"
+[confirmation]: ./docs/images/confirmation.png "Confirm index pattern conflicts"
+[dashboard]: ./docs/images/dashboard.png "Apache Senssoft Dashboard"
+[management]: ./docs/images/management.png "Kibana management console"
 
 © Copyright 2016 The Charles Stark Draper Laboratory, Inc. All rights reserved.

docker/docker-compose.5.6.3.yml

@@ -0,0 +1,133 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Apache SensSoft Single-Node Build
+
+version: "3"
+services:
+
+  # Startup
+  startup:
+    build: ./startup
+    image: senssoft/startup:latest
+    container_name: senssoft-start
+    depends_on:
+      - "loadbalancer"
+    command: bash /opt/entrypoint.sh
+
+  # # Site
+  site:
+    build: ./site
+    image: senssoft/website:latest
+    container_name: senssoft-app
+    ports:
+      - 8080:8080
+    depends_on:
+      - "logstash"
+    command: python -m SimpleHTTPServer 8080
+
+  # Elasticsearch Loadbalancer/Entry
+  loadbalancer:
+    build: ./elasticsearch/5.6.3
+    image: senssoft/elasticsearch:5.6.3
+    container_name: senssoft-loadbalancer
+    environment:
+      - NODE_MASTER=true
+      - NUMBER_OF_MASTERS=1
+      - NODE_COORDINATE=true
+      - "DISCOVERY_SERVICE=elasticsearch"
+      - NODE_DATA=true
+      - NODE_INGEST=true
+      - MAX_LOCAL_STORAGE_NODES=1
+      - NETWORK_HOST=0.0.0.0
+      - HTTP_ENABLE=true
+      - HTTP_CORS_ENABLE=true
+      - HTTP_CORS_ALLOW_ORIGIN=*
+      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
+    ports:
+      - "9200:9200"
+      - "9300"
+    networks:
+      - esnet
+
+  # Kibana
+  kibana:
+    build: ./kibana/5.6.3
+    image: senssoft/kibana:5.6.3
+    container_name: senssoft-kibana
+    environment:
+      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
+      - "ELASTICSEARCH_URL=http://loadbalancer:9200"
+    ports:
+      - "5601:5601"
+    depends_on:
+      - "loadbalancer"
+    networks:
+      - esnet
+
+  # Logstash
+  logstash:
+    build: ./logstash/5.6.3
+    image: senssoft/logstash:5.6.3
+    container_name: senssoft-logstash
+    environment:
+      - "ELASTICSEARCH_URL=loadbalancer:9200"
+    ports:
+      - 8100:8100
+    volumes:
+      - ./logstash/pipeline/logstash-userale.conf:/usr/share/logstash/pipeline/logstash-userale.conf
+      - ./logstash/templates/userale.json:/usr/share/logstash/templates/userale.json
+    depends_on:
+      - "loadbalancer"
+    networks:
+      - esnet
+
+   # Distill
+  distill:
+     build: ./distill
+     image: senssoft/distill:0.1.0
+     container_name: senssoft-distill
+     ports:
+       - 8090:8090
+     depends_on:
+       - "loadbalancer"
+     networks:
+       - esnet
+
+   # TAP's Database
+  db:
+     build: ./db
+     image: senssoft/postgres:latest
+     container_name: senssoft-postgres
+     ports:
+       - "5432:5432"
+
+  # TAP
+  tap:
+     build: ./tap
+     image: senssoft/tap:0.1.0
+     container_name: senssoft-tap
+     # command: ["bash", "./wait-for-postgres.sh"]
+     command: bash -c "/usr/src/app/wait-for-postgres.sh"
+     ports:
+       - "8010:8010"
+     depends_on:
+       - db
+     links:
+       - db:db
+
+networks:
+  esnet:
+    driver: bridge