Skip to content
Permalink
Browse files
[FLAGON-382, 347, 406] Updated Docker files and README for Flagon nam…
…echange, updated userale index to coerce interval log times into date format
  • Loading branch information
poorejc committed Jul 13, 2019
1 parent 9ee34a1 commit ba8bcbdc877d7c97dd09495e26fca30ebce1a4e3
Show file tree
Hide file tree
Showing 11 changed files with 181 additions and 98 deletions.
@@ -1,6 +1,6 @@
Building Apache Flagon Docker Containers
===================================
*Last Tested 24 MAR 2019 using Docker Engine v18.09.2, Compose v1.23.2, Machine v0.16.1*
*Last Integration Test: 12 JUL 2019 using Docker Engine v18.09.2, Compose v1.23.2, Machine v0.16.1*

Prerequisites
-------------
@@ -25,7 +25,7 @@ requires special configuration. Please reach out to us at [our dev list](mailto:
Before installation, check that virtualbox version is at least 5.2. [``Reinstall virtualbox``](https://www.virtualbox.org/wiki/Downloads), if needed.

```bash
$ docker-machine create --virtualbox-memory 3072 --virtualbox-cpu-count 2 senssoft
$ docker-machine create --virtualbox-memory 3072 --virtualbox-cpu-count 2 flagon
```

1. Before launching the Docker containers, ensure your ``vm_max_map_count``
@@ -34,7 +34,7 @@ requires special configuration. Please reach out to us at [our dev list](mailto:

```bash
# Example for Linux systems
$ docker-machine ssh senssoft sudo sysctl -w vm.max_map_count=262144
$ docker-machine ssh flagon sudo sysctl -w vm.max_map_count=262144
```

1. Create externel docker network to enable system monitoring. Only enable if running
@@ -59,12 +59,12 @@ requires special configuration. Please reach out to us at [our dev list](mailto:
1. Confirm state:
```bash
# if Flagon vm is remote
$ docker-machine ssh senssoft curl -XGET http://localhost:9200/_cluster/health?pretty
$ docker-machine ssh flagon curl -XGET http://localhost:9200/_cluster/health?pretty
# if Flagon virtual machine is running on your local machine, no need for ssh, instead:
$ curl -XGET http://localhost:9200/_cluster/health?pretty
#output should look like this:
"cluster_name" : "SensSoft",
"status" : "yellow",
"cluster_name" : "Flagon",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
@@ -86,7 +86,7 @@ requires special configuration. Please reach out to us at [our dev list](mailto:
```bash
$ docker-compose up -d logstash
# if Flagon vm is remote
$ docker-machine ssh senssoft curl -XGET http://localhost:8100
$ docker-machine ssh flagon curl -XGET http://localhost:8100
# if Flagon virtual machine is running on your local machine, no need for ssh, instead:
$ curl -XGET http://localhost:8100
#ouput should look like this:
@@ -98,7 +98,7 @@ requires special configuration. Please reach out to us at [our dev list](mailto:
```bash
$ docker-compose up -d site
# for remote users, forwards port to localhost
$ ssh docker@$(docker-machine ip senssoft) -L 8080:localhost:8080
$ ssh docker@$(docker-machine ip flagon) -L 8080:localhost:8080
```

Visit `http://localhost:8080` and you will see Apache Flagon's home page.
@@ -114,7 +114,7 @@ requires special configuration. Please reach out to us at [our dev list](mailto:
```bash
$ docker-compose up -d kibana
# for remote users, forwards port to localhost
$ ssh docker@$(docker-machine ip senssoft) -L 5601:localhost:5601
$ ssh docker@$(docker-machine ip flagon) -L 5601:localhost:5601
```

1. Register an index in Kibana to see the logs:
@@ -127,7 +127,7 @@ requires special configuration. Please reach out to us at [our dev list](mailto:
1. Load example Dashboard and Visualizations under docker/kibana/.

Goto: Management -> Saved Objects and select the `Import` button. Import the
`Apache SensSoft Visualizations.json`, `Drill-Down Search.json`, `Apache SensSoft Page Usage Dashboard.json`, and `Apache SensSoft User Access Dashboard.json` files from the "Saved Objects" folder in the kibana directory.
`Apache Flagon Visualizations.json`, `Drill-Down Search.json`, `Apache Flagon Page Usage Dashboard.json`, and `Apache Flagon User Access Dashboard.json` files from the "Saved Objects" folder in the kibana directory.

![alt text][management]

@@ -136,7 +136,7 @@ requires special configuration. Please reach out to us at [our dev list](mailto:
![alt text][viz_import]

Once that is complete, navigate to the `Dashboard` view in Kibana and click the
`Apache SensSoft Page Usage Dashboard` object.
`Apache Flagon Page Usage Dashboard` object.

![alt text][dashboard]

@@ -157,7 +157,12 @@ requires special configuration. Please reach out to us at [our dev list](mailto:

1. To kill the Flagon machine.
```bash
$ docker-machine rm senssoft
$ docker-machine stop flagon
```

1. To remove the Flagon machine.
```bash
$ docker-machine rm flagon
```

If running on a single machine, on reboot or restart your Flagon machine is available, but
@@ -166,11 +171,11 @@ compose up commands above to restart containers.

1. Restart the Flagon machine.
```bash
$ docker-machine start senssoft
$ docker-machine start flagon
$ docker-machine ls #confirm state
#output should look like this:
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
senssoft - virtualbox Running tcp://192. ... v18.09.3
flagon - virtualbox Running tcp://192. ... v18.09.3
```

Having Issues?
@@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Apache SensSoft Single-Node Build
# Apache Flagon Single-Node Build

version: "3"
services:
@@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Apache SensSoft Single-Node Build
# Apache Flagon Single-Node Build

version: "3"
services:
@@ -22,7 +22,7 @@ COPY config /usr/share/elasticsearch/config
RUN chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/config

# Set environment
ENV CLUSTER_NAME SensSoft
ENV CLUSTER_NAME Flagon

# Kubernetes requires swap is turned off
ENV MEMORY_LOCK false
@@ -16,7 +16,7 @@
# Custom config borrowed from Pires: https://github.com/pires/docker-elasticsearch-kubernetes

cluster:
name: ${CLUSTER_NAME:SensSoft}
name: ${CLUSTER_NAME:FLAGON}

node:
master: ${NODE_MASTER:true}
@@ -3,10 +3,10 @@
"_id": "90b264d0-2cd4-11e9-9b9a-95b7b006b436",
"_type": "dashboard",
"_source": {
"title": "Apache SensSoft Business Analytics Dashboard",
"title": "Apache Flagon Business Analytics Dashboard",
"hits": 0,
"description": "Abstracts Multi-User Interaction with Single Pages/Apps",
"panelsJSON": "[{\"embeddableConfig\":{\"vis\":{\"defaultColors\":{\"0 - 400\":\"rgb(255,245,240)\",\"1,200 - 1,600\":\"rgb(203,24,29)\",\"400 - 800\":\"rgb(252,187,161)\",\"800 - 1,200\":\"rgb(251,106,74)\"},\"legendOpen\":false}},\"gridData\":{\"h\":21,\"i\":\"5\",\"w\":48,\"x\":0,\"y\":84},\"id\":\"429c0940-fce6-11e8-a156-87bcaf41506e\",\"panelIndex\":\"5\",\"title\":\"Unique User Activity by Day\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"h\":15,\"i\":\"6\",\"w\":12,\"x\":0,\"y\":14},\"id\":\"5b467330-06e2-11e9-8c58-f38cfff498f6\",\"panelIndex\":\"6\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{\"vis\":{\"legendOpen\":false}},\"gridData\":{\"h\":15,\"i\":\"10\",\"w\":23,\"x\":25,\"y\":14},\"id\":\"20b291b0-12f2-11e9-a1e0-631cf5fb9335\",\"panelIndex\":\"10\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"h\":15,\"i\":\"13\",\"w\":13,\"x\":12,\"y\":14},\"id\":\"d1ba2560-2d03-11e9-9b9a-95b7b006b436\",\"panelIndex\":\"13\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"h\":29,\"i\":\"14\",\"w\":23,\"x\":25,\"y\":29},\"id\":\"e1b6c290-06e1-11e9-8c58-f38cfff498f6\",\"panelIndex\":\"14\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"h\":29,\"i\":\"15\",\"w\":25,\"x\":0,\"y\":29},\"id\":\"822808b0-2d0c-11e9-9b9a-95b7b006b436\",\"panelIndex\":\"15\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"h\":26,\"i\":\"16\",\"w\":48,\"x\":0,\"y\":105},\"id\":\"d5a95ce0-06e3-11e9-8c58-f38cfff498f6\",\"panelIndex\":\"16\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"h\":14,\"i\":\"17\",\"w\":48,\"x\":0,\"y\":0},\"id\":\"36a0eb90-2cd6-11e9-9b9a-95b7b006b436\",\"panelIndex\":\"17\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"h\":26,\"i\":\"18\",\"w\":48,\"x\":0,\"y\":58},\"id\":\"33e09c60-2ff2-11e9-be20-a737e788a0aa\",\"panelIndex\":\"18\",\"type\":\"visualization\",\"version\":\"6.5.4\"}]",
"panelsJSON": "[{\"embeddableConfig\":{\"vis\":{\"defaultColors\":{\"0 - 400\":\"rgb(255,245,240)\",\"1,200 - 1,600\":\"rgb(203,24,29)\",\"400 - 800\":\"rgb(252,187,161)\",\"800 - 1,200\":\"rgb(251,106,74)\"},\"legendOpen\":false}},\"gridData\":{\"x\":0,\"y\":84,\"w\":48,\"h\":21,\"i\":\"5\"},\"id\":\"429c0940-fce6-11e8-a156-87bcaf41506e\",\"panelIndex\":\"5\",\"title\":\"Unique User Activity by Day\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{\"vis\":{\"legendOpen\":false}},\"gridData\":{\"x\":25,\"y\":14,\"w\":23,\"h\":15,\"i\":\"10\"},\"id\":\"20b291b0-12f2-11e9-a1e0-631cf5fb9335\",\"panelIndex\":\"10\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":12,\"y\":14,\"w\":13,\"h\":15,\"i\":\"13\"},\"id\":\"d1ba2560-2d03-11e9-9b9a-95b7b006b436\",\"panelIndex\":\"13\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":25,\"y\":29,\"w\":23,\"h\":29,\"i\":\"14\"},\"id\":\"e1b6c290-06e1-11e9-8c58-f38cfff498f6\",\"panelIndex\":\"14\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":0,\"y\":29,\"w\":25,\"h\":29,\"i\":\"15\"},\"id\":\"822808b0-2d0c-11e9-9b9a-95b7b006b436\",\"panelIndex\":\"15\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":0,\"y\":105,\"w\":48,\"h\":26,\"i\":\"16\"},\"id\":\"d5a95ce0-06e3-11e9-8c58-f38cfff498f6\",\"panelIndex\":\"16\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":0,\"y\":0,\"w\":48,\"h\":14,\"i\":\"17\"},\"id\":\"36a0eb90-2cd6-11e9-9b9a-95b7b006b436\",\"panelIndex\":\"17\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":0,\"y\":58,\"w\":48,\"h\":26,\"i\":\"18\"},\"id\":\"33e09c60-2ff2-11e9-be20-a737e788a0aa\",\"panelIndex\":\"18\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"x\":0,\"y\":14,\"w\":12,\"h\":15,\"i\":\"19\"},\"version\":\"6.6.2\",\"panelIndex\":\"19\",\"type\":\"visualization\",\"id\":\"a712e550-a4da-11e9-baf2-196230c538eb\",\"embeddableConfig\":{}}]",
"optionsJSON": "{\"darkTheme\":true,\"useMargins\":false}",
"version": 1,
"timeRestore": false,
@@ -3,7 +3,7 @@
"_id": "3f02bf70-12eb-11e9-a1e0-631cf5fb9335",
"_type": "dashboard",
"_source": {
"title": "Apache SensSoft Page Usage Dashboard",
"title": "Apache Flagon Page Usage Dashboard",
"hits": 0,
"description": "Abstracts Multi-User Interaction with Single Pages/Apps. Ideal for script-tag, page-level UserALE.js deployments.",
"panelsJSON": "[{\"gridData\":{\"h\":15,\"i\":\"2\",\"w\":32,\"x\":16,\"y\":0},\"id\":\"f7ca4560-be7c-11e7-8adf-fd0a12531eca\",\"panelIndex\":\"2\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{\"vis\":{\"defaultColors\":{\"0 - 400\":\"rgb(255,245,240)\",\"1,200 - 1,600\":\"rgb(203,24,29)\",\"400 - 800\":\"rgb(252,187,161)\",\"800 - 1,200\":\"rgb(251,106,74)\"},\"legendOpen\":false}},\"gridData\":{\"h\":15,\"i\":\"5\",\"w\":20,\"x\":12,\"y\":15},\"id\":\"429c0940-fce6-11e8-a156-87bcaf41506e\",\"panelIndex\":\"5\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"h\":15,\"i\":\"6\",\"w\":12,\"x\":0,\"y\":15},\"id\":\"5b467330-06e2-11e9-8c58-f38cfff498f6\",\"panelIndex\":\"6\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"h\":45,\"i\":\"7\",\"w\":48,\"x\":0,\"y\":45},\"id\":\"2bf40010-be7d-11e7-8adf-fd0a12531eca\",\"panelIndex\":\"7\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"h\":15,\"i\":\"8\",\"w\":16,\"x\":0,\"y\":0},\"id\":\"ba30fd80-12ee-11e9-a1e0-631cf5fb9335\",\"panelIndex\":\"8\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"h\":15,\"i\":\"9\",\"w\":24,\"x\":0,\"y\":30},\"id\":\"bcd7c950-be7c-11e7-8adf-fd0a12531eca\",\"panelIndex\":\"9\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{\"vis\":{\"legendOpen\":false}},\"gridData\":{\"h\":15,\"i\":\"10\",\"w\":16,\"x\":32,\"y\":15},\"id\":\"20b291b0-12f2-11e9-a1e0-631cf5fb9335\",\"panelIndex\":\"10\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{\"spy\":null},\"gridData\":{\"h\":15,\"i\":\"11\",\"w\":24,\"x\":24,\"y\":30},\"id\":\"bf327a20-12f3-11e9-a1e0-631cf5fb9335\",\"panelIndex\":\"11\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"h\":40,\"i\":\"12\",\"w\":48,\"x\":0,\"y\":90},\"id\":\"887fec40-1886-11e9-bdac-e790c2bff4d4\",\"panelIndex\":\"12\",\"type\":\"visualization\",\"version\":\"6.5.4\"}]",
@@ -3,7 +3,7 @@
"_id": "ab9f1f90-fce6-11e8-a156-87bcaf41506e",
"_type": "dashboard",
"_source": {
"title": "Apache SensSoft User Access Dashboard",
"title": "Apache Flagon User Access Dashboard",
"hits": 0,
"description": "Abstracts Multi-User Utilization of IT Assets (and Web Usage). Ideal for UserALE.js Web Extension deployments.",
"panelsJSON": "[{\"gridData\":{\"h\":15,\"i\":\"2\",\"w\":32,\"x\":16,\"y\":0},\"id\":\"f7ca4560-be7c-11e7-8adf-fd0a12531eca\",\"panelIndex\":\"2\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{\"spy\":null},\"gridData\":{\"h\":15,\"i\":\"4\",\"w\":16,\"x\":0,\"y\":0},\"id\":\"e36b1f40-be7c-11e7-8adf-fd0a12531eca\",\"panelIndex\":\"4\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"h\":15,\"i\":\"5\",\"w\":36,\"x\":12,\"y\":15},\"id\":\"429c0940-fce6-11e8-a156-87bcaf41506e\",\"panelIndex\":\"5\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"h\":15,\"i\":\"6\",\"w\":12,\"x\":0,\"y\":15},\"id\":\"5b467330-06e2-11e9-8c58-f38cfff498f6\",\"panelIndex\":\"6\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"h\":15,\"i\":\"7\",\"w\":36,\"x\":12,\"y\":45},\"id\":\"e1b6c290-06e1-11e9-8c58-f38cfff498f6\",\"panelIndex\":\"7\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"embeddableConfig\":{\"vis\":{\"colors\":{\"0 - 6\":\"#FCE2DE\"},\"defaultColors\":{\"0 - 6\":\"rgb(255,245,240)\",\"11 - 17\":\"rgb(251,106,74)\",\"17 - 22\":\"rgb(203,24,29)\",\"6 - 11\":\"rgb(252,187,161)\"},\"legendOpen\":true}},\"gridData\":{\"h\":15,\"i\":\"8\",\"w\":48,\"x\":0,\"y\":30},\"id\":\"d5a95ce0-06e3-11e9-8c58-f38cfff498f6\",\"panelIndex\":\"8\",\"type\":\"visualization\",\"version\":\"6.5.4\"},{\"gridData\":{\"h\":15,\"i\":\"9\",\"w\":12,\"x\":0,\"y\":45},\"id\":\"5eeda1e0-06e5-11e9-8c58-f38cfff498f6\",\"panelIndex\":\"9\",\"type\":\"visualization\",\"version\":\"6.5.4\"}]",

0 comments on commit ba8bcbd

Please sign in to comment.