Skip to content
Permalink
Browse files

wip

  • Loading branch information
mountaindude committed Oct 13, 2019
1 parent 45ad1c7 commit 9d5d00377f357ea4deca6de9a6b929e70e265f27
@@ -25,7 +25,7 @@

Runs on Windows, Linux, Mac OS, in Docker or as a Kubernetes service.

Eager to try it out? The [getting started](docs/getting started/_index.md) pages will tell you everything you need.
Eager to try it out? The [getting started](docs/getting_started/) pages will tell you everything you need.
{{% /blocks/lead %}}


This file was deleted.

This file was deleted.

This file was deleted.

@@ -12,7 +12,5 @@ description: >
Butler SOS is written in Node.js, which is a cross-platform programming environment.
This means most kinds of computers and servers can be used to run Butler SOS, including Windows, Linux and Mac OS.

Setting up Butler SOS is pretty straight forward, but you do need a working understanding of Qlik Sense admin tasks. For example, you need to export certificates from the QMC, as well as installing Butler SOS itself.
Setting up Butler SOS is pretty straightforward, but you do need a working understanding of Qlik Sense admin tasks. For example, you need to export certificates from the QMC, as well as installing Butler SOS itself.
{{% /pageinfo %}}


@@ -0,0 +1,284 @@
---
title: "Docker"
linkTitle: "Docker"
weight: 3
description: >
Running Butler SOS in Docker. Installation and configuration.
---

Docker is great in that it runs on many different platforms.
This means that as long as the Docker runtime environment is installed, you can run Butler SOS on your Mac laptop, on a Linux server or on a Windows server.

Installing and getting started with Butler SOS in Docker can look something like this:

Create a directory for Butler SOS. Config files and logs will be stored here.

```bash
proton:code goran$ mkdir -p butler-sos-docker/config/certificate
proton:code goran$ mkdir -p butler-sos-docker/logs
proton:code goran$ cd butler-sos-docker
proton:butler-sos-docker goran$
```

* Copy the [YAML config file](https://github.com/ptarmiganlabs/butler-sos/blob/master/src/config/production_template.yaml) from the GitHub repository into the ./config directory, rename it to `production.yaml` (or something else, as long as it matches the NODE_ENV environment variable) and edit it as needed. Note that for the Docker setup the path to certificates (as specified in the YAML config file) should be `/nodeapp/config/certificate/` (this is the Docker container's internal path to the certificate directory).
* Copy [docker-compose.yml](https://github.com/ptarmiganlabs/butler-sos/blob/master/src/docker-compose.yml) from the GitHub repository to the main Butler SOS directory.
* Export certifiates from the QMC in Qlik Sense Enterprise, place them in the `./config/certificate` directory.

Let's do this one step at a time.
What files are there?

```bash
proton:butler-sos-docker goran$ ls -la
total 8
drwxr-xr-x 4 goran staff 128 Oct 14 17:10 .
drwxr-xr-x 51 goran staff 1632 Oct 14 17:08 ..
drwxr-xr-x 3 goran staff 96 Oct 14 17:08 config
-rw-r--r--@ 1 goran staff 357 Oct 14 17:10 docker-compose.yml
proton:butler-sos-docker goran$
proton:butler-sos-docker goran$ ls -la config/
total 8
drwxr-xr-x 4 goran staff 128 Oct 14 17:11 .
drwxr-xr-x 4 goran staff 128 Oct 14 17:10 ..
drwxr-xr-x 2 goran staff 64 Oct 14 17:08 certificate
-rw-r--r--@ 1 goran staff 1335 Oct 14 17:11 production.yaml
proton:butler-sos-docker goran$
proton:butler-sos-docker goran$ ls -la config/certificate/
total 24
drwxr-xr-x 5 goran staff 160 Oct 14 17:13 .
drwxr-xr-x 4 goran staff 128 Oct 14 17:11 ..
-rw-r--r--@ 1 goran staff 1166 Oct 14 17:13 client.pem
-rw-r--r--@ 1 goran staff 1702 Oct 14 17:13 client_key.pem
-rw-r--r--@ 1 goran staff 1192 Oct 14 17:13 root.pem
proton:butler-sos-docker goran$
```

What des the config file look like?

```bash
proton:butler-sos-docker goran$ cat config/production.yaml
Butler-SOS:
# All configuration items are mandatory, unless otherwise noted.
# Logging configuration
logLevel: info # Log level. Possible log levels are silly, debug, verbose, info, warn, error
fileLogging: true # true/false to enable/disable logging to disk file
logDirectory: logs # Subdirectory where log files are stored
# Qlik Sense logging db config parameters
logdb:
enableLogDb: true
# Items below are mandatory if enableLogDb=true
pollingInterval: 60000 # How often (milliseconds) should Postgres log db be queried for warnings and errors?
queryPeriod: 5 minutes # How far back should Butler SOS query for log entries? Default is 5 min
host: <IP or FQDN of Qlik Sense logging db>
port: 4432
qlogsReaderUser: qlogs_reader
qlogsReaderPwd: <pwd>
extractErrors: true # Should error level entries be extracted from log db into Influxdb?
extractWarnings: true # Should warn level entries be extracted from log db into Influxdb?
extractInfo: true # Should info level entries be extracted from log db into Influxdb? Warning! Seting this to true will result in LOTS of log messages being retrrieved by Butler SOS!
# Certificates to use when querying Sense for healthcheck data. Get these from the Certificate Export in QMC.
cert:
# If running Butler SOS in a Docker container, the cert paths MUST be the following
clientCert: /nodeapp/config/certificate/client.pem
clientCertKey: /nodeapp/config/certificate/client_key.pem
clientCertCA: /nodeapp/config/certificate/root.pem
# MQTT config parameters
mqttConfig:
enableMQTT: false
# Items below are mandatory if enableMQTT=true
brokerHost: <IP of MQTT server>
brokerPort: 1883
baseTopic: butler-sos/ # Topic should end with /
# Influx db config parameters
influxdbConfig:
enableInfluxdb: true
# Items below are mandatory if enableInfluxdb=true
hostIP: <IP or FQDN of Influxdb server>
hostPort: <Port where Influxdb is listening> # Optional. Default value=8086
auth:
enable: false # Does influxdb instance require authentication (true/false)?
username: <username> # Username for Influxdb authentication. Mandatory if auth.enable=true
password: <password> # Password for Influxdb authentication. Mandatory if auth.enable=true
dbName: SenseOps
# Default retention policy that should be created in InfluxDB when Butler SOS creates a new database there.
# Any data older than retention policy threshold will be purged from InfluxDB.
retentionPolicy:
name: 4weeks
duration: 4w
# Control whether certain fields are stored in InfluxDB or not
# Use with caution! Enabling activeDocs, loadedDocs or inMemoryDocs may result in lots of data sent to InfluxDB.
includeFields:
activeDocs: false # Should data on what docs are active be stored in Influxdb (true/false)?
loadedDocs: false # Should data on what docs are loaded be stored in Influxdb (true/false)?
inMemoryDocs: false # Should data on what docs are in memory be stored in Influxdb (true/false)?
# Sessions per virtual proxy
userSessions:
enableSessionExtract: true # Query unique user IDs of what users have sessions open (true/false)?
# Items below are mandatory if enableSessionExtract=true
pollingInterval: 15000 # How often (milliseconds) should detailed session data be polled?
serversToMonitor:
pollingInterval: 30000 # How often (milliseconds) should the healthcheck API be polled?
# List of extra tags for each server. Useful for creating more advanced Grafana dashboards.
# Each server below MUST include these tags in its serverTags property.
# The tags below are just examples - define your own as needed
serverTagsDefinition:
- server_group
- serverLocation
- server-type
- serverBrand
# Sense Servers that should be queried for healthcheck data
servers:
- host: <server1.my.domain>
serverName: <server1>
serverDescription: <description>
logDbHost: <host name as used in QLogs db>
userSessions:
enable: true
# Items below are mandatory if userSessions.enable=true
host: <server1.my.domain>:4243
virtualProxies:
- virtualProxy: / # Default virtual proxy
- virtualProxy: /hdr # "hdr" virtual proxy
serverTags:
server_group: DEV
serverLocation: Asia
server-type: virtual
serverBrand: Dell
- host: <server2.my.domain>
serverName: <server2>
serverDescription: <description>
logDbHost: <host name as used in QLogs db>
userSessions:
enable: true
# Items below are mandatory if userSessions.enable=true
host: <server2.my.domain>:4243
virtualProxies:
- virtualProxy: /finance # "finance" virtual proxy
serverTags:
server_group: PROD
serverLocation: Europe
server-type: physical
serverBrand: HP
proton:butler-sos-docker goran$
```

What does the docker-compose.yml file look like?

```bash
proton:butler-sos-docker goran$ cat docker-compose.yml
# docker-compose.yml
version: '3.3'
services:
butler-sos:
image: ptarmiganlabs/butler-sos:5.0.0
container_name: butler-sos
restart: always
volumes:
# Make config file accessible outside of container
- "./config:/nodeapp/config"
- "./logs:/nodeapp/logs"
environment:
- "NODE_ENV=production" # Means that Butler SOS will read config data from production.yaml
logging:
driver: "json-file"
options:
max-file: "5"
max-size: "5m"
proton:butler-sos-docker goran$
```

Ok, all good. Let's start Butler SOS using docker-compose:

```bash
proton:butler-sos-docker goran$ docker-compose up
➜ docker-compose up
Creating butler-sos ... done
Attaching to butler-sos
butler-sos | 2019-10-12T20:14:21.814Z debug: CONFIG: Setting up new Influx database: Found server tag : server_group
butler-sos | 2019-10-12T20:14:21.824Z debug: CONFIG: Setting up new Influx database: Found server tag : serverLocation
butler-sos | 2019-10-12T20:14:21.826Z debug: CONFIG: Setting up new Influx database: Found server tag : server-type
butler-sos | 2019-10-12T20:14:21.829Z debug: CONFIG: Setting up new Influx database: Found server tag : serverBrand
butler-sos | 2019-10-12T20:14:21.831Z info: CONFIG: Influxdb enabled: true
butler-sos | 2019-10-12T20:14:21.832Z info: CONFIG: Influxdb host IP: 192.168.100.20
butler-sos | 2019-10-12T20:14:21.833Z info: CONFIG: Influxdb host port: 8086
butler-sos | 2019-10-12T20:14:21.834Z info: CONFIG: Influxdb db name: SenseOps
butler-sos | 2019-10-12T20:14:22.006Z info: --------------------------------------
butler-sos | 2019-10-12T20:14:22.006Z info: Starting Butler SOS
butler-sos | 2019-10-12T20:14:22.009Z info: Log level is: debug
butler-sos | 2019-10-12T20:14:22.010Z info: App version is: 5.0.0
butler-sos | 2019-10-12T20:14:22.011Z info: --------------------------------------
butler-sos | 2019-10-12T20:14:22.012Z debug: Client cert: /nodeapp/config/certificate/client.pem
butler-sos | 2019-10-12T20:14:22.013Z debug: Client cert key: /nodeapp/config/certificate/client_key.pem
butler-sos | 2019-10-12T20:14:22.014Z debug: CA cert: /nodeapp/config/certificate/root.pem
butler-sos | 2019-10-12T20:14:22.018Z debug: USER SESSIONS: Monitor user sessions for these servers/virtual proxies: [
butler-sos | {
butler-sos | "host": "pro2-win1.ptarmiganlabs.net:4747",
butler-sos | "serverName": "sense1",
butler-sos | "serverDescription": "Central",
butler-sos | "logDbHost": "pro2-win1",
butler-sos | "userSessions": {
butler-sos | "enable": true,
butler-sos | "host": "pro2-win1.ptarmiganlabs.net:4243",
butler-sos | "virtualProxies": [
butler-sos | {
butler-sos | "virtualProxy": "/"
butler-sos | },
butler-sos | {
butler-sos | "virtualProxy": "/hdr"
butler-sos | }
butler-sos | ]
butler-sos | },
butler-sos | "serverTags": {
butler-sos | "server_group": "CENTRAL",
butler-sos | "serverLocation": "Europe",
butler-sos | "server-type": "virtual",
butler-sos | "serverBrand": "HP"
butler-sos | }
butler-sos | }
butler-sos | ]
butler-sos | 2019-10-12T20:14:22.028Z info: MAIN: Docker healthcheck server now listening
butler-sos | 2019-10-12T20:14:22.075Z info: CONFIG: Found InfluxDB database: SenseOps
...
...
```
Once everything everything looks good you can stop the container (ctrl-C), then start it again in daemon mode (i.e. running unattended in the background):
```bash
proton:butler-sos-docker goran$ docker-compose up -d
Starting butler-sos ... done
proton:butler-sos-docker goran$
```
Setting the log level to info in the config file will reduce log output.
The Docker container implements Docker healthchecks, which means you can run `docker ps` to see whether the container is healthy or not:
```bash
➜ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
905f21443f97 mountaindude/butler-sos:5.0.0 "docker-entrypoint.s…" 6 minutes ago Up 22 seconds (healthy) butler-sos
```

0 comments on commit 9d5d003

Please sign in to comment.
You can’t perform that action at this time.