Create Quickstart Documentation

.gitignore

@@ -1,3 +1,8 @@
+# Custom ignores
+observability/examples/simple/observability-simple
+
+
+# Python ignores
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]

observability/docs/get-started/quickstart-manual.md

@@ -0,0 +1,36 @@
+
+## Manual Quickstart
+
+The quickstart page uses a script to setup the folders for you.
+
+This page instead details how to do it manually, to provide clarity.
+
+## Step 1: Setup directory
+Create the necessary directory structure
+```
+mkdir -p observability-simple/prometheus/scrape-configs/probers
+mkdir -p observability-simple/prometheus/scrape-configs/exporters
+```
+
+
+Download these two files from github, and place in the right folder
+
+- [docker-compose.yml](https://raw.githubusercontent.com/CogStack/cogstack-platform-toolkit/main/observability/examples/simple/docker-compose.yml) in observability-simple/
+- [prometheus/scrape-configs/probers/probe-simple.yml](https://raw.githubusercontent.com/CogStack/cogstack-platform-toolkit/main/observability/examples/simple/prometheus/scrape-configs/probers/probe-simple.yml)
+
+### Step 2: Start the stack
+
+In the folder containing the downloaded files, run:
+
+```bash
+docker compose up -d
+```
+
+### Step 3: Access the dashboards
+Open your web browser and go to:
+
+`localhost/grafana`
+
+You should see the Grafana dashboard displaying the availability of the sample web page.
+
+There it is, you can now see the availability of the cogstack homepage, as well as the observability stack

observability/docs/get-started/quickstart.md

@@ -0,0 +1,61 @@
+## QuickStart
+
+This tutorial guides you through running the simplest setup of the observability stack using example configuration files and Docker Compose.
+
+After completing these steps, you will have a full observability stack running locally, showing the availability of web pages you want to target
+
+### Requirements
+
+- Docker installed ([install Docker](https://docs.docker.com/get-docker/))
+- Docker Compose installed ([install Docker Compose](https://docs.docker.com/compose/install/))
+- A terminal with network access
+
+### Step 1: Run the Quickstart script
+
+Run this quickstart script to setup the project
+```bash
+curl https://raw.githubusercontent.com/CogStack/cogstack-platform-toolkit/main/observability/examples/simple/quickstart.sh | bash
+```
+Now go to "http://localhost/grafana" to see the dashboards
+
+Thats everything. The stack is running and you can see the availability.
+
+
+### Optional Step: Probe your own web page
+Now you can look at getting monitoring of your own page
+
+In your current folder, edit the file `prometheus/scrape-configs/probers/probe-simple.yml` that you downloaded from git.
+
+Add the following yml to the bottom of the file:
+
+```yaml
+- targets:
+    - https://google.com
+  labels:
+    name: google
+    job: probe-my-own-site
+```
+
+
+The change should get applied automatically, but if you dont want to wait then run
+```
+docker compose restart
+```
+
+Now refresh the grafana dashboard, and you can see the availability of google.com, it's probably 100%!
+
+
+## Next steps
+This is the end of this quickstart tutorial, that enables probing availability of endpoints.
+
+For the next steps we can:
+- Productionise our deployment to enable further features
+- Enable *Telemetry* like VM memory usage, and Elasticsearch index size, by running Exporters
+- Enable *Alerting* based on our availability and a defined Service Level Objective (SLO)
+- Look further into the available dashboards
+- Fully customize the stack with our own dashboards, recording rules and metrics
+
+
+
+
+

observability/docs/get-started/setup-tutorial.md

@@ -0,0 +1,17 @@
+
+This page 
+
+
+
+## Run the Observability Stack using Docker Compose
+
+See the /examples folder for a working example of running this
+
+To setup the stack for your deployment:
+- Create prometheus configurations as listed below
+- Copy this docker-compose.yml file
+- Mount your site config files into `/etc/prometheus/cogstack/site` 
+- Run with docker compose
+
+To collect metrics from VMs to fill out the dashboards
+- Run the Exporters on each VM as detailed below

observability/docs/overview.md

@@ -0,0 +1,18 @@
+# Cogstack Observability Stack
+
+This project provides observability of a cogstack deployment.
+
+It provides the following features:
+- Dashboards for availability
+- Telemetry of infrastructure such as host memory usage, and elasticsearch index size
+- Alerting based on a defined Service Level Objective (SLO) and burn rates
+- Blackbox Probing of services to find service level indicators of uptime and latency
+- A working inventory of what is running where
+
+## Contents
+
+See the [Quickstart](./get-started/quickstart.md) to see how to easily run this stack.
+
+
+
+

observability/examples/simple/docker-compose.yml

@@ -6,7 +6,7 @@ services:
     image: cogstacksystems/cogstack-observability-prometheus:latest
     restart: unless-stopped
     volumes:
-      - ./prometheus:/etc/prometheus/cogstack/site
+      - ./prometheus:/etc/prometheus/cogstack/site/
       - prometheus-data:/prometheus
     networks:
       - observability
@@ -17,8 +17,6 @@ services:
       - grafana-data:/var/lib/grafana
     networks:
       - observability
-    ports:
-     - '3000:3000'
     environment:
       - GF_AUTH_ANONYMOUS_ENABLED=true # Allows use of grafana without sign in 
       - GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
@@ -29,16 +27,23 @@ services:
     restart: unless-stopped
     ports:
       - "80:80"
-      - "8080:8080"
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock:ro       # So that Traefik can listen to the Docker events
   blackbox-exporter:
     image: cogstacksystems/cogstack-observability-blackbox-exporter:latest
     restart: unless-stopped
-    ports:
-    - "9115:9115"
     networks:
-      - observability-exporters
+      - observability
+  node-exporter:
+    image: prom/node-exporter
+    restart: unless-stopped
+    networks:
+      - observability
+    labels:
+      - "traefik.enable=true"
+      - "traefik.http.routers.node-exporter.rule=PathPrefix(`/node-exporter`)"
+      - "traefik.http.middlewares.node-exporter-stripprefix.stripprefix.prefixes=/node-exporter"
+      - "traefik.http.routers.node-exporter.middlewares=node-exporter-stripprefix@docker"
 networks:
   observability:
     driver: bridge

observability/examples/simple/prometheus/scrape-configs/exporters/exporters-simple.yml

@@ -0,0 +1,5 @@
+- targets:
+    - cogstack-observability-node-exporter-1:9100
+  labels:
+    job: node_exporter
+    host: localhost

observability/examples/simple/prometheus/scrape-configs/probers/probe-simple.yml

@@ -3,4 +3,4 @@
     - https://cogstack.org
   labels:
     name: cogstack-homepage
-    job: "probe-simple-example"
+    job: probe-services

observability/examples/simple/quickstart.sh

@@ -0,0 +1,26 @@
+#!/bin/bash
+set -e
+
+mkdir -p observability-simple/prometheus/scrape-configs/probers
+mkdir -p observability-simple/prometheus/scrape-configs/exporters
+cd observability-simple
+
+echo "Downloading docker-compose.yml..."
+curl -fsSL -o docker-compose.yml \
+  https://raw.githubusercontent.com/CogStack/cogstack-platform-toolkit/main/observability/examples/simple/docker-compose.yml
+
+echo "Downloading probe-simple.yml into prometheus/scrape-configs/probers/..."
+curl -fsSL -o prometheus/scrape-configs/probers/probe-simple.yml \
+  https://raw.githubusercontent.com/CogStack/cogstack-platform-toolkit/main/observability/examples/simple/prometheus/scrape-configs/probers/probe-simple.yml
+
+echo "Downloading exporters-simple.yml into prometheus/scrape-configs/exporters/..."
+curl -fsSL -o prometheus/scrape-configs/exporters/exporters-simple.yml \
+  https://raw.githubusercontent.com/CogStack/cogstack-platform-toolkit/main/observability/examples/simple/prometheus/scrape-configs/exporters/exporters-simple.yaml
+
+
+echo "Setup complete in observability-simple/"
+
+echo "Starting the observability stack"
+docker compose up -d
+
+ echo "Please open http://localhost/grafana in your browser"