Skip to content

Giveth/giveth-monitor

Repository files navigation

giveth-monitor

A generalized monitoring stack to be installed on individual servers that we want to monitor.

Consists of:

  • cAdvisor: Exports docker container metrics for consumption by Prometheus
  • Node-Exporter: Exports server system metrics for consumption by Prometheus
  • Promtail: Exports application logs for consumption by Loki

1 - Configure and run the monitoring stack

Clone the repo

git clone https://github.com/Giveth/giveth-monitor.git

Copy .env.template to .env and fill in the values

cp .env.template .env $$ nano .env

2 - Promtail configuration

Promtail is the log client that will be shipping the logs from host machines to the Loki Server

  1. Modify the docker-compose.yml file to add a new volume for a persistent log directory
  2. Configure the paths of all application logs to monitor in the promtail-config.yml file. Out of the box giveth-monitor is configured to observe a deployed giveth-all stack.
  3. Make sure .env had been copied and modified from .env.template - promtail depends on a Loki URL

Operate

Start

docker-compose up -d

Stop

docker-compose down

Restart a service

docker-compose up -d containername

Add to a giveth-all host

If the host is configured to run caddy, you will need to open the necessary ports for the monitoring stack:

allow cAdvisor:

sudo ufw route allow proto tcp from any to any port 8080

allow node-exporter:

sudo ufw route allow proto tcp from any to any port 9100

allow promtail

sudo ufw route allow proto tcp from any to any port 9080

Or just run 01-allow-ports.sh

Install Loki Docker log driver

This is specifically configured to send the logs to the Giveth instance.

Run 02-install-loki-docker-log-driver.sh

Further configuration

Have a look at the two config files.

1 - check docker-compose.yml:

  • correct network to observe other containers:
networks:
  external_network:
    external:
      name: giveth-all_giveth

2 - check promtail-config.yml if you send more than one promtail exporter to your Loki instance:

  • the tenant_id should be unique
  • the labels for job nameand jobshould be unique