Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics #1102

Merged
43 commits merged into from Jun 24, 2021
Merged

Metrics #1102

Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
784faa4
utils, forest, blockchain: Add initial metrics
May 14, 2021
a1ee979
blockchain: Collect gossipsub_message_total metric
May 14, 2021
9352433
blockchain: Use const for metric labels
May 17, 2021
8f8bc73
blockchain: Tipset processing time metrics
May 17, 2021
6668573
blockchain: License
May 18, 2021
c388aa7
blockchain: Remove unnecessary comment
May 18, 2021
197fe3c
.maintain: Docker compose stack for monitoring
May 27, 2021
2d5b28f
blockchain: Clippy
May 27, 2021
0862588
blockchain: statrs -> 0.14.0
Jun 7, 2021
58cc60c
blockchain, node: Update metric labels
Jun 7, 2021
e05c6af
.maintain: Grafana config
Jun 9, 2021
f7422c8
.maintain, blockchain: Log, configuration changes
Jun 10, 2021
942721f
.maintain: Clean up networking
Jun 10, 2021
ef577a1
.maintain: Update host for prometheus
Jun 10, 2021
8c2af7c
utils: Remove print statement
Jun 11, 2021
010e914
utils: Manually add process metrics to collector
Jun 13, 2021
14cf7af
forest, utils: Add DB size metrics collector
Jun 15, 2021
bbfae89
utils/metrics: Get size of entire DB directory
Jun 15, 2021
2927bdd
.maintain: Upload preconfigured dashboard
Jun 15, 2021
c86e369
utils: Add license
Jun 15, 2021
a944439
forest: Flag to encrypt the keystore
Jun 16, 2021
c92aabe
blockchain: Remove metrics
Jun 17, 2021
3b70b5d
.maintain: Rename dashboard
Jun 17, 2021
44d3672
blockchain: Add head epoch gauge
Jun 18, 2021
145ee27
.maintain: Include latest head epoch in dashboard
Jun 18, 2021
8e72960
.maintain: Update documentation
Jun 18, 2021
5a1bd1b
.maintain: Update dashboard widget name
Jun 18, 2021
a4aae2d
blockchain: Remove extra clone
Jun 18, 2021
8986f7f
utils/metrics: Fail gracefully measuring DB
Jun 21, 2021
706eb42
utils/metrics: Stop collecting DB size metric
Jun 21, 2021
27a6c7d
utils/metrics: Re-enable db size metrics
Jun 21, 2021
43cccea
.maintain: Set dashboard to refresh 10s
Jun 22, 2021
825e2b4
blockchain: Address PR feedback, add peer metrics
Jun 22, 2021
745218d
blockchain: Register metrics with registry
Jun 22, 2021
d219dde
blockchain: Add print statement
Jun 22, 2021
7218d0b
blockchain: Fix full peers metric
Jun 22, 2021
194f68e
blockchain: Bug in full_peers metric
Jun 22, 2021
a45d827
.maintain/monitoring: Update forest dashboard
Jun 22, 2021
0498701
.maintain: Stylize dashboards
Jun 22, 2021
d9b0bfe
.maintain: Persist metrics, update dashboard
Jun 23, 2021
878e6f0
.maintain: Persist Prometheus data
Jun 23, 2021
d9dcf1a
.maintain: Update dashboard widget names
Jun 23, 2021
ae51372
blockchain, forest, utils: Global metrics
Jun 24, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
18 changes: 18 additions & 0 deletions .maintain/monitoring/README.md
@@ -0,0 +1,18 @@
# Metrics

## Requirements

1. `forest` node running locally
2. `docker` & `docker-compose` must be available in the `$PATH`

## Run

To run the metrics stack, use the provided Docker compose file to instantiate the `prometheus` and `grafana` processes.

``` sh
$ sudo docker-compose up --build --force-recreate -d
```

This will create a `grafana` process which is preloaded with dashboards which render metrics collected by the `prometheus` process from the `forest` node running locally.
Once the metrics stack is running, open up the `grafana` webapp to view the predefined dashboards.

32 changes: 32 additions & 0 deletions .maintain/monitoring/docker-compose.yml
@@ -0,0 +1,32 @@
# Docker compose file to start the metrics and monitoring stack for a local Forest node
#
# # Processes
# - Prometheus server
# - Grafana server

version: "3.7"
services:
prometheus:
image: prom/prometheus
entrypoint:
- "/bin/prometheus"
- "--log.level=debug"
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus"
- "--web.console.libraries=/usr/share/prometheus/console_libraries"
- "--web.console.templates=/usr/share/prometheus/consoles"
network_mode: host
volumes:
- ./prometheus/:/etc/prometheus/
restart: always

grafana:
image: grafana/grafana
user: "104"
depends_on:
- prometheus
network_mode: host
volumes:
- ./grafana/provisioning/:/etc/grafana/provisioning
- ./grafana/dashboards/:/etc/grafana/provisioning/dashboard-definitions
restart: always
9 changes: 9 additions & 0 deletions .maintain/monitoring/grafana/dashboards/README.md
@@ -0,0 +1,9 @@
# Preloaded Dashboards for Forest

## Dashboards

- `forest`: The forest dashboard keeps track of process, syncing, and execution metrics

## Updating

To update any dashboard, make changes to the dashboard in the Grafana web application, export the dashboard, and replace the dashboard JSON definition in this directory.