Kamila's monitoring toolkit. Revolves around Prometheus.
Note: I made this for DataCenterLight. Check it out!
cdist is an interesting take on configuration management developed at ungleich. It is simple, versatile, and has no dependencies on the target system. In fact, it is a bit too simple compared to other, far more sophisticated configuration management solutions -- and that's precisely why I like it.
I use it for configuring all my monitoring servers (among other things). Below you can find a cdist manifest snippet that comes from our real configuration at DataCenterLight.
I have created several cdist types useful for installing a Prometheus monitoring server, as well as many exporters:
__prometheus_server
: Installs and configures Prometheus server. Yay!__prometheus_alertmanager
: Installs and configures Prometheus Alertmanager. Yay too!__grafana_dashboard
: Installs and configures Grafana. Yay three!
Daemontools are awesome. In line with the Unix philosophy, they do one thing and do it well. In this case they make sure that services run. Without the messy parts!
__daemontools
: Installs daemontools or your favorite derivative, plus an init script in case your package doesn't come with one.__daemontools_service
: Creates a service directory. I gladly use this for everything that doesn't come with a good init script, such as some exporters. It's so simple!
Most distros provide a reasonably new version of Prometheus and the basic exporters (worst case you'll have to use the --install-from-backports
with my __prometheus_server
). This is the recommended way. However, if you need something more, these types let you install a recent version of golang and go packages. Note that I am currently not using these in production. I am listing them here anyway for completeness.
__golang_from_vendor
: Installs golang from https://golang.org. This makes it easy to install any version of golang, including those that your old and obsolete distro doesn't have.__go_get
: Easy installation of golang packages straight from online repos. Useful for installing a newer version of Prometheus or various exporters.
Put this in you cdist manifest, edit to your needs, and rejoice!
Monitoring server:
__prometheus_server \
--install-from-backports \
--config "$__manifest/files/prometheus.yml" \
--retention-days 14 \
--storage-path /data/prometheus \
--rule-files "$__manifest/files/*.rules"
__prometheus_alertmanager \
--install-from-backports \
--config "$__manifest/files/alertmanager.yml" \
--storage-path /data/alertmanager
__grafana_dashboard
git clone https://github.com/ungleich/cdist.git
cd cdist
$EDITOR cdist/conf/manifest/init
bin/cdist config -vv <hostname>
Read more at cdist quickstart.
Questions, ideas, and pull requests are welcome!
- ttn2prom: Trivial Python webserver that accepts webhooks (i.e. HTTP requests), processes the data in them, and exports the stuff in the Prometheus format. Very easy to adapt. This version exports data from some LoRaWan temperature sensors through webhooks from The Things Network.
- opennebula exporter: Exports useful metrics for OpenNebula. For now it is an evil hack, but it will get better with time.
- OpenNebula integration test: tests VM creation and connectivity, and exports the results in a Prometheus-compatible format.
TBD