The Ubuntu project has an instance of Grafana which is a place for Ubuntu teams to track their metrics.
This repository hosts the scripts which collect data and push it to the Influxdb database that the Grafana reads from.
There are three branches in this repository:
metrics
. The default branch which contains the scripts themselves. These will be run periodically.charm
. This branch contains the Juju charm to deploy an instance which checks outmetrics
and arranges for them to be run regularly.mojo
. This is a delivery system for Juju charms. Most people won't need to touch this, but the Mojo spec is run on a "controller" host to arrange to collect secrets - the InfluxDB credentials - and deploy the charm.
Collectors are python modules in the metrics.collectors
namespace which
expose a run_metric()
function.
The simplest way to write a collector is to import metrics.lib.basemetric
and write a class which inherits from Metric
in there. Provide a collect
function which returns data points suitable for passing to
InfluxDBClient.write_points().
Metric
will take care of submitting to Influx for you when you invoke
run()
, which your run_metric()
top level function should do. For
development, when invoked as a script (by providing a __main__.py
file),
you can pass --dry-run
to see what would be submitted. Make use of
self.log
to log progress. With --verbose
, debug messages will be logged.
Raise a metrics.lib.errors.CollectorError
on error, and the message will be
reported along with the failure.
If you make your collector run when executed too (python3 -m metrics.collectors.your_collector
), you can provide a symlink in bin/
so
that it's easier for people to run for testing purposes.
But when starting a new metric, just copy the structure of an existing script!
The metrics
branch is auto pulled, so after merging your new collector will
be run automatically.
The charm will handle arranging for each metric to be run periodically, but
you can control how frequently this happens. Provide a top level variable in
your __init__.py
called RUN_INTERVAL
with a value suitable for passing
to OnUnitInactiveSec
in a systemd timer
unit.
The default value if you don't specify this is 5m
.
This is for the deployment itself, and not any collectors which run inside it.
You need to have access to the live environment, which means you need to be a Canonical employee.
Build the charm using charmcraft build
(charmcraft
comes from a snap:
snap install charmcraft
). Copy the output of the build
directory to the
build
branch, and push it. Then execute mojo run -m manifest-upgrade
in
the live environment to re-run the Mojo spec (which anyone can see in the
mojo
branch here).