Domo arigato, Mr. Librato
Arigato is a Python script for performing a one-time migration of your Graphite metrics to the Librato monitoring service. It's assumed that you've already transitioned your live metrics feed over to Librato. This utility should only be used to backfill your historical data.
Rather than attempting to coerce the raw Whisper archives to match Librato's resolution levels, Arigato performs logical queries against the Graphite API, using
summarize() to roll the results up into compatible resolutions for each interval.
Note: By default, Librato will not accept any data older than 2 hours. To bypass this restriction you'll need to contact Librato Support for a temporary lifting of this policy using their beta "historical import" feature. Users will experience a slight delay (before the data is visible in the UI) while Librato generates summary statistics from the new data.
$ pip install librato-metrics
The following variables are used to authenticate with the Librato API and are mandatory. See this article which explains how to create your own Librato API token.
LIBRATO_USER- Your account email associated with the token.
LIBRATO_TOKEN- Your Record Only API token.
The following arguments are available.
-n, --node- Zero-indexed node from the Graphite metric string to use as the Librato source. The default value is
-p, --prefix- A string prefix for the Librato metric. All whitespace and leading/trailing dots are removed from the string before prefixing to the original metric string. Delimiters found within the prefix are preserved (e.g.
foo.bar) but multiples (e.g.
foo..bar) will be normalized to a single dot. There is no default prefix.
-s, --source- Manual override of the
sourcevalue derived from
node. When set to a string, it will set the source to that value. When set to an empty string (e.g.
-s ""), it will report the source as
-u, --url- The Graphite server URL to query. Uses
Arigato accepts a list of metric names passed to
In both of the examples below, we're filtering the results to only match
collectd results, so the
node argument doesn't need to be specified. The default collectd format passes the hostname in the 2nd (index value
1) node of the metric string.
Note: It may be necessary to migrate portions of your metric namespace at a time to ensure proper
Using Carbonate locally
carbon-list utility provided by the Carbonate project is ideal for listing metrics on the Graphite server itself. The results can then by piped into Arigato.
$ export PYTHONPATH=$PYTHONPATH:/opt/graphite/lib $ export LIBRATO_USER=abc $ export LIBRATO_TOKEN=123 $ carbon-list | grep '^collectd' | \ python arigato/script.py -p migrated Processing collectd.graphite-0-1.load.load.shortterm: Archive submitted successfully Archive submitted successfully Archive submitted successfully Archive submitted successfully ...
Curl a Remote Graphite
This method is slightly more convenient if you don't have local access to your Graphite server or you can't install Carbonate or you want to iterate through a cluster Graphite servers. It does require a bit more effort to prep the output before passing it onto Arigato, however.
$ export LIBRATO_USER=abc $ export LIBRATO_TOKEN=123 $ curl -s http://graphite/metrics/index.json | python -m json.tool | \ grep '"' | sed 's/ //g' | sed 's/"//g' | sed 's/,//g' | grep '^collectd' | \ python arigato/script.py -p migrated Processing collectd.graphite-0-1.load.load.shortterm: Archive submitted successfully Archive submitted successfully Archive submitted successfully Archive submitted successfully ...
The following screenshots demonstrate Arigato's ability to maintain the proper resolution during the migration. The first chart is a 3-hour window of a single metric on the source Graphite server. The second chart is the same 3-hour window after the metric has been copied to the Librato service.
Arigato is distributed under the MIT license.