A network daemon for aggregating statistics (counters and timers), rolling them up, then sending them to graphite.
buckets Each stat is in it's own "bucket". They are not predefined anywhere. Buckets can be named anything that will translate to Graphite (periods make folders, etc)
values Each stat will have a value. How it is interpreted depends on modifiers
flush After the flush interval timeout (default 10 seconds), stats are munged and sent over to Graphite.
This is a simple counter. Add 1 to the "gorets" bucket. It stays in memory until the flush interval.
The glork took 320ms to complete this time. StatsD figures out 90th percentile, average (mean), lower and upper bounds for the flush interval.
Tells StatsD that this counter is being sent sampled every 1/10th of the time.
Tells StatsD that the "load" average keys should be averaged before being reported.
Will buffer a set of commands to send to carbon without any preprocessing. (As the major point of StatsD is to serve as an aggregator, be careful with this metric. Try to use counts, timers, and averages when you can instead of pushing lots of raw data to carbon. However, to keep reporting interfaces consistent, this is useful.)
Graphite uses "schemas" to define the different round robin datasets it houses (analogous to RRAs in rrdtool). Here's what Etsy is using for the stats databases:
[stats] priority = 110 pattern = ^stats\..* retentions = 10:2160,60:10080,600:262974
That translates to:
- 6 hours of 10 second data (what we consider "near-realtime")
- 1 week of 1 minute data
- 5 years of 10 minute data
This has been a good tradeoff so far between size-of-file (round robin databases are fixed size) and data we care about. Each "stats" database is about 3.2 megs with these retentions.
Installation and Configuration
- Install node.js
- Clone the project
- Create a config file from exampleConfig.js and put it somewhere
Start the Daemon:
node stats.js /path/to/config
You're interested in contributing to StatsD? AWESOME. Here are the basic steps:
fork StatsD from here: http://github.com/etsy/statsd
- Clone your fork
- Hack away
- If you are adding new functionality, document it in the README
- If necessary, rebase your commits into logical chunks, without errors
- Push the branch up to GitHub
- Send a pull request to the etsy/statsd project.
We'll do our best to get your changes in!
In lieu of a list of contributors, check out the commit history for the project: http://github.com/etsy/statsd/commits/master