Skip to content
Newer
Older
100644 101 lines (64 sloc) 2.88 KB
75097d6 @kastner adding a basic readme
kastner authored
1 StatsD
2 ======
3
a0636b8 @kastner Edited README.md via GitHub
kastner authored
4 A network daemon for aggregating statistics (counters and timers), rolling them up, then sending them to [graphite][graphite].
5
6 We ([Etsy][etsy]) [blogged][blog post] about how it works and why we created it.
75097d6 @kastner adding a basic readme
kastner authored
7
8
9 Concepts
10 --------
11
12 * *buckets*
13 Each stat is in it's own "bucket". They are not predefined anywhere. Buckets can be named anything that will translate to Graphite (periods make folders, etc)
14
15 * *values*
16 Each stat will have a value. How it is interpreted depends on modifiers
17
18 * *flush*
19 After the flush interval timeout (default 10 seconds), stats are munged and sent over to Graphite.
20
21 Counting
22 --------
23
24 gorets:1|c
25
26 This is a simple counter. Add 1 to the "gorets" bucket. It stays in memory until the flush interval.
27
28
29 Timing
30 ------
31
32 glork:320|ms
33
34 The glork took 320ms to complete this time. StatsD figures out 90th percentile, average (mean), lower and upper bounds for the flush interval.
35
36 Sampling
37 --------
38
39 gorets:1|c|@0.1
40
41 Tells StatsD that this counter is being sent sampled ever 1/10th of the time.
42
43
44 Guts
45 ----
46
a0636b8 @kastner Edited README.md via GitHub
kastner authored
47 * [UDP][udp]
75097d6 @kastner adding a basic readme
kastner authored
48 Client libraries use UDP to send information to the StatsD daemon.
49
a0636b8 @kastner Edited README.md via GitHub
kastner authored
50 * [NodeJS][node]
51 * [Graphite][graphite]
d5c62bd @kastner adding link to the flickr post about their StatsD
kastner authored
52
c14677e @kastner Add the schema config for stats.* in graphite
kastner authored
53 Graphite uses "schemas" to define the different round robin datasets it houses (analogous to RRAs in rrdtool). Here's what Etsy is using for the stats databases:
54
55 [stats]
56 priority = 110
57 pattern = ^stats\..*
58 retentions = 10:2160,60:10080,600:262974
59
3009ed8 @kastner more info on our graphite settings for stats.*
kastner authored
60 That translates to:
61 * 6 hours of 10 second data (what we consider "near-realtime")
62 * 1 week of 1 minute data
63 * 5 years of 10 minute data
64
65 This has been a good tradeoff so far between size-of-file (round robin databases are fixed size) and data we care about. Each "stats" database is about 3.2 megs with these retentions.
c14677e @kastner Add the schema config for stats.* in graphite
kastner authored
66
d5c62bd @kastner adding link to the flickr post about their StatsD
kastner authored
67 Inspiration
68 -----------
69
70 StatsD was inspired (heavily) by the project (of the same name) at Flickr. Here's a post where Cal Henderson described it in depth:
1b4d284 @kastner Adding a pointer to the "other" StatsD
kastner authored
71 [Counting and timing](http://code.flickr.com/blog/2008/10/27/counting-timing/). Cal re-released the code recently: [Perl StatsD](https://github.com/iamcal/Flickr-StatsD)
d5c62bd @kastner adding link to the flickr post about their StatsD
kastner authored
72
a0636b8 @kastner Edited README.md via GitHub
kastner authored
73
74 Contribute
75 ---------------------
76
77 You're interested in contributing to StatsD? *AWESOME*. Here are the basic steps:
78
79 fork StatsD from here: http://github.com/etsy/statsd
80
81 1. Clone your fork
82 2. Hack away
83 3. If you are adding new functionality, document it in the README
84 4. If necessary, rebase your commits into logical chunks, without errors
85 5. Push the branch up to GitHub
86 6. Send a pull request to the etsy/statsd project.
87
88 We'll do our best to get your changes in!
89
90 [graphite]: http://graphite.wikidot.com
91 [etsy]: http://www.etsy.com
92 [blog post]: http://codeascraft.etsy.com/2011/02/15/measure-anything-measure-everything/
93 [node]: http://nodejs.org
94 [udp]: http://enwp.org/udp
95
96
97 Contributors
98 -----------------
99
100 In lieu of a list of contributors, check out the commit history for the project:
101 http://github.com/etsy/statsd/commits/master
Something went wrong with that request. Please try again.