Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 147 lines (94 sloc) 5.106 kb
75097d6 Erik Kastner adding a basic readme
kastner authored
1 StatsD
2 ======
3
a0636b8 Erik Kastner Edited README.md via GitHub
kastner authored
4 A network daemon for aggregating statistics (counters and timers), rolling them up, then sending them to [graphite][graphite].
5
6 We ([Etsy][etsy]) [blogged][blog post] about how it works and why we created it.
75097d6 Erik Kastner adding a basic readme
kastner authored
7
8
9 Concepts
10 --------
11
12 * *buckets*
458594b Wei Hsu Update README.md
yayitswei authored
13 Each stat is in its own "bucket". They are not predefined anywhere. Buckets can be named anything that will translate to Graphite (periods make folders, etc)
75097d6 Erik Kastner adding a basic readme
kastner authored
14
15 * *values*
16 Each stat will have a value. How it is interpreted depends on modifiers
845b294 Dave Dash Fixed a typo in the README.
davedash authored
17
75097d6 Erik Kastner adding a basic readme
kastner authored
18 * *flush*
19 After the flush interval timeout (default 10 seconds), stats are munged and sent over to Graphite.
20
21 Counting
22 --------
23
24 gorets:1|c
25
99b8a15 Jerry Chen Documented some more config variables in exampleConfig.js and README
jcsalterego authored
26 This is a simple counter. Add 1 to the "gorets" bucket. It stays in memory until the flush interval `config.flushInterval`.
75097d6 Erik Kastner adding a basic readme
kastner authored
27
28
29 Timing
30 ------
31
32 glork:320|ms
33
99b8a15 Jerry Chen Documented some more config variables in exampleConfig.js and README
jcsalterego authored
34 The glork took 320ms to complete this time. StatsD figures out 90th percentile, average (mean), lower and upper bounds for the flush interval. The percentile threshold can be tweaked with `config.percentThreshold`.
75097d6 Erik Kastner adding a basic readme
kastner authored
35
36 Sampling
37 --------
38
39 gorets:1|c|@0.1
40
3b07b5a avleen Typo in README.md
avleen authored
41 Tells StatsD that this counter is being sent sampled every 1/10th of the time.
75097d6 Erik Kastner adding a basic readme
kastner authored
42
99b8a15 Jerry Chen Documented some more config variables in exampleConfig.js and README
jcsalterego authored
43 Debugging
44 ---------
45
46 There are additional config variables available for debugging:
47
48 * `debug` - log exceptions and periodically print out information on counters and timers
49 * `debugInterval` - interval for printing out information on counters and timers
50 * `dumpMessages` - print debug info on incoming messages
51
52 For more information, check the `exampleConfig.js`.
75097d6 Erik Kastner adding a basic readme
kastner authored
53
54 Guts
55 ----
56
a0636b8 Erik Kastner Edited README.md via GitHub
kastner authored
57 * [UDP][udp]
75097d6 Erik Kastner adding a basic readme
kastner authored
58 Client libraries use UDP to send information to the StatsD daemon.
59
a0636b8 Erik Kastner Edited README.md via GitHub
kastner authored
60 * [NodeJS][node]
61 * [Graphite][graphite]
d5c62bd Erik Kastner adding link to the flickr post about their StatsD
kastner authored
62
c14677e Erik Kastner Add the schema config for stats.* in graphite
kastner authored
63 Graphite uses "schemas" to define the different round robin datasets it houses (analogous to RRAs in rrdtool). Here's what Etsy is using for the stats databases:
64
65 [stats]
845b294 Dave Dash Fixed a typo in the README.
davedash authored
66 priority = 110
c14677e Erik Kastner Add the schema config for stats.* in graphite
kastner authored
67 pattern = ^stats\..*
68 retentions = 10:2160,60:10080,600:262974
69
3009ed8 Erik Kastner more info on our graphite settings for stats.*
kastner authored
70 That translates to:
ed5a1ed Erik Kastner Damn you markdown!
kastner authored
71
3009ed8 Erik Kastner more info on our graphite settings for stats.*
kastner authored
72 * 6 hours of 10 second data (what we consider "near-realtime")
73 * 1 week of 1 minute data
74 * 5 years of 10 minute data
75
76 This has been a good tradeoff so far between size-of-file (round robin databases are fixed size) and data we care about. Each "stats" database is about 3.2 megs with these retentions.
c14677e Erik Kastner Add the schema config for stats.* in graphite
kastner authored
77
d2c7540 Updated readme with details on tcp management interface
Marcus Barczak authored
78 TCP Stats Interface
79 -------------------
80
81 A really simple TCP management interface is available by default on port 8126 or overriden in the configuration file. Inspired by the memcache stats approach this can be used to monitor a live statsd server. You can interact with the management server by telnetting to port 8126, the following commands are available:
82
83 * stats - some stats about the running server
84 * counters - a dump of all the current counters
85 * timers - a dump of the current timers
86
87 The stats output currently will give you:
88
89 * uptime: the number of seconds elapsed since statsd started
90 * graphite.last_flush: the number of seconds elapsed since the last successful flush to graphite
91 * graphite.last_exception: the number of seconds elapsed since the last exception thrown whilst flushing to graphite
92 * messages.last_msg_seen: the number of elapsed seconds since statsd received a message
93 * messages.bad_lines_seen: the number of bad lines seen since startup
94
3adbb3d Soenke Ruempler added simple install / config instructions
s0enke authored
95 Installation and Configuration
96 ------------------------------
97
98 * Install node.js
99 * Clone the project
100 * Create a config file from exampleConfig.js and put it somewhere
101 * Start the Daemon:
102
103 node stats.js /path/to/config
104
766876b Mike Stipicevic Add basic test framework
wickedchicken authored
105 Tests
106 -----
107
108 A test framework has been added using node-unit and some custom code to start and manipulate statsd. Please add tests under test/ for any new features or bug fixes encountered. Testing a live server can be tricky, attempts were made to eliminate race conditions but it may be possible to encounter a stuck state. If doing dev work, a `killall node` will kill any stray test servers in the background (don't do this on a production machine!).
109
110 Tests can be executd with `./run_tests.sh`.
3adbb3d Soenke Ruempler added simple install / config instructions
s0enke authored
111
d5c62bd Erik Kastner adding link to the flickr post about their StatsD
kastner authored
112 Inspiration
113 -----------
114
115 StatsD was inspired (heavily) by the project (of the same name) at Flickr. Here's a post where Cal Henderson described it in depth:
1b4d284 Erik Kastner Adding a pointer to the "other" StatsD
kastner authored
116 [Counting and timing](http://code.flickr.com/blog/2008/10/27/counting-timing/). Cal re-released the code recently: [Perl StatsD](https://github.com/iamcal/Flickr-StatsD)
d5c62bd Erik Kastner adding link to the flickr post about their StatsD
kastner authored
117
a0636b8 Erik Kastner Edited README.md via GitHub
kastner authored
118
119 Contribute
120 ---------------------
121
122 You're interested in contributing to StatsD? *AWESOME*. Here are the basic steps:
123
124 fork StatsD from here: http://github.com/etsy/statsd
125
126 1. Clone your fork
127 2. Hack away
128 3. If you are adding new functionality, document it in the README
129 4. If necessary, rebase your commits into logical chunks, without errors
130 5. Push the branch up to GitHub
131 6. Send a pull request to the etsy/statsd project.
132
133 We'll do our best to get your changes in!
134
135 [graphite]: http://graphite.wikidot.com
136 [etsy]: http://www.etsy.com
137 [blog post]: http://codeascraft.etsy.com/2011/02/15/measure-anything-measure-everything/
138 [node]: http://nodejs.org
139 [udp]: http://enwp.org/udp
140
141
142 Contributors
143 -----------------
144
845b294 Dave Dash Fixed a typo in the README.
davedash authored
145 In lieu of a list of contributors, check out the commit history for the project:
3b07b5a avleen Typo in README.md
avleen authored
146 http://github.com/etsy/statsd/commits/master
Something went wrong with that request. Please try again.