Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 256 lines (195 sloc) 8.836 kB
70e9491 @davisp Initial commit.
davisp authored
1 Bucky
c353f58 @davisp Use ReST for Python
davisp authored
2 -----
70e9491 @davisp Initial commit.
davisp authored
3
dbbb421 @davisp Filling out the README
davisp authored
4 Bucky is a small server for collecting and translating metrics for
5 Graphite. It can current collect metric data from CollectD daemons
6 and from StatsD clients.
70e9491 @davisp Initial commit.
davisp authored
7
dbbb421 @davisp Filling out the README
davisp authored
8 Installation
9 ------------
70e9491 @davisp Initial commit.
davisp authored
10
dbbb421 @davisp Filling out the README
davisp authored
11 You can install with `easy_install` or `pip` as per normal modus
c353f58 @davisp Use ReST for Python
davisp authored
12 operandi::
dbbb421 @davisp Filling out the README
davisp authored
13
14 $ easy_install bucky
15 # or
16 $ pip install bucky
17
c353f58 @davisp Use ReST for Python
davisp authored
18 After installing, you can run Bucky like::
dbbb421 @davisp Filling out the README
davisp authored
19
20 $ bucky
21
22 By default, Bucky will open a CollectD UDP socket on 127.0.0.1:25826,
23 a StatsD socket on 127.0.0.1:8125 as well as attempt to connect to a
24 local Graphite (Carbon) daemon on 127.0.0.1:2003.
25
26 These are all optional as illustrated below. You can also disable the
27 CollectD or StatsD servers completely if you so desire.
28
29 Running Bucky For Real
30 ----------------------
31
32 The astute observer will notice that Bucky has no flags for
33 daemonization. This is quite on purpose. The recommended way to
34 run Bucky in production is via runit. There's an example service
c353f58 @davisp Use ReST for Python
davisp authored
35 directory in Bucky's source repository.
dbbb421 @davisp Filling out the README
davisp authored
36
37 Command Line Options
38 --------------------
39
8012294 @davisp Fix formatting.
davisp authored
40 The command line options are limited to controlling the network
41 parameters. If you want to configure some of the more intricate
42 workings you'll need to use a config file. Here's the `bucky -h`
43 output::
44
c86689f @mk-fg Reconnect to Graphite on error
mk-fg authored
45 Usage: main.py [OPTIONS] [CONFIG_FILE]
46
47 Options:
48 --debug Put server into debug mode. [False]
49 --metricsd-ip=IP IP address to bind for the MetricsD UDP socket
50 [127.0.0.1]
51 --metricsd-port=INT Port to bind for the MetricsD UDP socket [23632]
52 --disable-metricsd Disable the MetricsD UDP server
53 --collectd-ip=IP IP address to bind for the CollectD UDP socket
54 [127.0.0.1]
55 --collectd-port=INT Port to bind for the CollectD UDP socket [25826]
56 --collectd-types=FILE
57 Path to the collectd types.db file, can be specified
58 multiple times
59 --disable-collectd Disable the CollectD UDP server
60 --statsd-ip=IP IP address to bind for the StatsD UDP socket
61 [127.0.0.1]
62 --statsd-port=INT Port to bind for the StatsD UDP socket [8125]
63 --disable-statsd Disable the StatsD server
64 --graphite-ip=IP IP address of the Graphite/Carbon server [127.0.0.1]
65 --graphite-port=INT Port of the Graphite/Carbon server [2003]
66 --full-trace Display full error if config file fails to load
67 --log-level=NAME Logging output verbosity [INFO]
68 --version show program's version number and exit
69 -h, --help show this help message and exit
dbbb421 @davisp Filling out the README
davisp authored
70
71
72 Config File Options
73 -------------------
74
75 The configuration file is a normal Python file that defines a number of
76 variables. Most of command line options can also be specified in this
77 file (remove the "--" prefix and replace "-" with "_") but if specified
8012294 @davisp Fix formatting.
davisp authored
78 in both places, the command line takes priority. The defaults as a
79 config file::
dbbb421 @davisp Filling out the README
davisp authored
80
55bc543 @davisp Update the config file example
davisp authored
81
82 # Standard debug and log level
83 debug = False
84 log_level = "INFO"
85
86 # Whether to print the entire stack trace for errors encountered
87 # when loading the config file
88 full_trace = False
89
90 # Basic metricsd conifguration
91 metricsd_ip = "127.0.0.1"
92 metricsd_port = 23632
93 metricsd_enabled = True
dbbb421 @davisp Filling out the README
davisp authored
94
55bc543 @davisp Update the config file example
davisp authored
95 # The default interval between flushes of metric data to Graphite
96 metricsd_default_interval = 10.0
dbbb421 @davisp Filling out the README
davisp authored
97
55bc543 @davisp Update the config file example
davisp authored
98 # You can specify the frequency of flushes to Graphite based on
99 # the metric name used for each metric. These are specified as
100 # regular expressions. An entry in this list should be a 3-tuple
101 # that is: (regexp, frequency, priority)
102 #
103 # The regexp is applied with the match method. Frequency should be
104 # in seconds. Priority is used to break ties when a metric name
105 # matches more than one handler. (The largest priority wins)
106 metricsd_handlers = []
107
108 # Basic collectd configuration
109 collectd_ip = "127.0.0.1"
110 collectd_port = 25826
111 collectd_enabled = True
dbbb421 @davisp Filling out the README
davisp authored
112
55bc543 @davisp Update the config file example
davisp authored
113 # A list of file names for collectd types.db
114 # files.
115 collectd_types = []
dbbb421 @davisp Filling out the README
davisp authored
116
55bc543 @davisp Update the config file example
davisp authored
117 # A mapping of plugin names to converter callables. These are
118 # explained in more detail in the README.
dbbb421 @davisp Filling out the README
davisp authored
119 collectd_converters = {}
120
55bc543 @davisp Update the config file example
davisp authored
121 # Whether to load converters from entry points. The entry point
122 # used to define converters is 'bucky.collectd.converters'.
dbbb421 @davisp Filling out the README
davisp authored
123 collectd_use_entry_points = True
55bc543 @davisp Update the config file example
davisp authored
124
125 # Basic statsd configuration
126 statsd_ip = "127.0.0.1"
127 statsd_port = 8125
128 statsd_enabled = True
129
130 # How often stats should be flushed to Graphite.
131 statsd_flush_time = 10.0
132
133 # Basic Graphite configuration
134 graphite_ip = "127.0.0.1"
135 graphite_port = 2003
136
137 # If the Graphite connection fails these numbers define how it
138 # will reconnect. The max reconnects applies each time a
139 # disconnect is encountered and the reconnect delay is the time
140 # in seconds between connection attempts. Setting max reconnects
141 # to a negative number removes the limit.
142 graphite_max_reconnects = 3
143 graphite_reconnect_delay = 5
144
145 # Bucky provides these settings to allow the system wide
146 # configuration of how metric names are processed before
147 # sending to Graphite.
148 #
149 # Prefix and postfix allow to tag all values with some value.
150 name_prefix = None
151 name_postfix = None
152
153 # The replacement character is used to munge any '.' characters
154 # in name components because it is special to Graphite. Setting
155 # this to None will prevent this step.
156 name_replace_char = '_'
157
158 # Optionally strip duplicates in path components. For instance
159 # a.a.b.c.c.b would be rewritten as a.b.c.b
160 name_strip_duplicates = True
dbbb421 @davisp Filling out the README
davisp authored
161
55bc543 @davisp Update the config file example
davisp authored
162 # Bucky reverses hostname components to improve the locality
163 # of metric values in Graphite. For instance, "node.company.tld"
164 # would be rewritten as "tld.company.node". This setting allows
165 # for the specification of hostname components that should
166 # be stripped from hostnames. For instance, if "company.tld"
167 # were specified, the previous example would end up as "node".
168 name_host_trim = []
dbbb421 @davisp Filling out the README
davisp authored
169
170
171 Configuring CollectD
172 --------------------
173
20a30fc @davisp ReST annoys me greatly
davisp authored
174 You should only need to add something like this to your collectd.conf::
175
dbbb421 @davisp Filling out the README
davisp authored
176 LoadPlugin "network"
177
178 <Plugin "network">
179 Server "127.0.0.1" "25826"
180 </Plugin>
181
182 Obviously, you'll want to match up the IP addresses and ports and make
183 sure that your firewall's are configured to allow UDP packets through.
184
185
186 Configuring StatsD
187 ------------------
188
189 Just point your StatsD clients at Bucky's IP/Port and you should be
190 good to go.
191
192
0118cc1 @mk-fg collectd: added support for multiple types_db's
mk-fg authored
193 Configuring MetricsD
194 ------------------
195
196 TODO
197
198
dbbb421 @davisp Filling out the README
davisp authored
199 A note on CollectD converters
200 -----------------------------
201
202 CollectD metrics aren't exactly directly translatable to Graphite
203 metric names. The default translator attempts to make a best guess
204 but this can result in slightly less than pretty Graphite trees.
205
206 For this reason, Bucky has configurable converters. These are
207 keyed off the CollectD plugin name. The input to these functions is
c353f58 @davisp Use ReST for Python
davisp authored
208 a representation of the CollectD metric that looks like such::
dbbb421 @davisp Filling out the README
davisp authored
209
210 {
211 'host': 'toroid.local',
212 'interval': 10.0,
213 'plugin': 'memory',
214 'plugin_instance': '',
215 'time': 1320970329.175534,
216 'type': 'memory',
217 'type_instance': 'inactive',
218 'value': 823009280.0,
219 'value_name': 'value',
220 'value_type': 1
221 }
222
084b986 @mk-fg collectd: treat None from sample converters as a signal to ignore sample
mk-fg authored
223 The result of this function should be a list of strings that represent
224 part of the Graphite metric name or `None` to drop sample
225 entirely. For instance, if a converter returned `["foo", "bar"]`, the
226 final metric name will end up as:
227 `$prefix.$hostname.foo.bar.$postfix`.
dbbb421 @davisp Filling out the README
davisp authored
228
c3771c8 @davisp Added a converter example
davisp authored
229 An example builtin converter looks like such::
230
231 # This might be how you define a converter in
232 # your config file
233
234 class MemoryConverter(object):
235 PRIORITY = 0
236 def __call__(self, sample):
237 return ["memory", sample["type_instance"]]
238
bc0d3dd @mk-fg README: updated collectd_converters definition example to reflect the…
mk-fg authored
239 collectd_converters = {"memory": MemoryConverter()}
c3771c8 @davisp Added a converter example
davisp authored
240
dbbb421 @davisp Filling out the README
davisp authored
241 Converters can either be declared and/or imported in the optional
242 config file, or they can be autodiscovered via entry points. The
243 entry point that is searched is "bucky.collectd.converters". The
244 entry point name should be the CollectD plugin name.
c7213e4 @mk-fg Allow the default converter to be overridden
mk-fg authored
245
246 `collectd_converters` in config file should be a mapping of collectd
247 plugin name to converter instance. The default catch-all converter
248 (used when no special converter is defined for a plugin) can be
249 overidden by specifying `_default` as the plugin name.
250
251 Converters also have a notion of priority in order to resolve
252 conflicts. This is merely a property on the callable named
253 "PRIORITY" and larger priorities are preferred. I don't imagine
254 this will need to be used very often, but its there just in
255 case.
Something went wrong with that request. Please try again.