Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

updated new namespacing to support suffixes instead of prefixes #215

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

updated new namespacing to support suffixes instead of prefixes #215

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
53cc84a
updated new namespacing to support prefixes instead of suffixes - all…
Dec 17, 2012
57799b5
update suffix patch to address file vs directory name conflict possib…
Jan 8, 2013
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,17 @@ prefixGauge: graphite prefix for gauge metrics [default: "gauges"]
prefixSet: graphite prefix for set metrics [default: "sets"]
```

Example prefix config:
```
, graphite: { legacyNamespace: false
, prefixCounter: "counters"
, prefixTimer: "timers"
, prefixGauge: "gauges"
, prefixSet: "sets"
, globalPrefix: "stats"
, }
```

If you decide not to use the legacy namespacing, besides the obvious changes
in the prefixing, there will also be a breaking change in the way counters are
submitted. So far counters didn't live under any namespace and were also a bit
Expand All @@ -343,6 +354,94 @@ under `stats.counters.counter_name.rate` and
`stats.counters.counter_name.count` now.


Along with disabling legacy namespacing, there's an option to use suffixes
instead of prefixes to label the data. When "useSuffixNames" is set to true,
prefixes are ignored and suffixes are used instead.

Example suffix config:
```
, graphite: { legacyNamespace: false
, useSuffixNames: true
, suffixCounter: "counters"
, suffixTimer: "timers"
, suffixGauge: "gauges"
, suffixSet: "sets"
, }
```

Some namespacing examples:

Legacy namespacing:
```
stats:
NAME-c.wsp gauges sets statsd timers
stats/gauges:
NAME-g.wsp
stats/sets/NAME-s:
count.wsp
stats/statsd:
bad_lines_seen.wsp graphiteStats packets_received.wsp processing_time.wsp
stats/statsd/graphiteStats:
calculationtime.wsp last_exception.wsp last_flush.wsp
stats/timers/NAME-ms:
count.wsp mean.wsp std.wsp sum_90.wsp upper_90.wsp
lower.wsp mean_90.wsp sum.wsp upper.wsp
stats_counts:
NAME-c.wsp statsd
stats_counts/statsd:
bad_lines_seen.wsp packets_received.wsp
statsd:
numStats.wsp
```

Prefix namespacing:
```
stats/counters:
NAME-c statsd
stats/counters/NAME-c:
count.wsp rate.wsp
stats/counters/statsd/bad_lines_seen:
count.wsp rate.wsp
stats/counters/statsd/packets_received:
count.wsp rate.wsp
stats/gauges:
NAME-g.wsp
stats/sets/NAME-s:
count.wsp
stats/statsd:
graphiteStats numStats.wsp processing_time.wsp
stats/statsd/graphiteStats:
calculationtime.wsp last_exception.wsp last_flush.wsp
stats/timers/NAME-ms:
count.wsp mean.wsp std.wsp sum_90.wsp upper_90.wsp
lower.wsp mean_90.wsp sum.wsp upper.wsp
```

Suffix namespacing:
```
stats/NAME-c/counters:
count.wsp rate.wsp
stats/NAME-g:
gauges.wsp
stats/NAME-ms/timers:
count.wsp mean.wsp std.wsp sum_90.wsp upper_90.wsp
lower.wsp mean_90.wsp sum.wsp upper.wsp
stats/NAME-s/sets:
count.wsp
stats/statsd:
bad_lines_seen numStats.wsp processing_time.wsp
graphiteStats packets_received
stats/statsd/bad_lines_seen/counters:
count.wsp rate.wsp
stats/statsd/graphiteStats:
calculationtime.wsp last_exception.wsp last_flush.wsp
stats/statsd/packets_received/counters:
count.wsp rate.wsp
```




Inspiration
-----------

Expand Down
73 changes: 58 additions & 15 deletions backends/graphite.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ var prefixTimer;
var prefixGauge;
var prefixSet;

// suffix configuration
var suffixCounter;
var suffixTimer;
var suffixGauge;
var suffixSet;

// set up namespaces
var legacyNamespace = true;
var globalNamespace = [];
Expand Down Expand Up @@ -83,7 +89,11 @@ var flush_stats = function graphite_flush(ts, metrics) {
var statsd_metrics = metrics.statsd_metrics;

for (key in counters) {
var namespace = counterNamespace.concat(key);
if (useSuffixNames === true) {
var namespace = counterNamespace.concat(key,suffixCounter);
} else {
var namespace = counterNamespace.concat(key);
}
var value = counters[key];
var valuePerSecond = counter_rates[key]; // pre-calculated "per second" rate

Expand All @@ -101,7 +111,11 @@ var flush_stats = function graphite_flush(ts, metrics) {
for (key in timer_data) {
if (Object.keys(timer_data).length > 0) {
for (timer_data_key in timer_data[key]) {
var namespace = timerNamespace.concat(key);
if (useSuffixNames === true) {
var namespace = timerNamespace.concat(key,suffixTimer);
} else {
var namespace = timerNamespace.concat(key);
}
var the_key = namespace.join(".");
statString += the_key + '.' + timer_data_key + ' ' + timer_data[key][timer_data_key] + ts_suffix;
}
Expand All @@ -111,13 +125,21 @@ var flush_stats = function graphite_flush(ts, metrics) {
}

for (key in gauges) {
var namespace = gaugesNamespace.concat(key);
if (useSuffixNames === true) {
var namespace = gaugesNamespace.concat(key,suffixGauge);
} else {
var namespace = gaugesNamespace.concat(key);
}
statString += namespace.join(".") + ' ' + gauges[key] + ts_suffix;
numStats += 1;
}

for (key in sets) {
var namespace = setsNamespace.concat(key);
if (useSuffixNames === true) {
var namespace = setsNamespace.concat(key,suffixSet);
} else {
var namespace = setsNamespace.concat(key);
}
statString += namespace.join(".") + '.count ' + sets[key].values().length + ts_suffix;
numStats += 1;
}
Expand Down Expand Up @@ -159,6 +181,12 @@ exports.init = function graphite_init(startup_time, config, events) {
prefixSet = config.graphite.prefixSet;
legacyNamespace = config.graphite.legacyNamespace;

useSuffixNames = config.graphite.useSuffixNames;
suffixCounter = config.graphite.suffixCounter;
suffixTimer = config.graphite.suffixTimer;
suffixGauge = config.graphite.suffixGauge;
suffixSet = config.graphite.suffixSet;

// set defaults for prefixes
globalPrefix = globalPrefix !== undefined ? globalPrefix : "stats";
prefixCounter = prefixCounter !== undefined ? prefixCounter : "counters";
Expand All @@ -167,6 +195,17 @@ exports.init = function graphite_init(startup_time, config, events) {
prefixSet = prefixSet !== undefined ? prefixSet : "sets";
legacyNamespace = legacyNamespace !== undefined ? legacyNamespace : true;

// set defaults for suffixes
useSuffixNames = useSuffixNames !== undefined ? useSuffixNames : false;
suffixCounter = suffixCounter !== undefined ? suffixCounter : "counters";
suffixTimer = suffixTimer !== undefined ? suffixTimer : "timers";
// gauges is a special case. to deconflict with possible keys and to keep change limited
// https://github.com/etsy/statsd/pull/215#issuecomment-12007571
suffixGauge = suffixGauge !== undefined ? suffixGauge : "gauges";
// append gauges to whatever suffix you've defined to give you suffixGauge/gauges.wsp
// when it hits carbon
suffixGauge = suffixGauge + ".gauges";
suffixSet = suffixSet !== undefined ? suffixSet : "sets";

if (legacyNamespace === false) {
if (globalPrefix !== "") {
Expand All @@ -177,18 +216,22 @@ exports.init = function graphite_init(startup_time, config, events) {
setsNamespace.push(globalPrefix);
}

if (prefixCounter !== "") {
counterNamespace.push(prefixCounter);
}
if (prefixTimer !== "") {
timerNamespace.push(prefixTimer);
}
if (prefixGauge !== "") {
gaugesNamespace.push(prefixGauge);
}
if (prefixSet !== "") {
setsNamespace.push(prefixSet);
// use prefixes if we're not using suffixes
if (useSuffixNames === false) {
if (prefixCounter !== "") {
counterNamespace.push(prefixCounter);
}
if (prefixTimer !== "") {
timerNamespace.push(prefixTimer);
}
if (prefixGauge !== "") {
gaugesNamespace.push(prefixGauge);
}
if (prefixSet !== "") {
setsNamespace.push(prefixSet);
}
}

} else {
globalNamespace = ['stats'];
counterNamespace = ['stats'];
Expand Down
6 changes: 6 additions & 0 deletions exampleConfig.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,12 @@ Optional Variables:
prefixTimer: graphite prefix for timer metrics [default: "timers"]
prefixGauge: graphite prefix for gauge metrics [default: "gauges"]
prefixSet: graphite prefix for set metrics [default: "sets"]
useSuffixNames: use the suffix namespacing instead of prefix (requires legacyNamespace=false) [default: false]
suffixCounter: graphite suffix for counter metrics [default: "counters"]
suffixTimer: graphite suffix for timer metrics [default: "timers"]
suffixGauge: graphite suffix for gauge metrics [default: "gauges"]
suffixSet: graphite suffix for set metrics [default: "sets"]


repeater: an array of hashes of the for host: and port:
that details other statsd servers to which the received
Expand Down
Loading