statsd plugin: please be more flexible with parsing

[davidc0le]

With version 5.4 , I see many messages from my test setup saying that statsd messages cannot be parsed :

Nov  5 18:45:20 hostname collectd[8236]: statsd plugin: Unable to parse line: "object-server.REPLICATE.timing:0.399112701416|ms|@0.1"
Nov  5 18:45:20 hostname collectd[8236]: statsd plugin: Unable to parse line: "object-server.REPLICATE.timing:0.399112701416|ms|@0.1"
Nov  5 18:45:20 hostname collectd[8236]: statsd plugin: Unable to parse line: "object-server.REPLICATE.timing:0.375986099243|ms|@0.1"
Nov  5 18:45:20 hostnam collectd[8236]: statsd plugin: Unable to parse line: "object-server.REPLICATE.timing:0.375986099243|ms|@0.1"

I understand that the extra field ( after the type field "ms") are unnecessary for timing messages.

Within the plugin statsd.c , in statsd_parse_line() we return -1 if there is extra data for non-counters:

  /* extra is only valid for counters */
  if (extra != NULL)
    return (-1);

Other statsd parsers would allow this, would you mind please making your code more flexible - maybe just removing the two lines above ?

Thanks

[dvenza]

Yes, please. It is Swift that is generating these lines with extra data appended and it needs to be fixed, but in the meantime collectd is flooding the logs and loosing valuable data.

[johnl]

This bug has been reported in swift: https://bugs.launchpad.net/swift/+bug/1248348

But having done a bit more investigation, it is collectd which is at fault.

According to the statsd docs, timing (ms) metric types should not have sample rates:

https://github.com/etsy/statsd/blob/master/docs/metric_types.md#timing

But if you look at the statsd code, it actually does use the sample rate if it's available for timing stats:

https://github.com/etsy/statsd/blob/bad366cd438cd59b2064dd03ff3d415fc7db0c4b/stats.js#L206-L225

So technically this is a bug in collectd, but an understandable one given the faulty statsd docs.

Collectd needs to accept sample rates for timing metric types and use them accordingly.

[johnl]

as above, I've addd support for the samplerate field in timing metric types to collectd. I've built some Ubuntu Trusty packages for testing (they're the upstream Ubuntu upstream packages modified only to add this patch). The packages are available on this ppa: https://launchpad.net/~brightbox/+archive/ubuntu/collectd-statsd-testing

[mfournier]

This was fixed in collectd 5.4.2. Closing. Thanks to everyone for reporting and tracking this down !