statsd: option to namespace all metrics

[igor47]

we are currently unable to run datadog on the machines in our staging or
testing clusters. this is because the code in the applications running there is
set to report metrics directly to datadog. those non-production metrics
nontheless become mixed in with production metrics in datadog.

there are two existing solutions:

• in datadog, always specify an environment tag to exclude non-production metrics
• in applications, always manually namespace the metrics

the first is difficult. we already have many dashboards and alerts which don't
specify any environment. also, naive examination in, e.g., the metric explorer
will still conflate the metrics and cause confusion.

the second is annoying, because we have to make an identical configuration
change throughout multiple applications.

instead, we propose the mechanism here, which allows statsd metrics being
reported to be automatically namespaced.

i wasn't sure how to test this functionality, since there are no existing tests
for the init function. i am also not sure about grabbing a direct reference
to the api_formatter function elsewhere in the code.

[igor47]

ping. any update on this? it would be really helpful for us

[alq666]

@remh can you review and consider for inclusion in 5.2.0?

[remh]

Looks good!
Could you put that in a new function something like:

def get_formatter(c):
  formatter = api_formatter

  if c['statsd_metric_namespace']:
    def metric_namespace_formatter_wrapper(*args, **kwargs):
      metric_prefix = c['statsd_metric_namespace']
      if metric_prefix[-1] != '.':
        metric_prefix += '.'

      metric = args[0]
      new_metric = metric_prefix + metric
      new_args = [new_metric] + args[1:]
      return api_formatter(*new_args, **kwargs)

    formatter = metric_namespace_formatter_wrapper
  return formatter

and then call that function in the init() function ?

[remh]

Looks good beside the nitpick! Thanks @igor47 we'll merge it for 5.2.0

For tests, you can add some here: https://github.com/DataDog/dd-agent/blob/master/tests/test_dogstatsd.py where you would pass the formatter returned by the get_formatter function to the aggregator.

And then test for the metric names same as we do with other tests in that test module.
Does that make sense?
Thanks!