Add instrumentation hooks

[josegonzalez]

It should be possible to instrument the application with your favorite metrics suite - StatsD comes to mind, as most things speak it's language.

This would be useful for the following metrics:

• Seeing the number of requests you are making to sixpack
• Viewing response times from the sixpack service
• Tracking redis performance
• Potentially tracking redis space usage

Would be useful in diagnosing trouble spots or performance issues.

[zackkitzmiller]

I really like this idea. Putting it on the 2.0 milestone.

[josegonzalez]

Of note, this is the only major service we run that does not have instrumentation of any kind, and thus we don't get any alerts when sixpack is slow and killing our wobsite.

[erwaller]

If we should do this, we shouldn't bother supporting anything other than StatsD. We should also make it an optional dependency: http://pythonhosted.org/distribute/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies, and probably just define STATSD_HOST and STATSD_PORT as env vars to set it you want to use it.

[zackkitzmiller]

I think it should at least be extensible. If someone wants to add a drive from a different tool, that should be fine. We can provide a standard interface for timing or statistics or whatever and they can hook in whatever tool they want, and possibly even contribute that back.

The amount of overhead it would take to develop a common interface would be minimal.

[erwaller]

I think it's just so overkill for a project of this size, and it's easy enough to just fork if you really need something else

[erwaller]

Whoops, accidental close

[zackkitzmiller]

It's possible that it's overkill, but I'm sure others are using instrumentation other than StatsD. I feel that it is in the spirit of being a good open source maintainer to provide an interface for others to write their own driver.

[josegonzalez]

Eh, the statsd format is pretty well supported. For the purposes of our internal tooling, that would be enough.

[zackkitzmiller]

I agree. For us, we would just use the default driver. I'm actually surprised by the push back of shipping a reasonable feature.

I would be beyond happy to work on this 'off sprint' if that's the issue.

[josegonzalez]

I mean if its just going to be an adapter or something, sure. I just thought you were going to write all the adapters, which would be silly :)

[josegonzalez]

This was completed in #225.