Use class variable for instantiating statsd object to use timer decorator

[tasdikrahman]

I am trying to use the statsd timer decorator

My file structure is something like this

├── config.py
├── consumers
│   ├── foo_consumer.py

Inside foo_consumer.py

from statsd import StatsClient
import socket

# I am specifying the statsd client IP in the file "config.py"
statsd = ""
statsd_timer_name = 'foo_consumer_IOPerf_{0}'.format(socket.gethostname())

class FooConsumer(Consumer):
    def __init__(self, config):
        # Consumer class passed on the config.py file to this class
        global statsd
        statsd = StatsClient(config['statsd_client'], 8125)

    @statsd.timer(statsd_timer_name)
    def handle(self):
        """my heavyweight function"""

This class is inherited by another class and then instantiated by it in the parent folder.

Error that I get

AttributeError: 'str' object has no attribute 'timer'

What else I have tried

It is necessary that the statsd_client IP be specified over at the config.py file

I thought about passing the statsd_client parameter to the decorator like this

class FooConsumer(Consumer):
    def __init__(self, config):
        self.statsd = StatsClient(config['statsd_client'], 8125)

    @self.statsd.timer(statsd_timer_name)
    def handle(self):
        """my heavyweight function"""

Now wouldn't work as the class hasn't been instantiated and self is not known at this point.

Any alternatives?

[jsocol]

There are a couple of options. Let's look at why the first couple don't work.

statsd = ''
class Foo(object):
    def __init__(self):
        statsd = StatsClient()
    @statsd.timer('foo.bar')
    def bar(self):
        pass

The expression in the decorator (statsd.timer('foo.bar')) is called at class definition time, not class instantiation time, so statsd is still a string. Similarly, self.statsd.timer is evaluated at definition time, so there is no self to reference yet.

One other important thing: Using the hostname in the timer name is going to cause some weird behavior. socket.gethostname() returns the long hostname by default, which is probably going to have at least one . in it (e.g. host1.domain), which graphite is going to break into a nested metric. It'll also be a little bit challenging to try to compare metrics across hosts because they'll all have different names (e.g. timer_host1.domain and timer_host2.domain, vs host1.timer and host2.timer, which could be globbed together with *.timer).

In almost any case, probably the best option is not to create the StatsClient instance in the class __init__, but do it in the module. It doesn't look like anything in this example requires data from class instantiation time. E.g.

from .. import config
statsd = StatsClient(config['statsd_host'])

class Foo(object):
    @statsd.timer('my_timer_name')
    def heavy_method(self):
        pass

Even better would be to configure statsd via environment variables, including a per-host prefix, and use the default instance:

from statsd.defaults.env import statsd

class Foo(object):
    @statsd.timer('foo_consumer_IOPerf')
    def heavy_method(self):
        pass

and then set the environment variables before running the program:

STATSD_HOST="my.statsd.host" STATSD_PREFIX="$(hostname)" my_program.py

If the config that gets passed to the constructor may actually change the statsd host, then the client really needs to be an instance variable (self.statsd). You can't decorate the methods with that, but you could use the context manager form:

class Foo(object):
    def __init__(self, config):
        self.config = config
        self.statsd = StatsClient(config['statsd_host'], prefix=socket.gethostname())

    def heavy_method(self):
        with self.statsd.timer(self.config['timer_metric']):
            # heavy stuff goes here

or

def heavy_method(self, *args, **kwargs):
    with self.statsd.timer(self.config['timer_metric']):
        self._heavy_wrapped(*args, **kwargs)

def _heavy_wrapped(self, arg1, kwarg1=None):
    # heavy stuff goes here

Hope that helps!