statistics collector written primarily for gunicorn, but usable on anything that can do a pre_request and a post_request action and report time taken
Latest commit ae600d3 Sep 6, 2011 Sabin Iacob fix stupid bug, remained uncaught until now due to a catch-all try-ex…
…cept in gunicorn


Stats gathering thing for gunicorn

... or, really, anything that supports custom pre_request and post_request hooks

This is a simple app able to gather stats from gunicorn (or similar) and return them as a JSON object; you get:

  • gstats-collectd -- stats collecting daemon
  • gstatsctl -- provides a few handy commands to talk to the daemon and can serve as an example for your own code
  • munin plug-ins to graph requests/second and average request, usable out of the box or as a basis for your own code
  • a simple WSGI app that returns stats

Intended audience

... is mostly developers with (more than) one foot in systems administration looking to graph application parameters; if pretty graphs are your thing, you may find this useful


  • Ubuntu PPA (Lucid / 10.04 LTS; if needed, I'll add builds for Maverick and / or Natty as well)

if anyone finds this useful enough to build packages for their distro of choice, contact me and I'll add the links here


  • you should be able to speak Python (for the hooks)
  • pyzmq (PPA for Ubuntu)
  • [optional] python-setproctitle for nice process names (PPA for ubuntu)


The scripts:

gstats-collectd -h
Usage: gstats-collectd [options]

  -h, --help            show this help message and exit
  -s ADDR, --stats-address=ADDR
                        set collector address to ADDR [tcp://]
  -c ADDR, --comm-address=ADDR
                        set communication address to ADDR
  -l LENGTH, --buffer-length=LENGTH
                        compute average load times over the last LENGTH
                        seconds [600]

for the format of ZeroMQ addresses, please refer to zmq_connect(3) (online at


gstatsctl -h
Usage: gstatsctl [OPTIONS] <quit|reset|rtimes|stats>

  -h, --help            show this help message and exit
  -c ADDR, --comm-address=ADDR
                        set communication address to ADDR

for the format of ZeroMQ addresses, please refer to zmq_connect(3) (online at

The simplest collecting code looks like this (examples/

from gstats import start_request, end_request

_collector_addr = 'tcp://'

def pre_request(req, worker):
    start_request(req, collect=True, collector=_collector_addr, prefix='my_app')

def post_request(req, worker):
    end_request(req, collector=_collector_addr, prefix='my_app')

The included munin plug-ins are standard wildcard plug-ins; link plugin_name_ to /etc/munin/plugins/plugin_name_my-label and reload munin-node; if you want to use another address than the default, add an entry like this in /etc/munin/plugin-conf.d/munin-node:

user <my_user>
env.status_addr <zmq addr>

The included WSGI application can be started with

gunicorn gstats.wsgi:app -b localhost:8002

if you need to configure things, set the GSTATS_COMM_ADDR and / or GSTATS_ALLOWED_IPS environment variables before starting the web server; the results should be available at http://localhost:8002/_status

One last thing

Due to the way zeromq works, writes will block if collector is down, which means your application will hang; this may not be a problem if you use only post_request and maybe eventlet workers, because the user has already received the response; however, if a pre_request hangs the request won't even be processed.

Also, this thing doesn't have tests because I have no idea what I could/should test. I will hopefully have sphinx documentation soon, though, so at least I hope to be a good citizen in this respect :)