Unicorn and Gunicorn are awesome tools for people writing web services in Ruby and Python. One of the more nifty features of both programs is their ability to reload application code on-the-fly, by spawning a new master process (or "arbiter", in Gunicorn's language) in response to an operating system signal (SIGUSR2). Unfortunately, this reloading process is incompatible with process-tracking supervisors such as Upstart, because the old master process dies as part of the reload.
Unicorn Herder is a utility designed to assist in the use of Upstart and similar supervisors with Unicorn. It does this by polling the pidfile written by the Unicorn master process, and automating the sequence of signals that must be sent to the master to do a "hot-reload". If Unicorn quits, so will the Unicorn Herder, meaning that if you supervise the herder (which does not daemonize), you are effectively supervising the Unicorn process.
$ pip install unicornherder
$ unicornherder -- -w 4 myapp:app
With unicorn (using Bundler):
$ bundle exec unicornherder -u unicorn
Unicorn Herder forwards the following signals to the unicorn master process:
INT QUIT TERM TTIN TTOU USR1 USR2
Notably, Unicorn Herder does not forward
SIGWINCH, because it is not
intended to be daemonized.
Unicorn Herder also intercepts
SIGHUP, because this is the signal sent by
Upstart when you call
initctl reload, and uses it to trigger a hot-reload of
its Unicorn instance. This process will take two minutes by default, in order to
give the new workers time to start up.
NB: There will be a period during hot-reload when requests are served by both old and new workers. This might have serious implications if you are running data migrations between deploying versions of your application. Please bear this in mind when deciding if you should use Unicorn Herder's hot-reloading feature.
This diagram shows the interactions between unicornherder and (g)unicorn masters at boot and around hot-reloading with HUP (ignoring workers):
An example upstart config (compatible with Upstart v1.4 and later) for use with Unicorn Herder is given below:
description "Unicorn Herder" start on runlevel  stop on runlevel [!2345] respawn respawn limit 5 20 env PORT=4567 setuid www setgid www chdir /var/apps/myapp exec bundle exec unicornherder -u unicorn -- --port $PORT # Or, for a gunicorn installation with a virtualenv # at /var/venv/myapp... #script # . /var/venv/myapp/bin/activate # exec unicornherder -- -w 4 -b "127.0.0.1:$PORT" myapp:app #end script
You can discuss this tool with our open source mailing list: email@example.com
Unicorn Herder is released under the MIT license, a copy of which can be found
To upload a new version of
unicornherder to PyPI, you can follow these
$ python setup.py sdist $ pip install twine $ twine upload dist/*
The credentials for the GDS PyPI account are available in govuk-secrets.