Dispatcher Queue Can Grow Large Under Heavy Loads

[hayesdavis]

The dispatcher throughput is too low to ensure that the queue doesn't grow and that tweets are dispatched in real time under heavy stream rates. On a rackspace cloud VM this is topping out at around 1.2k events per minute after which we see queue growth.

This is likely due to the forking overhead imposed by using resque for the dispatching infrastructure.