When we first receive a message, we put ourselves to sleep to wait for a moment. The reason for this is that, when things are 'calm' on the bus, we receive messages "too fast". A message that arrives to the badge awarder triggers (usually) a check against datanommer to count messages. But if we try to count them before this message arrives at datanommer, we'll get skewed results! Race condition. We go to sleep to allow ample time for datanommer to consume this one before we go and start doing checks on it. When fedbadges was first released, this was absolutely necessary. Since that time, the fedmsg bus has become much more congested. So, to improve our average speed at handling messages, we only do that sleep statement if we're not already backlogged. If we know we have a huge workload ahead of us, then go ahead and start handling messages as fast as we can.
The fedbadges backend was designed to only have one thread. This past week we gave it many threads (so that it could keep up with its increasingly demanding workload), but this has started to generate crashes from race conditions (mostly inside sqlalchemy). This patch will not solve all of those problems, but it will solve some. This will make it so that *each* thread will get its very own database session so that they don't stomp on each other quite so much.
fedbadges implemented this little callLater mechanism a long time ago before fedmsg and moksha provided a much more sophisticated queueing and monitoring framework. This callLater logic actually serves to **hide** how many messages are in the fedbadges backlog. By removing it, we can get a better sense of what the consumer is doing.