Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Provides a way to control how jobs are picked up by redis workers
Ruby
Branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
lib
spec
.gitignore
Gemfile
LICENSE
README.md
Rakefile
resque-brokered.gemspec

README.md

resque-brokered

Introduction

Resque-brokered is an effort at allowing you to inject different strategies for picking resque jobs off queues. It was initial created to meet the requirement of having workers reading from multiple queues, but only having one job from each queue being worked on at any one time.

Installation

If you're using bundler, then something like this will be required in your Gemfile:

gem 'resque-brokered'

If you're not using bundler, then you should take a good hard look at yourself, then if you're still not using bundler, then you'll need to do something like this:

$ gem install resque-brokered

Then somewhere in your code (before you do any queueing stuff):

require 'resque'
require 'resque-brokered'

Using resque-brokered

Brokered queues are defined by having a two-part queue name, the group-name, then the queue name. Thus you could have a queue system with queues big_process:user1, big_process:user2 ... big_process:usern then you can have as many big_process workers as you need to cater to your user base, but only handle one big_process per user at any one time.

To start a worker to consume from all the big_process queues, simply specify the first part of the queue name like so:

QUEUES=big_process: rake resque:work

Note the colon. It's the special sauce. The worker you just started will consume from all the queues which start with big_process:. If you start more workers with the same QUEUES environment variable, they will do the same, but they won't pick jobs up from queues which are already active.

Normally named queues and queues without colons work as per usual. It's suggested that you don't use the catch-all QUEUES=* in conjunction with resque-brokered as that will ignore all the limiting and consistency work which resque-brokered brings.

Going forward

Right now resque-brokered only supports one strategy, so it isn't really a broker in the truest sense. Moving forward, we would like to support additional strategies, such as weighted/priority queues, rate limiting and more advanced concurrency limitation.

We also need to improve our test coverage and documentation

Contributors

James Sadler and Andrew Harvey

Contributions

You know the deal:

  • Fork the project
  • Write code with tests
  • Submit a pull request
  • ???
  • Profit.
Something went wrong with that request. Please try again.