A Resque plugin to ensure only one job instance is queued or running at a time
Add this line to your application's Gemfile:
And then execute:
Or install it yourself as:
$ gem install resque-uniq
Make your job class extend
Resque::Plugins::UniqueJob, like this
class BigJob extend Resque::Plugins::UniqueJob # <--- add this line @queue = :big_job def self.perform # ... end end
How it works
resque-uniq associates a unique lock with each job instance being enqueued. A lock is a simple Redis key/value pair.
The key name is derived uniquely from the job class name and job args. The value is
Time.now.to_i. If the lock already
exists a new job instance is not enqueued. If not a new lock is created and a job is enqueued. The lock is removed after
perform method has finished.
There is another lock, called run lock, which is being held during the execution of
perform method. Before enqueuing
a new job instance, resque-uniq checks if there is any orphaned run lock. This way it can detect if Resque workers
have crashed during job execution and left behind stale locks.
You can tell resque-uniq to auto-expire job locks by setting
class BigJob extend Resque::Plugins::UniqueJob @queue = :big_job @unique_lock_autoexpire = 6 * 3600 # TTL = 6 hours def self.perform # ... end end
Lock autoexpiration was necessary to protect against stale locks. With the new run lock trick you probably don't need it anymore.
If you want to define a default
unique_lock_autoexpire in the base class and let other jobs to extend that base class
you cannot use
@unique_lock_autoexpire since it's not inherited by subclasses. Define a class method with the same
class BaseJob extend Resque::Plugins::UniqueJob def self.queue :default_queue end def self.unique_lock_autoexpire 600 # TTL = 10 minutes end end class SmallJob < BaseJob def self.perform ... end end class BiglJob < BaseJob def self.unique_lock_autoexpire 3 * 3600 # TTL = 3 hours end def self.perform ... end end
There are several similar Resque plugins. We tried them all but for one reason or another they didn't work reliably for us. Therefore we wrote our own version. Nonetheless we would like to thank the authors of those plugins for inspiration.
- Fork it
- Create your feature branch (
git checkout -b my-new-feature)
- Commit your changes (
git commit -am 'Added some feature')
- Push to the branch (
git push origin my-new-feature)
- Create new Pull Request