Delayed job queueing for Resque.
Enqueue jobs that will only appear for processing after a specified delay or at a particular time in the future.
Useful for jobs that would be awkward to run in crons. For example:
- expiring stale orders to free up reserved inventory
- retrying failed webhook deliveries with progressively increasing delays
Also useful for jobs that are typically run in crons. For example:
- sending call-to-action reminder emails a few days after each signup
- checking whether pending payments have cleared
Fine-grained job scheduling avoids the need for monolithic crons that are often slow, resource intensive and single-process. Instead of needing to stagger crons to avoid overlap or parallelize crons that are too slow, jobs can be spread throughout the entire day and amongst multiple worker processes.
Resque::Delayed is very simple. Call
Resque.enqueue_at instead of
class User after_create :send_call_to_action_email private def send_call_to_action_email Resque.enqueue_in 3.days, CallToActionEmailJob, self.id end end
class RecurringInvoice def generate_invoice # snip... Resque.enqueue_at self.next_billing_date, RecurringInvoiceJob, self.id end end
class Webhook MAX_RETRIES = 15 def deliver unless Webhook.post(payload).success? return if retries == Webhook::MAX_RETRIES update_attribute :retries, retries + 1 Resque.enqueue_in (2**retries).minutes, WebhookDeliveryJob, self.id end end end
$ gem install resque-delayed
to your Gemfile and run
$ bundle install
Resque::Delayed piggybacks on your existing Resque setup so it will use whatever Redis instance Resque has been configured to use.
The above will provide
Resque.enqueue_at to your application but you will also need to run a Resque::Delayed worker process. The worker is responsible for harvesting future-queued jobs and pushing them onto the appropriate Resque queues at the right time.
Resque::Delayed::Worker is a stripped-down version of
Resque::Worker so you can use the same configuration options like
Like Resque, Resque::Delayed provides a rake task to run workers. Add
require 'resque-delayed/tasks' to your
Rakefile and run
$ cd app_root $ LOGGING=1 INTERVAL=10 rake resque_delayed:work
NOTE: Resque::Delayed workers only take future-queued jobs and push them onto Resque queues when they need to be run. They do not actually process jobs so any setup using Resque::Delayed also needs one or more regular Resque workers.
Resque::Delayed workers are very lean as they do not need to load either your application or your Resque job classes. Even so you will probably want to monitor them in production using something like monit, god, or bluepill. Also, because they are not actually performing any of the job processing work it is unlikely you will need to run more than one.1
1 a single Resque::Delayed worker on a laptop with unexciting hardware can push a few thousand jobs per second into Resque while new delayed jobs are simultaneously being added.
- fork this repo
- create a topic branch (
$ git checkout -b my_branch)
- make your changes along with specs
- push to your branch (
$ git push origin my_branch)
- send me a pull request
Thanks to defunkt and all Resque contributors. Resque is a pleasure to use and adapts well to new challenges. Also some code in this project,
Resque::Delayed::Worker in particular, borrows heavily from the Resque implementation.
Copyright (c) Justin Giancola. See LICENSE for details.