Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Merge pull request #142 from carlosantoniodasilva/resque-inline

Support Resque.inline configuration option. Closes #81
  • Loading branch information...
commit 29ab0b0930494d1dbe531b4d60644e1bc45c7290 2 parents d9e7198 + 5c47d87
Ben VandenBos bvandenbos authored
6 HISTORY.md
View
@@ -1,3 +1,7 @@
+##
+
+* Add support for Resque.inline configuration (carlosantoniodasilva)
+
## 2.0.0.h (2012-03-19)
* Adding plugin support with hooks (andreas)
@@ -108,7 +112,7 @@
## 1.0.4 (2010-02-26)
-* Added support for specifying the queue to put the job onto. This allows for
+* Added support for specifying the queue to put the job onto. This allows for
you to have one job that can go onto multiple queues and be able to schedule
jobs without having to load the job classes.
30 README.markdown
View
@@ -21,7 +21,7 @@ The syntax is pretty explanatory:
### Documentation
-This README covers what most people need to know. If you're looking for
+This README covers what most people need to know. If you're looking for
details on individual methods, you might want to try the [rdoc](http://rdoc.info/github/bvandenbos/resque-scheduler/master/frames).
### Installation
@@ -32,7 +32,7 @@ To install:
Adding the resque:scheduler rake task:
- require 'resque_scheduler/tasks'
+ require 'resque_scheduler/tasks'
There are three things `resque-scheduler` needs to know about in order to do
it's jobs: the schedule, where redis lives, and which queues to use. The
@@ -45,35 +45,35 @@ to know.
# Resque tasks
require 'resque/tasks'
- require 'resque_scheduler/tasks'
-
+ require 'resque_scheduler/tasks'
+
namespace :resque do
task :setup do
require 'resque'
require 'resque_scheduler'
- require 'resque/scheduler'
-
+ require 'resque/scheduler'
+
# you probably already have this somewhere
Resque.redis = 'localhost:6379'
# If you want to be able to dynamically change the schedule,
# uncomment this line. A dynamic schedule can be updated via the
# Resque::Scheduler.set_schedule (and remove_schedule) methods.
- # When dynamic is set to true, the scheduler process looks for
+ # When dynamic is set to true, the scheduler process looks for
# schedule changes and applies them on the fly.
# Note: This feature is only available in >=2.0.0.
#Resque::Scheduler.dynamic = true
-
+
# The schedule doesn't need to be stored in a YAML, it just needs to
# be a hash. YAML is usually the easiest.
Resque.schedule = YAML.load_file('your_resque_schedule.yml')
-
+
# If your schedule already has +queue+ set for each job, you don't
# need to require your jobs. This can be an advantage since it's
# less code that resque-scheduler needs to know about. But in a small
# project, it's usually easier to just include you job classes here.
# So, someting like this:
- require 'jobs'
+ require 'jobs'
end
end
@@ -82,7 +82,7 @@ queueing items from the schedule and polling the delayed queue for items
ready to be pushed on to the work queues. For obvious reasons, this process
never exits.
- $ rake resque:scheduler
+ $ rake resque:scheduler
Supported environment variables are `VERBOSE` and `MUTE`. If either is set to
any nonempty value, they will take effect. `VERBOSE` simply dumps more output
@@ -118,7 +118,7 @@ time is in the past, the job moves from the delayed queue to the actual resque
work queue and will be completed as workers as free to process it.
Also supported is `Resque.enqueue_at` which takes a timestamp to queue the
-job, and `Resque.enqueue_at_with_queue` which takes both a timestamp and a
+job, and `Resque.enqueue_at_with_queue` which takes both a timestamp and a
queue name.
The delayed queue is stored in redis and is persisted in the same way the
@@ -163,7 +163,7 @@ is most likely stored in a YAML like so:
# If you want to have a different job name and class name, provide the 'class' option
class: QueueDocuments
queue: high
- args:
+ args:
description: "This job queues all content for indexing in solr"
clear_leaderboards_contributors:
@@ -174,7 +174,7 @@ is most likely stored in a YAML like so:
description: "This job resets the weekly leaderboard for contributions"
The queue value is optional, but if left unspecified resque-scheduler will
-attempt to get the queue from the job class, which means it needs to be
+attempt to get the queue from the job class, which means it needs to be
defined. If you're getting "uninitialized constant" errors, you probably
need to either set the queue in the schedule or require your jobs in your
"resque:setup" rake task.
@@ -253,7 +253,7 @@ And then a schedule:
cron: "30 6 * * 1"
queue: scoring
custom_job_class: FakeLeaderboard
- args:
+ args:
rails_env: demo
description: "This job will auto-create leaderboards for our online demo and the status will update as the worker makes progress"
13 lib/resque/scheduler.rb
View
@@ -17,7 +17,7 @@ class << self
# If set, will try to update the schulde in the loop
attr_accessor :dynamic
-
+
# Amount of time in seconds to sleep between polls of the delayed
# queue. Defaults to 5
attr_writer :poll_sleep_amount
@@ -26,7 +26,7 @@ class << self
def scheduled_jobs
@@scheduled_jobs
end
-
+
def poll_sleep_amount
@poll_sleep_amount ||= 5 # seconds
end
@@ -75,7 +75,7 @@ def register_signal_handlers
end
end
- def print_schedule
+ def print_schedule
if rufus_scheduler
log! "Scheduling Info\tLast Run"
scheduler_jobs = rufus_scheduler.all_jobs
@@ -89,7 +89,7 @@ def print_schedule
# rufus scheduler instance
def load_schedule!
procline "Loading Schedule"
-
+
# Need to load the schedule from redis for the first time if dynamic
Resque.reload_schedule! if dynamic
@@ -103,7 +103,7 @@ def load_schedule!
Resque.redis.del(:schedules_changed)
procline "Schedules Loaded"
end
-
+
# modify interval type value to value with options if options available
def optionizate_interval_value(value)
args = value
@@ -155,7 +155,6 @@ def rails_env_matches?(config)
# Handles queueing delayed items
# at_time - Time to start scheduling items (default: now).
def handle_delayed_items(at_time=nil)
- item = nil
if timestamp = Resque.next_delayed_timestamp(at_time)
procline "Processing Delayed Items"
while !timestamp.nil?
@@ -184,7 +183,7 @@ def handle_shutdown
yield
exit if @shutdown
end
-
+
def handle_errors
begin
yield
42 lib/resque_scheduler.rb
View
@@ -59,12 +59,12 @@ def schedule=(schedule_hash)
def schedule
@schedule ||= {}
end
-
+
# reloads the schedule from redis
def reload_schedule!
@schedule = get_schedules
end
-
+
# gets the schedule as it exists in redis
def get_schedules
if redis.exists(:schedules)
@@ -77,12 +77,12 @@ def get_schedules
nil
end
end
-
+
# Create or update a schedule with the provided name and configuration.
#
# Note: values for class and custom_job_class need to be strings,
# not constants.
- #
+ #
# Resque.set_schedule('some_job', {:class => 'SomeJob',
# :every => '15mins',
# :queue => 'high',
@@ -95,12 +95,12 @@ def set_schedule(name, config)
end
config
end
-
+
# retrive the schedule configuration for the given name
def get_schedule(name)
decode(redis.hget(:schedules, name))
end
-
+
# remove a given schedule by name
def remove_schedule(name)
redis.hdel(:schedules, name)
@@ -117,13 +117,19 @@ def enqueue_at(timestamp, klass, *args)
end
# Identical to +enqueue_at+, except you can also specify
- # a queue in which the job will be placed after the
- # timestamp has passed.
+ # a queue in which the job will be placed after the
+ # timestamp has passed. It respects Resque.inline option, by
+ # creating the job right away instead of adding to the queue.
def enqueue_at_with_queue(queue, timestamp, klass, *args)
return false unless Plugin.run_before_schedule_hooks(klass, *args)
-
- delayed_push(timestamp, job_to_hash_with_queue(queue, klass, args))
-
+
+ if Resque.inline?
+ # Just create the job and let resque perform it right away with inline.
+ Resque::Job.create(queue, klass, *args)
+ else
+ delayed_push(timestamp, job_to_hash_with_queue(queue, klass, args))
+ end
+
Plugin.run_after_schedule_hooks(klass, *args)
end
@@ -134,7 +140,7 @@ def enqueue_in(number_of_seconds_from_now, klass, *args)
end
# Identical to +enqueue_in+, except you can also specify
- # a queue in which the job will be placed after the
+ # a queue in which the job will be placed after the
# number of seconds has passed.
def enqueue_in_with_queue(queue, number_of_seconds_from_now, klass, *args)
enqueue_at_with_queue(queue, Time.now + number_of_seconds_from_now, klass, *args)
@@ -210,7 +216,7 @@ def reset_delayed_queue
end
# Given an encoded item, remove it from the delayed_queue
- #
+ #
# This method is potentially very expensive since it needs to scan
# through the delayed queue for every timestamp.
def remove_delayed(klass, *args)
@@ -221,7 +227,7 @@ def remove_delayed(klass, *args)
end
destroyed
end
-
+
# Given a timestamp and job (klass + args) it removes all instances and
# returns the count of jobs removed.
#
@@ -238,16 +244,16 @@ def count_all_scheduled_jobs
total_jobs = 0
Array(redis.zrange(:delayed_queue_schedule, 0, -1)).each do |timestamp|
total_jobs += redis.llen("delayed:#{timestamp}").to_i
- end
+ end
total_jobs
- end
+ end
private
-
+
def job_to_hash(klass, args)
{:class => klass.to_s, :args => args, :queue => queue_from_class(klass)}
end
-
+
def job_to_hash_with_queue(queue, klass, args)
{:class => klass.to_s, :args => args, :queue => queue}
end
10 lib/resque_scheduler/server.rb
View
@@ -1,7 +1,7 @@
# Extend Resque::Server to add tabs
module ResqueScheduler
-
+
module Server
def self.included(base)
@@ -29,7 +29,7 @@ def queue_from_class_name(class_name)
Resque::Scheduler.enqueue_from_config(config)
redirect u("/overview")
end
-
+
get "/delayed" do
# Is there a better way to specify alternate template locations with sinatra?
erb File.read(File.join(File.dirname(__FILE__), 'server/views/delayed.erb'))
@@ -39,13 +39,13 @@ def queue_from_class_name(class_name)
# Is there a better way to specify alternate template locations with sinatra?
erb File.read(File.join(File.dirname(__FILE__), 'server/views/delayed_timestamp.erb'))
end
-
+
post "/delayed/queue_now" do
timestamp = params['timestamp']
Resque::Scheduler.enqueue_delayed_items_for_timestamp(timestamp.to_i) if timestamp.to_i > 0
redirect u("/overview")
end
-
+
post "/delayed/clear" do
Resque.reset_delayed_queue
redirect u('delayed')
@@ -59,5 +59,5 @@ def queue_from_class_name(class_name)
Resque::Server.tabs << 'Delayed'
end
-
+
end
43 test/delayed_queue_test.rb
View
@@ -123,26 +123,26 @@
test "delayed_timestamp_size returns 0 when nothing is queue" do
t = Time.now + 60
- assert_equal(0, Resque.delayed_timestamp_size(t))
+ assert_equal(0, Resque.delayed_timestamp_size(t))
end
-
+
test "delayed_timestamp_size returns 1 when one thing is queued" do
t = Time.now + 60
Resque.enqueue_at(t, SomeIvarJob)
- assert_equal(1, Resque.delayed_timestamp_size(t))
+ assert_equal(1, Resque.delayed_timestamp_size(t))
end
test "delayed_timestamp_peek returns empty array when nothings in it" do
t = Time.now + 60
assert_equal([], Resque.delayed_timestamp_peek(t, 0, 1), "make sure it's an empty array, not nil")
end
-
+
test "delayed_timestamp_peek returns an array containing one job when one thing is queued" do
t = Time.now + 60
Resque.enqueue_at(t, SomeIvarJob)
assert_equal [{'args' => [], 'class' => 'SomeIvarJob', 'queue' => 'ivar'}], Resque.delayed_timestamp_peek(t, 0, 1)
end
-
+
test "delayed_timestamp_peek returns an array of multiple jobs when more than one job is queued" do
t = Time.now + 60
Resque.enqueue_at(t, SomeIvarJob)
@@ -150,7 +150,7 @@
job = {'args' => [], 'class' => 'SomeIvarJob', 'queue' => 'ivar'}
assert_equal([job, job], Resque.delayed_timestamp_peek(t, 0, 2))
end
-
+
test "delayed_timestamp_peek only returns an array of one job if only asked for 1" do
t = Time.now + 60
Resque.enqueue_at(t, SomeIvarJob)
@@ -258,14 +258,14 @@
assert_equal(2, Resque.remove_delayed(SomeIvarJob, "bar"))
assert_equal(2, Resque.count_all_scheduled_jobs)
end
-
+
test "remove_delayed_job_from_timestamp removes instances of jobs at a given timestamp" do
t = Time.now + 120
Resque.enqueue_at(t, SomeIvarJob, "foo")
assert_equal 1, Resque.remove_delayed_job_from_timestamp(t, SomeIvarJob, "foo")
assert_equal 0, Resque.delayed_timestamp_size(t)
end
-
+
test "remove_delayed_job_from_timestamp doesn't remove items from other timestamps" do
t1 = Time.now + 120
t2 = t1 + 1
@@ -275,29 +275,29 @@
assert_equal 1, Resque.delayed_timestamp_size(t1)
assert_equal 0, Resque.delayed_timestamp_size(t2)
end
-
+
test "remove_delayed_job_from_timestamp removes nothing if there are no matches" do
t = Time.now + 120
assert_equal 0, Resque.remove_delayed_job_from_timestamp(t, SomeIvarJob, "foo")
end
-
+
test "remove_delayed_job_from_timestamp only removes items that match args" do
t = Time.now + 120
Resque.enqueue_at(t, SomeIvarJob, "foo")
Resque.enqueue_at(t, SomeIvarJob, "bar")
assert_equal 1, Resque.remove_delayed_job_from_timestamp(t, SomeIvarJob, "foo")
- assert_equal 1, Resque.delayed_timestamp_size(t)
+ assert_equal 1, Resque.delayed_timestamp_size(t)
end
-
+
test "remove_delayed_job_from_timestamp returns the number of items removed" do
t = Time.now + 120
Resque.enqueue_at(t, SomeIvarJob, "foo")
assert_equal 1, Resque.remove_delayed_job_from_timestamp(t, SomeIvarJob, "foo")
end
-
+
test "remove_delayed_job_from_timestamp should cleanup the delayed timestamp list if not jobs are left" do
t = Time.now + 120
- Resque.enqueue_at(t, SomeIvarJob, "foo")
+ Resque.enqueue_at(t, SomeIvarJob, "foo")
assert_equal 1, Resque.remove_delayed_job_from_timestamp(t, SomeIvarJob, "foo")
assert !Resque.redis.exists("delayed:#{t.to_i}")
assert Resque.delayed_queue_peek(0, 100).empty?
@@ -311,4 +311,19 @@
Resque.enqueue_in(10, String) # string serves as invalid Job class
end
end
+
+ test "inlining jobs with Resque.inline config" do
+ begin
+ Resque.inline = true
+ Resque::Job.expects(:create).once.with(:ivar, SomeIvarJob, "foo", "bar")
+
+ timestamp = Time.now + 120
+ Resque.enqueue_at(timestamp, SomeIvarJob, "foo", "bar")
+
+ assert_equal 0, Resque.count_all_scheduled_jobs
+ assert !Resque.redis.exists("delayed:#{timestamp.to_i}")
+ ensure
+ Resque.inline = false
+ end
+ end
end
Please sign in to comment.
Something went wrong with that request. Please try again.