Skip to content

Commit

Permalink
Merge remote branch 'collectiveidea/master' into delayed_job_daemon
Browse files Browse the repository at this point in the history
  • Loading branch information
guns committed Sep 9, 2010
2 parents 6f63660 + 83fe8d8 commit 8bc1abd
Show file tree
Hide file tree
Showing 13 changed files with 212 additions and 135 deletions.
1 change: 1 addition & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ group :development do
gem 'rake'
gem 'rails', '~>3'
gem 'sqlite3-ruby'
gem 'ruby-debug'
end
8 changes: 8 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,12 @@ GEM
arel (1.0.1)
activesupport (~> 3.0.0)
builder (2.1.2)
columnize (0.3.1)
daemons (1.1.0)
erubis (2.6.6)
abstract (>= 1.0.0)
i18n (0.4.1)
linecache (0.43)
mail (2.2.5)
activesupport (>= 2.3.6)
mime-types
Expand All @@ -61,6 +63,11 @@ GEM
thor (~> 0.14.0)
rake (0.8.7)
rspec (1.3.0)
ruby-debug (0.10.3)
columnize (>= 0.1)
ruby-debug-base (~> 0.10.3.0)
ruby-debug-base (0.10.3)
linecache (>= 0.3)
sqlite3-ruby (1.3.1)
thor (0.14.0)
treetop (1.4.8)
Expand All @@ -76,4 +83,5 @@ DEPENDENCIES
rails (~> 3)
rake
rspec
ruby-debug
sqlite3-ruby
24 changes: 14 additions & 10 deletions README.textile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
h1. Delayed::Job

Delated_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background.
Delated_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background.

It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks. Amongst those tasks are:

Expand All @@ -9,8 +9,8 @@ It is a direct extraction from Shopify where the job table is responsible for a
* http downloads
* updating smart collections
* updating solr, our search server, after product changes
* batch imports
* spam checks
* batch imports
* spam checks

h2. Installation

Expand Down Expand Up @@ -92,19 +92,19 @@ $ RAILS_ENV=production script/delayed_job stop

Workers can be running on any computer, as long as they have access to the database and their clock is in sync. Keep in mind that each worker will check the database at least every 5 seconds.

You can also invoke @rake jobs:work@ which will start working off jobs. You can cancel the rake task with @CTRL-C@.
You can also invoke @rake jobs:work@ which will start working off jobs. You can cancel the rake task with @CTRL-C@.

h2. Custom Jobs

Jobs are simple ruby objects with a method called perform. Any object which responds to perform can be stuffed into the jobs table. Job objects are serialized to yaml so that they can later be resurrected by the job runner.
Jobs are simple ruby objects with a method called perform. Any object which responds to perform can be stuffed into the jobs table. Job objects are serialized to yaml so that they can later be resurrected by the job runner.

<pre>
class NewsletterJob < Struct.new(:text, :emails)
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
end
end
end

Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email))
</pre>

Expand All @@ -114,6 +114,10 @@ You can define hooks on your job that will be called at different stages in the

<pre>
class ParanoidNewsletterJob < NewsletterJob
def enqueue(job)
record_stat 'newsletter_job/enqueue'
end

def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
Expand All @@ -137,12 +141,12 @@ class ParanoidNewsletterJob < NewsletterJob
def failure
page_sysadmin_in_the_middle_of_the_night
end
end
end
</pre>

h2. Gory Details

The library evolves around a delayed_jobs table which looks as follows:
The library evolves around a delayed_jobs table which looks as follows:

<pre>
create_table :delayed_jobs, :force => true do |table|
Expand Down
25 changes: 18 additions & 7 deletions lib/delayed/backend/base.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,25 @@ def self.included(base)
module ClassMethods
# Add a job to the queue
def enqueue(*args)
object = args.shift
unless object.respond_to?(:perform)
options = {
:priority => Delayed::Worker.default_priority
}

if args.size == 1 && args.first.is_a?(Hash)
options.merge!(args.first)
else
options[:payload_object] = args.shift
options[:priority] = args.first || options[:priority]
options[:run_at] = args[1]
end

unless options[:payload_object].respond_to?(:perform)
raise ArgumentError, 'Cannot enqueue items which do not respond to perform'
end

priority = args.first || Delayed::Worker.default_priority
run_at = args[1]
self.create(:payload_object => object, :priority => priority.to_i, :run_at => run_at)
self.create(options).tap do |job|
job.hook(:enqueue)
end
end

# Hook method that is called before a new worker is forked
Expand Down Expand Up @@ -50,6 +61,7 @@ def name
end

def payload_object=(object)
@payload_object = object
self.handler = object.to_yaml
end

Expand Down Expand Up @@ -80,8 +92,7 @@ def unlock
def hook(name, *args)
if payload_object.respond_to?(name)
method = payload_object.method(name)
args.unshift(self)
method.call(*args.slice(0, method.arity))
method.arity == 0 ? method.call : method.call(self, *args)
end
end

Expand Down
Loading

0 comments on commit 8bc1abd

Please sign in to comment.