Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
Merge branch 'delayed_job_daemon' into guns
  • Loading branch information
guns committed Sep 9, 2010
2 parents 76795ea + e196028 commit 1c73fd3
Show file tree
Hide file tree
Showing 14 changed files with 272 additions and 159 deletions.
1 change: 1 addition & 0 deletions Gemfile
Expand Up @@ -7,4 +7,5 @@ group :development do
gem 'rake'
gem 'rails', '~>3'
gem 'sqlite3-ruby'
gem 'ruby-debug'
end
8 changes: 8 additions & 0 deletions Gemfile.lock
Expand Up @@ -31,10 +31,12 @@ GEM
arel (1.0.1)
activesupport (~> 3.0.0)
builder (2.1.2)
columnize (0.3.1)
daemons (1.1.0)
erubis (2.6.6)
abstract (>= 1.0.0)
i18n (0.4.1)
linecache (0.43)
mail (2.2.5)
activesupport (>= 2.3.6)
mime-types
Expand All @@ -61,6 +63,11 @@ GEM
thor (~> 0.14.0)
rake (0.8.7)
rspec (1.3.0)
ruby-debug (0.10.3)
columnize (>= 0.1)
ruby-debug-base (~> 0.10.3.0)
ruby-debug-base (0.10.3)
linecache (>= 0.3)
sqlite3-ruby (1.3.1)
thor (0.14.0)
treetop (1.4.8)
Expand All @@ -76,4 +83,5 @@ DEPENDENCIES
rails (~> 3)
rake
rspec
ruby-debug
sqlite3-ruby
24 changes: 14 additions & 10 deletions README.textile
@@ -1,6 +1,6 @@
h1. Delayed::Job

Delated_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background.
Delated_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background.

It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks. Amongst those tasks are:

Expand All @@ -9,8 +9,8 @@ It is a direct extraction from Shopify where the job table is responsible for a
* http downloads
* updating smart collections
* updating solr, our search server, after product changes
* batch imports
* spam checks
* batch imports
* spam checks

h2. Installation

Expand Down Expand Up @@ -92,19 +92,19 @@ $ RAILS_ENV=production script/delayed_job stop

Workers can be running on any computer, as long as they have access to the database and their clock is in sync. Keep in mind that each worker will check the database at least every 5 seconds.

You can also invoke @rake jobs:work@ which will start working off jobs. You can cancel the rake task with @CTRL-C@.
You can also invoke @rake jobs:work@ which will start working off jobs. You can cancel the rake task with @CTRL-C@.

h2. Custom Jobs

Jobs are simple ruby objects with a method called perform. Any object which responds to perform can be stuffed into the jobs table. Job objects are serialized to yaml so that they can later be resurrected by the job runner.
Jobs are simple ruby objects with a method called perform. Any object which responds to perform can be stuffed into the jobs table. Job objects are serialized to yaml so that they can later be resurrected by the job runner.

<pre>
class NewsletterJob < Struct.new(:text, :emails)
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
end
end
end

Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email))
</pre>

Expand All @@ -114,6 +114,10 @@ You can define hooks on your job that will be called at different stages in the

<pre>
class ParanoidNewsletterJob < NewsletterJob
def enqueue(job)
record_stat 'newsletter_job/enqueue'
end

def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
Expand All @@ -137,12 +141,12 @@ class ParanoidNewsletterJob < NewsletterJob
def failure
page_sysadmin_in_the_middle_of_the_night
end
end
end
</pre>

h2. Gory Details

The library evolves around a delayed_jobs table which looks as follows:
The library evolves around a delayed_jobs table which looks as follows:

<pre>
create_table :delayed_jobs, :force => true do |table|
Expand Down
25 changes: 18 additions & 7 deletions lib/delayed/backend/base.rb
Expand Up @@ -11,14 +11,25 @@ def self.included(base)
module ClassMethods
# Add a job to the queue
def enqueue(*args)
object = args.shift
unless object.respond_to?(:perform)
options = {
:priority => Delayed::Worker.default_priority
}

if args.size == 1 && args.first.is_a?(Hash)
options.merge!(args.first)
else
options[:payload_object] = args.shift
options[:priority] = args.first || options[:priority]
options[:run_at] = args[1]
end

unless options[:payload_object].respond_to?(:perform)
raise ArgumentError, 'Cannot enqueue items which do not respond to perform'
end

priority = args.first || Delayed::Worker.default_priority
run_at = args[1]
self.create(:payload_object => object, :priority => priority.to_i, :run_at => run_at)
self.create(options).tap do |job|
job.hook(:enqueue)
end
end

# Hook method that is called before a new worker is forked
Expand Down Expand Up @@ -50,6 +61,7 @@ def name
end

def payload_object=(object)
@payload_object = object
self.handler = object.to_yaml
end

Expand Down Expand Up @@ -80,8 +92,7 @@ def unlock
def hook(name, *args)
if payload_object.respond_to?(name)
method = payload_object.method(name)
args.unshift(self)
method.call(*args.slice(0, method.arity))
method.arity == 0 ? method.call : method.call(self, *args)
end
end

Expand Down

0 comments on commit 1c73fd3

Please sign in to comment.