Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

changed the plugin to be time based

  • Loading branch information...
commit 2e2927ab2a8b24ee2921dbc78a4621bd457ab22f 1 parent 0c3bb8f
@staugaard staugaard authored
View
6 README.rdoc
@@ -2,11 +2,11 @@
If you have very frequent and fast resque jobs, the overhead of forking and running your after_fork hook, might get too big. Using this resque plugin, you can have your workers perform more than one job, before terminating.
-You simple specify the number of jobs to perform using the JOBS_PER_FORK environment variable:
+You simply specify the number of minutes you want each fork to run using the MINUTES_PER_FORK environment variable:
- QUEUE=* JOBS_PER_FORK=100 rake resque:work
+ QUEUE=* MINUTES_PER_FORK=5 rake resque:work
-This will have each fork process 100 jobs, before terminating. If less than 100 jobs are enqueued, the fork will terminate when the queue is empty.
+This will have each fork process jobs for 5 minutes, before terminating.
This plugin also defines a new hook, that gets called right before the fork terminates:
View
4 Rakefile
@@ -5,14 +5,14 @@ begin
require 'jeweler'
Jeweler::Tasks.new do |gem|
gem.name = "resque-multi-job-forks"
- gem.version = "0.1.2"
+ gem.version = "0.2.0"
gem.summary = %Q{Have your resque workers process more that one job}
gem.description = %Q{When your resque jobs are frequent and fast,
the overhead of forking and running your after_fork might get too big.}
gem.email = "mick@staugaard.com"
gem.homepage = "http://github.com/staugaard/resque-multi-job-forks"
gem.authors = ["Mick Staugaard"]
- gem.add_dependency "resque", ">= 1.6.0", "< 1.8.0"
+ gem.add_dependency "resque", "< 1.8.0"
end
Jeweler::GemcutterTasks.new
rescue LoadError
View
24 lib/resque-multi-job-forks.rb
@@ -21,26 +21,36 @@ class Worker
attr_reader :jobs_processed
unless method_defined?(:done_working_without_multi_job_forks)
+ def process_with_multi_job_forks(job = nil)
+ @jobs_processed ||= 0
+ @kill_fork_at ||= Time.now.to_i + (ENV['MINUTES_PER_FORK'].to_i * 60)
+ process_without_multi_job_forks(job)
+ end
+ alias_method :process_without_multi_job_forks, :process
+ alias_method :process, :process_with_multi_job_forks
+
def done_working_with_multi_job_forks
done_working_without_multi_job_forks
- self.jobs_per_fork ||= [ ENV['JOBS_PER_FORK'].to_i, 1 ].max
- @jobs_processed ||= 0
-
@jobs_processed += 1
if @jobs_processed == 1
old_after_fork = Resque.after_fork
Resque.after_fork = nil
-
- while @jobs_processed < jobs_per_fork && job = reserve
- process(job)
+
+ while Time.now.to_i < @kill_fork_at
+ if job = reserve
+ process(job)
+ else
+ sleep(1)
+ end
end
Resque.after_fork = old_after_fork
run_hook :before_child_exit, self
- @jobs_processed = 0
+ @jobs_processed = nil
+ @kill_fork_at = nil
end
end
alias_method :done_working_without_multi_job_forks, :done_working
View
7 test/test_resque-multi-job-forks.rb
@@ -3,6 +3,8 @@
class SomeJob
def self.perform(i)
$SEQUENCE << "work_#{i}".to_sym
+ puts 'working...'
+ sleep(25)
end
end
@@ -22,16 +24,17 @@ def setup
def test_sequence_of_events
Resque.redis.flush_all
- ENV['JOBS_PER_FORK'] = '2'
+ ENV['MINUTES_PER_FORK'] = '1'
worker = Resque::Worker.new(:jobs)
Resque::Job.create(:jobs, SomeJob, 1)
Resque::Job.create(:jobs, SomeJob, 2)
Resque::Job.create(:jobs, SomeJob, 3)
+ Resque::Job.create(:jobs, SomeJob, 4)
worker.work(0)
- assert_equal([:after_fork, :work_1, :work_2, :before_child_exit_2, :after_fork, :work_3, :before_child_exit_1], $SEQUENCE)
+ assert_equal([:after_fork, :work_1, :work_2, :work_3, :before_child_exit_3, :after_fork, :work_4, :before_child_exit_1], $SEQUENCE)
end
end
Please sign in to comment.
Something went wrong with that request. Please try again.