Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

requeue delay

  • Loading branch information...
commit a3dfcf12a2c240244dfbd827bf71234f550a56a7 1 parent a1bcecb
Bart Olsthoorn authored July 23, 2012
7  Gemfile
... ...
@@ -1,8 +1,3 @@
1 1
 source :rubygems
2 2
 
3  
-gem "resque"
4  
-
5  
-group :development do
6  
-  gem "turn"
7  
-  gem "rake"
8  
-end
  3
+gemspec
17  README.md
Source Rendered
@@ -26,4 +26,19 @@ class Scraper
26 26
     # do the work
27 27
   end
28 28
 end
29  
-```
  29
+```
  30
+
  31
+## Requeue loop
  32
+When a job is requeue'ed there is a small delay (1 second by default) before the worker places the job actually back in the queue. Let's say you have two jobs left, and one job is taking 15 seconds on the first worker and the other similar job is being blocked by the second worker. The second worker will continuously try to put the job back in the queue and it will try to process it again (racing for 15 seconds untill the other job has finished). This only happens when there are no other (not locked) jobs in the queue.
  33
+
  34
+To overwrite this delay in your class:
  35
+``` ruby
  36
+def self.requeue_perform_delay
  37
+	5.0
  38
+end
  39
+```
  40
+
  41
+Please note that setting this value to 5 seconds will keep the worker idle for 5 seconds when the job is locked.
  42
+
  43
+## Possibilities to prevent the loop 
  44
+Do a delayed resque (re)queue. However, this will have approximately the same results and will require a large extra chunk of code and rake configurations.
8  lib/resque/plugins/workers/lock.rb
@@ -7,8 +7,14 @@ def lock(*args)
7 7
           "lock:#{name}-#{args.to_s}"
8 8
         end
9 9
         
  10
+        def requeue_perform_delay
  11
+          1.0
  12
+        end
  13
+        
10 14
         def before_perform_lock(*args)
11  
-          if Resque.redis.setnx(lock(*args), true)
  15
+          nx = Resque.redis.setnx(lock(*args), true)
  16
+          if nx == false
  17
+            sleep(requeue_perform_delay)
12 18
             Resque.enqueue(self, *args)
13 19
             raise Resque::Job::DontPerform
14 20
           end
6  resque-workers-lock.gemspec
@@ -11,7 +11,11 @@ Gem::Specification.new do |s|
11 11
   s.files             = %w( README.md Rakefile LICENSE )
12 12
   s.files            += Dir.glob("lib/**/*")
13 13
   s.files            += Dir.glob("test/**/*")
14  
-
  14
+  
  15
+  s.add_dependency "resque"
  16
+  s.add_development_dependency "turn"
  17
+  s.add_development_dependency "rake"
  18
+  
15 19
   s.description       = <<desc
16 20
 A Resque plugin. If you want to prevent specific jobs to be processed simultaneously, 
17 21
 extend it with this module. It locks on the first argument in the perform method.

0 notes on commit a3dfcf1

Please sign in to comment.
Something went wrong with that request. Please try again.