Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Support for not_queues #28

Open
wants to merge 7 commits into from

2 participants

@arjes

Hello,

I added support for a not_queues option to force the worker to skip over specific queues. I required this functionality so the workers would continue working on jobs with a blank queue, or anything else. Specifically I had jobs that HAD to be run on a sub-set of machines.

I looked for specs (for queue for example) to emulate, but was unable to find any.

Pull coming to DJ master shortly.

--Brian

unknown and others added some commits
unknown Adding not_queues option
The not_queues option was added to the command via 0e3371e in DJ Master

not_queues allows instructs the worker to NOT work on specified queues
338aacc
unknown Fixing typo ed81a3c
@sferik
Owner

Please fix the tests and add some new tests for this behavior.

Brian Malinc... added some commits
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jan 9, 2013
  1. Adding not_queues option

    unknown authored
    The not_queues option was added to the command via 0e3371e in DJ Master
    
    not_queues allows instructs the worker to NOT work on specified queues
  2. Fixing typo

    unknown authored
Commits on Oct 6, 2013
  1. Adding check for deadlock

    Brian Malinconico authored
Commits on Oct 7, 2013
  1. Making generic AR catch

    Brian Malinconico authored
Commits on Nov 19, 2013
  1. Shuffleing the records as they are returned to reduce contention for …

    Brian Malinconico authored
    …the first job
Commits on Jan 8, 2014
  1. removing shuffle option

    Brian Malinconico authored
Commits on Mar 1, 2014
  1. Revert "removing shuffle option"

    Brian Malinconico authored
    This reverts commit f25dc7b.
This page is out of date. Refresh to see the latest.
Showing with 11 additions and 2 deletions.
  1. +11 −2 lib/delayed/backend/active_record.rb
View
13 lib/delayed/backend/active_record.rb
@@ -60,9 +60,10 @@ def self.find_available(worker_name, limit = 5, max_run_time = Worker.max_run_ti
scope = scope.scoped(:conditions => ['priority >= ?', Worker.min_priority]) if Worker.min_priority
scope = scope.scoped(:conditions => ['priority <= ?', Worker.max_priority]) if Worker.max_priority
scope = scope.scoped(:conditions => ["queue IN (?)", Worker.queues]) if Worker.queues.any?
+ scope = scope.scoped(:conditions => ["queue NOT IN (?)", Worker.not_queues]) if Worker.not_queues.any?
::ActiveRecord::Base.silence do
- scope.by_priority.all(:limit => limit)
+ scope.by_priority.all(:limit => limit).shuffle
end
end
@@ -72,7 +73,15 @@ def lock_exclusively!(max_run_time, worker)
now = self.class.db_time_now
affected_rows = if locked_by != worker
# We don't own this job so we will update the locked_by name and the locked_at
- self.class.update_all(["locked_at = ?, locked_by = ?", now, worker], ["id = ? and (locked_at is null or locked_at < ?) and (run_at <= ?)", id, (now - max_run_time.to_i), now])
+ begin
+ self.class.update_all(["locked_at = ?, locked_by = ?", now, worker], ["id = ? and (locked_at is null or locked_at < ?) and (run_at <= ?)", id, (now - max_run_time.to_i), now])
+ rescue Exception => e
+ if e.message =~ /Deadlock found when trying to get lock/
+ 0
+ else
+ raise e
+ end
+ end
else
# We already own this job, this may happen if the job queue crashes.
# Simply resume and update the locked_at
Something went wrong with that request. Please try again.