Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also compare across forks.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also compare across forks.
base fork: resque/resque
...
head fork: theflow/resque
compare: master
Checking mergeability… Don’t worry, you can still create the pull request.
  • 9 commits
  • 3 files changed
  • 4 commit comments
  • 1 contributor
Commits on Nov 05, 2009
@theflow theflow avoid some weird issues with the JSON gem and ActiveSupport 4f5e3b8
@theflow theflow added an optional fallback mode when Redis is down
In certain environments it makes sense for us to not care if Redis
is running. So just perform the job synchronously then.
e3cd445
Commits on Dec 09, 2009
@theflow theflow Merge remote branch 'upstream/master' 371eeff
@theflow theflow playing around with daemonization of workers 7acade9
Commits on Dec 29, 2009
@theflow theflow Merge commit 'upstream/master' 5d8a42e
@theflow theflow Revert "playing around with daemonization of workers"
This reverts commit 7acade9.
6e631bc
@theflow theflow workers now optionally write a pid file
The pid file handling is taken from Unicorn and simplified.
e7dbb40
Commits on Jan 05, 2010
@theflow theflow Merge remote branch 'upstream/master' a7409fc
Commits on Feb 22, 2010
@theflow theflow Merge branch 'master' of git://github.com/defunkt/resque
Conflicts:
	lib/resque.rb
	test/worker_test.rb
a944521
Showing with 110 additions and 0 deletions.
  1. +17 −0 lib/resque.rb
  2. +51 −0 lib/resque/worker.rb
  3. +42 −0 test/worker_test.rb
View
17 lib/resque.rb
@@ -48,6 +48,10 @@ def redis
self.redis
end
+ def synchronous_fallback=(value)
+ @synchronous_fallback = value
+ end
+
def to_s
"Resque Client connected to #{redis.server}"
end
@@ -137,8 +141,21 @@ def watch_queue(queue)
#
# This method is considered part of the `stable` API.
def enqueue(klass, *args)
+ if @synchronous_fallback
+ enqueue_with_fallback(klass, *args)
+ else
+ enqueue_without_fallback(klass, *args)
+ end
+ end
+
+ def enqueue_without_fallback(klass, *args)
Job.create(queue_from_class(klass), klass, *args)
end
+ def enqueue_with_fallback(klass, *args)
+ enqueue_without_fallback(klass, *args)
+ rescue Errno::ECONNREFUSED => e
+ Job.new(:synchronous, { 'class' => klass.to_s, 'args' => args }).perform
+ end
# This method can be used to conveniently remove a job from a queue.
# It assumes the class you're passing it is a real Ruby class (not
View
51 lib/resque/worker.rb
@@ -16,6 +16,10 @@ class Worker
# Whether the worker should log lots of info to STDOUT
attr_accessor :very_verbose
+ # Whether and where we should write a pid file
+ attr_accessor :pid_file
+ attr_accessor :pid
+
# Boolean indicating whether this worker can or can not fork.
# Automatically set if a fork(2) fails.
attr_accessor :cant_fork
@@ -134,6 +138,7 @@ def work(interval = 5, &block)
ensure
unregister_worker
+ unlink_pid_safe(pid) if pid
end
# Processes a single job. If none is given, it will try to produce
@@ -203,6 +208,7 @@ def startup
register_signal_handlers
prune_dead_workers
register_worker
+ create_pid_file
end
# Enables GC Optimizations if you're running REE.
@@ -237,6 +243,51 @@ def register_signal_handlers
log! "Registered signals"
end
+ # sets the path for the PID file of the master process
+ def create_pid_file
+ return if pid_file.nil?
+
+ if x = active_pid?(pid_file)
+ raise ArgumentError, "Already running on PID:#{x} " \
+ "(or pid=#{pid_file} is stale)"
+ end
+
+ fp = begin
+ tmp = "#{File.dirname(pid_file)}/#{rand}.#$$"
+ File.open(tmp, File::RDWR|File::CREAT|File::EXCL, 0644)
+ rescue Errno::EEXIST
+ retry
+ end
+ fp.syswrite("#$$\n")
+ File.rename(fp.path, pid_file)
+ fp.close
+
+ # if everything worked we store the path to be able to remove it later
+ self.pid = pid_file
+ end
+
+ # unlinks a PID file at given +path+ if it contains the current PID
+ # still potentially racy without locking the directory (which is
+ # non-portable and may interact badly with other programs), but the
+ # window for hitting the race condition is small
+ def unlink_pid_safe(path)
+ (File.read(path).to_i == $$ and File.unlink(path)) rescue nil
+ end
+
+ # returns a PID if a given file contains an active (non-stale) PID,
+ # nil otherwise
+ def active_pid?(path)
+ wpid = File.read(path).to_i
+ return nil if wpid <= 0
+ begin
+ Process.kill(0, wpid)
+ return wpid
+ rescue Errno::ESRCH
+ # don't unlink stale pid files, racy without non-portable locking...
+ end
+ rescue Errno::ENOENT
+ end
+
# Schedule this worker for shutdown. Will finish processing the
# current job.
def shutdown
View
42 test/worker_test.rb
@@ -236,6 +236,48 @@
end
end
+ test "writes a pid file" do
+ pid_file = File.dirname(__FILE__) + '/test_worker.pid'
+ assert !File.exists?(pid_file)
+
+ @worker.pid_file = pid_file
+ @worker.startup
+
+ assert File.exists?(pid_file)
+
+ @worker.unlink_pid_safe(pid_file)
+
+ assert !File.exists?(pid_file)
+ end
+
+ test "does not write a pid file when non configured" do
+ @worker.startup
+ assert_nil @worker.pid
+ assert_nil @worker.pid_file
+ end
+
+ test "does not start when pid file contains running process" do
+ pid_file = File.dirname(__FILE__) + '/test_worker.pid'
+ assert !File.exists?(pid_file)
+
+ @worker.pid_file = pid_file
+ @worker.startup
+
+ worker_pid = File.read(pid_file)
+
+ worker2 = Resque::Worker.new(:jobs)
+ worker2.pid_file = pid_file
+
+ assert_raises ArgumentError do
+ worker2.startup
+ end
+
+ # pid file shouldn't change
+ assert_equal worker_pid, File.read(pid_file)
+
+ @worker.unlink_pid_safe(pid_file)
+ end
+
test "Processed jobs count" do
@worker.work(0)
assert_equal 1, Resque.info[:processed]

Showing you all comments on commits in this comparison.

@nirvdrum

I like the idea here. If my redis server goes down for whatever reason, dropping jobs on the floor isn't going to be cool. Hopefully the server can be restored before my web handlers backup though.

Also good for testing locally where running a redis server all the time could be a pain.

@zapnap

I agree. Especially great for local dev mode where you're not running (and don't want to run) a Redis server. +1

@theflow

I mainly intentended this for dev and test environments. This removes the complexity of setting up Redis and have workers running all the time on all the developer and designer machines. I know there are different opinions about if you should do this kind of stuff, but this makes sense for us at the moment (but might change if we put more stuff into Resque).

On production I still prefer failing fast though instead of dying slowly.

@defunkt
Owner

We have a thin layer over Resque which handles this for us, based on our RAILS_ENV and whether we're using the console or not. I don't think it belongs in Resque proper.

Something went wrong with that request. Please try again.