Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

updated tests. readme

  • Loading branch information...
commit 34ebff15f38dece1e4b296b2626874fe550dcfc6 1 parent 41e9c42
@plukevdh authored
View
14 README.rdoc
@@ -44,14 +44,14 @@ an inbox or a file server.
=== Enter Kthxbye
Kthxbye is an attempt to solve this problem. It is based heavily off of
-"Resque":http://github.com/defunkt/resque and why not an addition to Resque?
+{Resque}[http://github.com/defunkt/resque] and why not an addition to Resque?
I needed some hack time with Redis on my own as I've never used it before...
"I can learn any language or tool in a matter of days if you give me
1. a good manual
2. an even better project to work on."
--Prof. Shumacher
+\_-Prof. Shumacher_
This project accomplishes both those goals. This is an attempt to learn
something, using Resque and Redis docs as a manual, while at the same time
@@ -112,22 +112,22 @@ These can be ignored if you really don't want to have an asynchronus widget to
display the job's status to the user in real-time. You can simply use the gem
and get the results on your own time.
-If are using the widget however, we depend on "node.js":http://nodejs.org/ to
-handle the Redis PUBSUB listening and updating and "Socket.io":http://socket.io/
+If are using the widget however, we depend on {node.js}[http://nodejs.org/] to
+handle the Redis PUBSUB listening and updating and {Socket.io}[http://socket.io/]
to stream the results back to the clients.
In order to install Socket.io on your application side, run
git clone http://github.com/LearnBoost/Socket.IO.git socket-io --recursive
-*Note:* due to the way Rails tries to serve these files and the seemingly
+\*Note:* due to the way Rails tries to serve these files and the seemingly
conflicting way that Socket.io has hardcoded some paths into the code, you may
need to create a symlink to the public/javascripts/socket.io directory in you
public directory. This was the easiest solution I found.
-There is a separate node.js client at "github.com/plukevdh/kthxbye-node":http://github.com/plukevdh/kthxbye-node
+There is a separate node.js client at http://github.com/plukevdh/kthxbye-node
Simply clone the repo or download the poll.js file and set it up to your hearts
-content. *NOTE:* You will need to configure your ports/redis server in the
+content. \*NOTE:* You will need to configure your ports/redis server in the
poll.js file as well as installing socket.io in the same dir as the poll.js file
via "git clone git://github.com/LearnBoost/Socket.IO-node.git socket.io-node --recursive"
(from the socket.io install instructions http://github.com/LearnBoost/Socket.IO-node)
View
20 lib/kthxbye/job.rb
@@ -57,7 +57,7 @@ def self.destroy(id, queue)
# remove the job's data as well
redis.hdel("data-store:#{queue}", id)
redis.hdel("result-store:#{queue}", id)
- redis.hdel( :faulure, id )
+ redis.hdel( :failure, id )
return ret
end
@@ -69,7 +69,7 @@ def initialize(id, queue, data)
@id = id.to_i
@queue = queue
@data = data
- @failed_attempts = Failure.fails_for_job(@id) # local tracking only, for rerun purposes
+ @failed_attempts = Failure.fails_for_job(@id) # local tracking only, for rerun purposes
end
# Simply requeues the job to be rerun.
@@ -124,15 +124,19 @@ def perform
redis.hset( "result-store:#{@queue}", @id, encode( result ) )
return result
rescue Object => ex
- @failed_attempts += 1
- log "Error occured: #{ex.message}. Try: #{@failed_attempts}/#{Kthxbye::Config.options[:attempts]}"
- redis.publish("job.failed", @id)
- Stats.incr("failures")
- return Kthxbye::Failure.create( self, ex ) if @failed_attempts >= Kthxbye::Config.options[:attempts]
- perform
+ # handled by worker
+ raise ex
end
end
+ def fail(ex)
+ @failed_attempts += 1
+ log "Error occured: #{ex.message}. Try: #{@failed_attempts}/#{Kthxbye::Config.options[:attempts]}"
+ redis.publish("job.failed", @id)
+ Stats.incr("failures")
+ Kthxbye::Failure.create( self, ex )
+ end
+
# Removes the job from the inactive queue.
def active
redis.srem("jobs:inactive", @id)
View
22 lib/kthxbye/worker.rb
@@ -69,8 +69,7 @@ def run(&block)
@child = fork {
log "Forking..."
- result = job.perform
- yield job if block_given?
+ perform(job)
exit!
}
@@ -86,6 +85,19 @@ def run(&block)
unregister_worker
end
+ def perform(job)
+ begin
+ result = job.perform
+ rescue Object => ex
+ job.fail(ex)
+ log "Error running job #{job}"
+ Stats.incr("failures:#{self}")
+ perform(job) if job.failed_attempts < Kthxbye::Config.options[:attempts]
+ ensure
+ yield job if block_given?
+ end
+ end
+
# Returns the queues this worker is attached toin alphabetical order.
def queues
@queues.sort
@@ -133,8 +145,8 @@ def unregister_worker
redis.srem :workers, self
redis.del "worker:#{self}:started"
- Stats.clear("processed:#{self}")
- Stats.clear("failed:#{self}")
+ Stats.reset("processed:#{self}")
+ Stats.reset("failures:#{self}")
end
# Gets the current job this worker is working.
@@ -178,7 +190,7 @@ def processed
end
def failed
- Stats["failure:#{self}"]
+ Stats["failures:#{self}"]
end
def started
View
10 test/test_kthxbye.rb
@@ -123,12 +123,12 @@ class TestKthxbye < Test::Unit::TestCase
should "retry a job if attempts setting is > 1" do
Kthxbye::Config.options[:attempts] = 2
+ worker = Kthxbye::Worker.new("test", 0)
id = Kthxbye.enqueue( "test", BadJob )
- job = Kthxbye::Job.find(id, "test")
- job.perform
- assert_equal 2, job.instance_variable_get(:@failed_attempts)
-
+ worker.run
+ job = Kthxbye::Job.find(id, "test")
+ assert_equal 2, job.failed_attempts
end
should "delete a job" do
@@ -191,8 +191,8 @@ class TestKthxbye < Test::Unit::TestCase
worker1.run
assert_equal :succeeded, job.status
- job = Kthxbye::Job.find(bad_job, "bad")
worker2.run
+ job = Kthxbye::Job.find(bad_job, "bad")
assert_equal :failed, job.status
end
View
25 test/test_worker.rb
@@ -2,7 +2,7 @@
class TestWorker < Test::Unit::TestCase
context "See Kthxbye::Worker" do
setup do
- Kthxbye::Config.setup(:verbose => false, :redis_port => 9876)
+ Kthxbye::Config.setup(:verbose => false, :redis_port => 9876, :attempts => 1)
Kthxbye.redis.flushall
@id = Kthxbye.enqueue( "test", SimpleJob, 1, 2 )
@@ -130,6 +130,29 @@ class TestWorker < Test::Unit::TestCase
end
end
+ should "report system stats" do
+ Kthxbye.enqueue( "test", BadJob )
+ assert_equal 0, @worker.processed
+ assert_equal 0, @worker.failed
+
+ @worker.run
+
+ assert_equal 2, Kthxbye::Stats["processed"]
+ assert_equal 1, Kthxbye::Stats["failures"]
+ end
+
+ should "track failures" do
+ Kthxbye.enqueue( "test", BadJob )
+ Kthxbye.enqueue( "test", BadJob )
+
+ 3.times do
+ job = @worker.grab_job
+ @worker.perform(job)
+ end
+
+ assert_equal 2, @worker.failed
+ end
+
should "requeue a job and report error if worker is killed mid-process" do
@worker.working( Kthxbye::Job.find( @id, "test" ) )
@worker.unregister_worker
Please sign in to comment.
Something went wrong with that request. Please try again.