Permalink
Browse files

update the docs

  • Loading branch information...
1 parent db414e4 commit 20ae53f8b63afdd8bbe1af9074316968c0f7d5aa @gnufied committed Oct 4, 2008
@@ -6,9 +6,7 @@ BackgrounDRb offers seamless integration with rails. You can invoke random tasks
workers from rails. You can pass arguments, collect results, monitor status of workers and other
stuff.
-%(entry-title)<a name="newer_api"> BackgrounDRb Client API </a>%
-
-p(sub-title). Invoke an asynchronous task on a worker :
+%(entry-title)<a name="async_task"> Invoke a task asynchronously on a worker </a>%
Let's say, you have following worker code:
@@ -45,7 +43,7 @@ one as follows:
Above snippet also demonstrates that, if your worker was started with a @worker_key@ you can use it to
get correct worker proxy.
-p(sub-title). Invoke a method on worker and get results :
+%(entry-title)<a name="sync_task"> Synchronous Task invocation (Invoke task and wait for results) </a>%
Following snippet will invoke method @some_task@ with argument @data@ in @foo_worker@. Also, method will block
until BackgrounDRb server returns a result.
@@ -70,7 +68,7 @@ end
As illustrated above, you can use @worker_key@ or make them in single line too.
-p(sub-title). Fetch Status/Result Objects of a worker :
+%(entry-title)<a name="worker_results"> Retrieve Cached Worker results </a>%
If you are using @cache@ in your worker code to store result objects, you can retrieve them from
rails using:
@@ -79,7 +77,7 @@ rails using:
You can as usual use @worker_key@ if *worker was started with a worker_key*.
-p(sub-title). Enqueue task to the persistent job queue :
+%(entry-title)<a name="persistent_task"> Enqueue task to the persistent job queue : </a>%
Jobs executed via synchronous and asynchronous APIs are fine, but these tasks are usually
kept in memory(and hence they are fast) and hence aren't entirely failsafe.
@@ -98,7 +96,7 @@ Above line will add specified task to the job queue and set to be invoked at spe
about scheduling see scheduling section.
-p(sub-title). Start a Worker :
+%(entry-title)<a name="new_worker"> Start a new worker from controller </a>%
To start a worker from rails:
@@ -112,7 +110,7 @@ Important thing to be kept in mind is, when you are creating a worker using abov
must use a unique @worker_key@ while starting the worker. Also, while invoking any of the other methods
like @ask_result@, @worker_info@ or one of the worker methods, you must user same @worker_key@.
-p(sub-title). Worker Statistics:
+%(entry-title)<a name="worker_info"> Worker Info </a>%
You can get worker specific information using:
@@ -132,7 +130,7 @@ Return value will look like:
[{:worker_key=>"", :status=>:running, :worker=>:log_worker},
{:worker_key=>"", :status=>:running, :worker=>:foo_worker}]}</pre>
-%(entry-title)<a name="newer_api"> BackgrounDRb Clustering </a>%
+%(entry-title)<a name="clustering"> BackgrounDRb Clustering </a>%
By using following option in your @backgroundrb.yml@ you can cluster more than
one backgroundrb server.
@@ -2,4 +2,4 @@
filters_pre: [ "redcloth" ]
# Custom
title: "Rails Integration of BackgrounDRb"
-sidebar_items: [["Introduction", "#introduction"], ["Task Invocation", "#task_invocation"], ["Worker Results", "#worker_results"], ["Worker Info", "#worker_info"], ["Advanced", "#advanced"]]
+sidebar_items: [["Introduction", "#introduction"], ["Async Task", "#async_task"], ["Sync Task", "#sync_task"], ["Worker Results", "#worker_results"], ["Persistent Tasks", "#persistent_task"], ["New Worker", "#new_worker"], ["Worker Info", "#worker_info"], ["Clustering", "#clustering"]]
@@ -1,6 +1,6 @@
<div id="content">
-%(entry-title)<a name="simple_schedule"> Timer Based Scheduling </a>%
+%(entry-title)<a name="timer_scheduling"> Timer Based Scheduling </a>%
Simple tasks in the workers can be scheduled using @add_timer@ or @add_periodic_timer@ methods.
For example:
@@ -20,7 +20,7 @@ end </pre>
Similarly one can use @add_timer@ to fire oneshot task execution.
-%(entry-title)<a name="unix_scheduler"> Unix Scheduler </a>%
+%(entry-title)<a name="unix_scheduling"> Unix Scheduler </a>%
_BackgrounDRb_ supports normal unix styled schedules which can be configured
from @backgroundrb.yml@ file. A sample configuration looks like:
@@ -128,7 +128,7 @@ from the 28th second plus the 59th second:
Note that if you specify an asterisk in the first field (seconds)
it will trigger every second for the subsequent match.
-p(sub-title). Restarting Workers on Scheduled Time
+%(entry-title)<a name="restart_on_schedule"> Restart worker on schedule </a>%
Usually when your worker is scheduled to execute at longer intervals, it
doesn't make sense to have worker around, when its doing nothing. Since, scheduling
@@ -151,4 +151,16 @@ end </pre>
In above worker @reload_on_schedule true@ makes sure that your worker is restarted on
scheduled time. This feature is only available in version 1.0.3 onwards.
+%(entry-title)<a name="schedule_at"> Schedule one shot execution of task at specified time </a>%
+
+<p><b> Only available for tasks persisted to database table </b></p>
+
+If you are using job queue table and want one shot execution of a task scheduled at a particular time. You can use:
+
+<pre class="multiline">MiddleMan(:hello_worker).enq_some_task(:arg => "hello_world",
+ :job_key => "boy",:scheduled_at => Time.now + 30.minutes)</pre>
+
+Which will schedule specified task to be executed after 30 minutes from now.
+
+
</div>
@@ -2,4 +2,4 @@
filters_pre: [ "redcloth" ]
# Custom
title: "Scheduling Tasks using BackgrounDRb"
-sidebar_items: [["Timers", "#timer_scheduling"], ["Unix Scheduler", "#unix_scheduling"], ["Cron Scheduler", "#cron_scheduling"]]
+sidebar_items: [["Timers", "#timer_scheduling"], ["Unix Scheduler", "#unix_scheduling"], ["Cron Scheduler", "#cron_scheduling"], ["Restart on Schedule", "#restart_on_schedule"], ["Schedule At", "#schedule_at"]]
@@ -66,7 +66,7 @@ end</pre>
When @reload_on_schedule@ is true, worker won't be loaded while _BackgrounDRb_ starts and hence you don't need
@set_no_auto_load@ option there.
-p(sub-title). Using Thread Pool
+%(entry-title)<a name="thread_pool"> Using Thread Pool </a>%
Remember _BackgrounDRb_ follows event model of network programming, but sad truth of life is not all networking
libraries follow this model and hence they make use of blocking IO and threads. BackgrounDRb allows you to run
@@ -81,7 +81,7 @@ So whatever code you write within @scrap_wikipedia@ method is going to run concu
*WARNING*: Many of the Ruby libraries out there aren't thread safe and they may not
work as advertised when used from threads(example: Mechanize,Scrubyt)
-p(sub-title). Storing result/status objects
+%(entry-title)<a name="result_caching"> Result Caching </a>%
<b>Update : </b> <em>Using MemCache to store result objects is strongly recommended. Inbuilt cache works, but may give
unpredictable results. Also, using Memcache serves as an out of process cache, which can be queried at any time.
@@ -122,7 +122,7 @@ You need to change @backgroundrb.yml@ file like this, for using memcache for obj
Everything else remains the same.
-%(entry-title)<a href="persistent_queue">Persistent Task Queue </a>%
+%(entry-title)<a name="persistent_job">Persistent Task Queue </a>%
BackgrounDRb now have out of box support for persistent job queues which are persisted to the
database. API to add a task in the job_queue is pretty simple:
@@ -155,7 +155,7 @@ class HelloWorker
end</pre>
-%(entry-title)<a href="testing">Testing Workers </a>%
+%(entry-title)<a name="worker_testing">Testing Workers </a>%
_BackgrounDRb_ comes with a baked in mechanism to write test cases. First make sure that you
have bdrb_test_helper.rb in the test directory of your rails app (run @rake backgroundrb:setup@, if you dont have one).
@@ -2,4 +2,4 @@
filters_pre: [ "redcloth" ]
# Custom
title: "Using BackgrounDRb workers"
-sidebar_items: [["Introduction", "#introduction"], ["Thread Pool", "#thread_pool"], ["Options", "#options"]]
+sidebar_items: [["Introduction", "#introduction"], ["Thread Pool", "#thread_pool"], ["Result Caching", "#result_caching"], ["Persistent Jobs", "#persistent_job"], ["Worker Testing", "#worker_testing"]]
@@ -20,7 +20,7 @@
<li><a href="/workers/"> Workers </a></li>
<li><a href="/rails/"> Rails Integration </a></li>
<li><a href="/advanced/"> Advanced </a></li>
- <li><a href="/manual/"> Manual </a></li>
+ <li><a href="/manual/index.html"> Manual </a></li>
<li><a href="/community/"> Community </a></li>
<li><a href="/bugs/"> Bugs </a></li>
</ul>
@@ -1,16 +1,21 @@
+# Exception class for BackgrounDRb connection errors
module BackgrounDRb
+ # raised when connection to a particular server failed
class BdrbConnError < RuntimeError
attr_accessor :message
def initialize(message)
@message = message
end
end
+ # raised when connection to all of the available servers failed
class NoServerAvailable < RuntimeError
attr_accessor :message
def initialize(message)
@message = message
end
end
+ # raised, when said task was submitted without a job key, whereas
+ # nature of the task requires a job key
class NoJobKey < RuntimeError; end
end
@@ -1,5 +1,8 @@
+# Model for storing jobs/tasks persisted to the database
+
class BdrbJobQueue < ActiveRecord::Base
validates_uniqueness_of :job_key,:scope => [:worker_name,:worker_key]
+ # find next task from the table
def self.find_next(worker_name,worker_key = nil)
returned_job = nil
transaction do
@@ -19,6 +22,8 @@ def self.find_next(worker_name,worker_key = nil)
returned_job
end
+ # release a job and mark it to be unfinished and free.
+ # useful, if inside a worker, processing of this job failed and you want it to process later
def release_job
self.class.transaction do
self.taken = 0
@@ -27,6 +32,7 @@ def release_job
end
end
+ # insert a new job for processing. jobs added will be automatically picked by the appropriate worker
def self.insert_job(options = { })
transaction do
options.merge!(:submitted_at => Time.now.utc,:finished => 0,:taken => 0)
@@ -35,13 +41,15 @@ def self.insert_job(options = { })
end
end
+ # remove a job from table
def self.remove_job(options = { })
transaction do
t_job_id = find(:first, :conditions => options.merge(:finished => 0,:taken => 0),:lock => true)
delete(t_job_id)
end
end
+ # Mark a job as finished
def finish!
self.class.transaction do
self.finished = 1

0 comments on commit 20ae53f

Please sign in to comment.