Home
girl_friday is a Ruby library for performing asynchronous tasks. Often times you don't want to block a web response by performing some task, like sending an email, so you can just use this gem to perform it in the background. It works with any Ruby application, including Rails 3 applications.
Why not use any of the zillions of other async solutions (Resque, dj, etc)? Because girl_friday is easier and more efficient than those solutions: girl_friday runs in your Rails process and uses the actor pattern for safe concurrency. Because it runs in the same process, you don't have to monitor a separate set of processes, deploy a separate codebase, waste hundreds of extra MB in RAM for those processes, etc. See my intro to Actors in Ruby for more detail.
You do need to write thread-safe code. This is not hard to do: the actor pattern means that you get a message and process that message. There is no shared data which requires locks and could lead to deadlock in your application code. Because girl_friday does use Threads under the covers, you need to ensure that your Ruby environment can execute Threads efficiently. JRuby, Rubinius 1.2 and Ruby 1.9.2 should be sufficient for most applications. I do not support Ruby 1.8 because of its poor threading support.
Each queue in girl_friday has two components: a supervisor Actor who listens for errors and manages the queue of work and a pool of worker Actors who know how to process a message. Only the supervisor modifies the internal state of the queue so the queue is inherently threadsafe despite using no locking internally.
See Rails for how to use girl_friday in your Rails application. See FAQ for deployment tips, troubleshooting help, etc.
If your processor block raises an exception, girl_friday will:
- kill the worker
- pass the error to an error handler, which by default will just log the error to $stderr
- spawn a new worker to replace the one that died
If you are using HoptoadNotifier, girl_friday will pass the exception to HoptoadNotifier.notify_or_ignore(ex) instead of $stderr logging.
You can customize the error handler by passing in a class which has a handle(ex) method:
class MyErrorHandler
def handle(ex)
puts ex.message
end
end
QUEUE = GirlFriday::WorkQueue.new('my_queue', :error_handler => MyErrorHandler) do |msg|
UserEmail.send_email(msg)
end
By default, girl_friday just persists jobs to memory but I'm guessing you probably don't want to lose any queued work if you restart your app server instances. To prevent this, girl_friday supports job persistence to a Redis server. Note that if you want your jobs to persist in the event that Redis crashes, you will need to configure it for that (append only file setting, and/or replication). To use Redis you just need to pass in the right options:
QUEUE = GirlFriday::WorkQueue.new('my_queue', :store => GirlFriday::Store::Redis, :store_config => [{ :host => 'hostname', :port => 12345 }]) do |msg|
UserEmail.send_email(msg)
end
You can remove the store_config parameter if you are using the default configuration ('127.0.0.1:6379').
You can also pass in a Redis instance if you have one you're using already.
QUEUE = GirlFriday::WorkQueue.new('my_queue', :store => GirlFriday::Store::Redis, :store_config => [{ :redis => $redis }]) do |msg|
UserEmail.send_email(msg)
end
What about when you want to shut down your application? You can't just kill the process, all the asynchronous work will be left in an unknown state. girl_friday supports a shutdown! method which tells all worker threads to stop picking up new work. They will complete their current jobs and then simply stop. You can continue to push new work onto the queues but that work will simply be pushed to Redis or stored in-memory, it will not be started.
GirlFriday.shutdown!
Note that shutdown! will block until all workers are quiet or timeout after 30 seconds. It will return the number of queues which are still processing, meaning 0 under ideal conditions. NOTE: by default girl_friday installs an at_exit block which calls shutdown!. Under normal circumstances, you need to do nothing to get clean shutdown.
GirlFriday can call a callback when a message is done processing:
QUEUE.push(:email => @user.email, :name => @user.name) do |result|
# result = whatever UserEmail.send_email(msg) returned
end
Note: callbacks are incompatible with Redis storage since callbacks can't be marshaled.
Processing in the background is great, but sometimes you want to process a bunch of stuff and block
while that stuff is being processed. girl_friday allows you to collect a set of elements to process, scatter those elements to a pool of workers and then block, waiting to gather the results. See examples/batch.rb for a real world example.
You can collect runtime metrics for each queue in your process:
GirlFriday.status
which returns a hash of data for each queue with interesting numbers for you to monitor or graph to your heart's delight:
{"test"=>
{:pid=>11543,
:pool_size=>1,
:ready=>1,
:busy=>0,
:backlog=>0,
:total_queued=>1,
:total_processed=>1,
:total_errors=>0,
:uptime=>0,
:created_at=>1303092751},
"image_crawler"=>
{:pid=>11543,
:pool_size=>3,
:ready=>0,
:busy=>3,
:backlog=>125,
:total_queued=>200,
:total_processed=>72,
:total_errors=>0,
:uptime=>0,
:created_at=>1303092751}}
You can view girl_friday metrics and status via a built-in Sinatra server. Just add the server to your Rails 3 app like so in your config.ru:
require ::File.expand_path('../config/environment', __FILE__)
require 'girl_friday/server'
run Rack::URLMap.new \
"/" => YourAppName::Application,
"/girl_friday" => GirlFriday::Server.newand then just browse to /girl_friday to see. The server is still under active development.
Since threads making testing difficult, girl_friday supports an "immediate mode" where jobs will be processed immediately on the same thread. Just add this to the bottom of your spec_helper.rb or test_helper.rb:
GirlFriday::Queue.immediate!