Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 84 lines (50 sloc) 4.091 kb
5aaef6b @jc00ke Adding immediate processing
jc00ke authored
1 girl\_friday
95d09a0 @mperham Initial commit
authored
2 ====================
3
5aaef6b @jc00ke Adding immediate processing
jc00ke authored
4 Have a task you want to get done sometime soon but don't want to do it yourself? Give it to girl\_friday! From wikipedia:
2db509a @mperham Add completion callback support, doc update
authored
5
6 > The term Man Friday has become an idiom, still in mainstream usage, to describe an especially faithful servant or
7 > one's best servant or right-hand man. The female equivalent is Girl Friday. The title of the movie His Girl Friday
8 > alludes to it and may have popularized it.
95d09a0 @mperham Initial commit
authored
9
5aaef6b @jc00ke Adding immediate processing
jc00ke authored
10 girl\_friday is a Ruby library for performing asynchronous tasks. Often times you don't want to block a web response by performing some task, like sending an email, so you can just use this gem to perform it in the background. It works with any Ruby application, including Rails 3 applications.
95d09a0 @mperham Initial commit
authored
11
12
13 Installation
14 ------------------
15
5aaef6b @jc00ke Adding immediate processing
jc00ke authored
16 We recommend using [JRuby 1.6+](http://jruby.org) or [Rubinius 2.0+](http://rubini.us) with girl\_friday. Both are excellent options for executing Ruby these days.
95d09a0 @mperham Initial commit
authored
17
18 gem install girl_friday
19
5aaef6b @jc00ke Adding immediate processing
jc00ke authored
20 girl\_friday does not support Ruby 1.8 (MRI) because of its poor threading support. Ruby 1.9 will work reasonably well if you use gems that release the GIL for network I/O (mysql2 is a good example of this, do **not** use the original mysql gem).
95d09a0 @mperham Initial commit
authored
21
22
23 Usage
24 --------------------
25
5aaef6b @jc00ke Adding immediate processing
jc00ke authored
26 Put girl\_friday in your Gemfile:
95d09a0 @mperham Initial commit
authored
27
28 gem 'girl_friday'
29
2db509a @mperham Add completion callback support, doc update
authored
30 In your Rails app, create a `config/initializers/girl_friday.rb` which defines your queues:
95d09a0 @mperham Initial commit
authored
31
d805fdd @mperham Implement clean shutdown, currently synchronous. Should I implement
authored
32 EMAIL_QUEUE = GirlFriday::WorkQueue.new(:user_email, :size => 3) do |msg|
95d09a0 @mperham Initial commit
authored
33 UserMailer.registration_email(msg).deliver
34 end
299b2b2 @jc00ke Adding connection_pool for redis
jc00ke authored
35
d805fdd @mperham Implement clean shutdown, currently synchronous. Should I implement
authored
36 IMAGE_QUEUE = GirlFriday::WorkQueue.new(:image_crawler, :size => 7) do |msg|
2db509a @mperham Add completion callback support, doc update
authored
37 ImageCrawler.process(msg)
38 end
39
299b2b2 @jc00ke Adding connection_pool for redis
jc00ke authored
40 SCRAPE_QUEUE = GirlFriday::WorkQueue.new(:scrape_sites, :size => 4, :store => GirlFriday::Store::Redis, :store_config => [{ :host => 'host' }]) do |msg|
216b5a5 @jc00ke Update README w/Redis store examples
jc00ke authored
41 Page.scrape(msg)
42 end
43
299b2b2 @jc00ke Adding connection_pool for redis
jc00ke authored
44 TRANSCODE_QUEUE = GirlFriday::WorkQueue.new(:scrape_sites, :size => 4, :store => GirlFriday::Store::Redis, :store_config => [{ :redis => $redis }]) do |msg|
216b5a5 @jc00ke Update README w/Redis store examples
jc00ke authored
45 VideoProcessor.transcode(msg)
46 end
47
299b2b2 @jc00ke Adding connection_pool for redis
jc00ke authored
48 require 'connection_pool'
49 redis_pool = ConnectionPool.new(:size => 5, :timeout => 5){ Redis.new }
50 CLEAN_FILTER_QUEUE =GirlFriday::WorkQueue.new(:clean_filter, :store => GirlFriday::Store::Redis, :store_config => [{ :redis => redis_pool}]) do |msg|
51 Filter.clean(msg)
52 end
53
d805fdd @mperham Implement clean shutdown, currently synchronous. Should I implement
authored
54 :size is the number of workers to spin up and defaults to 5. Keep in mind, ActiveRecord defaults to a connection pool size of 5 so if your workers are accessing the database you'll want to ensure that the connection pool is large enough by modifying `config/database.yml`.
95d09a0 @mperham Initial commit
authored
55
56 In your controller action or model, you can call `#push(msg)`
57
2db509a @mperham Add completion callback support, doc update
authored
58 EMAIL_QUEUE.push(:email => @user.email, :name => @user.name)
95d09a0 @mperham Initial commit
authored
59
60 The msg parameter to push is just a Hash whose contents are completely up to you.
61
3d32a27 @mperham Update docs
authored
62 Your message processing block should **not** access any instance data or variables outside of the block. That's shared mutable state and dangerous to touch! I also strongly recommend your queue processor block be **VERY** short, ideally just a method call or two. You can unit test those methods easily but not the processor block itself.
95d09a0 @mperham Initial commit
authored
63
5aaef6b @jc00ke Adding immediate processing
jc00ke authored
64 You can call `GirlFriday::WorkQueue.immediate!` to process jobs immediately, which is helpful when testing. `GirlFriday::WorkQueue.queue!` will revert this & jobs will be processed by actors.
65
95d09a0 @mperham Initial commit
authored
66 More Detail
67 --------------------
68
5aaef6b @jc00ke Adding immediate processing
jc00ke authored
69 Please see the [girl\_friday wiki](https://github.com/mperham/girl_friday/wiki) for more detail and advanced options and tuning. You'll find details on queue persistence with Redis, implementing clean shutdown, querying runtime metrics and SO MUCH MORE!
95d09a0 @mperham Initial commit
authored
70
71
72 Thanks
73 --------------------
74
5aaef6b @jc00ke Adding immediate processing
jc00ke authored
75 [Carbon Five](http://carbonfive.com), I write and maintain girl\_friday on their clock.
95d09a0 @mperham Initial commit
authored
76
77 This gem contains a copy of the Rubinius Actor API, modified to work on any Ruby VM. Thanks to Evan Phoenix, MenTaLguY and the Rubinius project for permission to use and distribute this code.
78
79
80 Author
81 --------------------
82
5aaef6b @jc00ke Adding immediate processing
jc00ke authored
83 Mike Perham, [@mperham](https://twitter.com/mperham), [mikeperham.com](http://mikeperham.com)
Something went wrong with that request. Please try again.