Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 164 lines (101 sloc) 5.292 kb
d11864c @jondot ii
authored
1 # FrenzyBunnies
2
a8a25be @jondot description wording
authored
3 A lightweight background workers library based on JRuby and the very efficient `hot_bunnies` RabbitMQ driver for very fast and
4 efficient processing.
9eada04 @jondot adding dash
authored
5
6 Unlike other background job processing libraries, a Frenzy Bunnies worker is offering its work to a native JVM-based thread pool, where threads are allocated and cached.
7
a8a25be @jondot description wording
authored
8 This firstly means that the processing model isn't process-per-worker (saving memory) and it also isn't fixed-thread-per-worker based allowing workers to be pooled(saving memory even further).
9eada04 @jondot adding dash
authored
9
3269e68 @jondot Update README.md
authored
10 RabbitMQ is a really awesome queue solution for background jobs as well as more real-time messaging processing. Within its strengths are its [performance](http://www.rabbitmq.com/blog/2012/04/17/rabbitmq-performance-measurements-part-1/), portability - [almost every worthy server-side language and platform](http://www.rabbitmq.com/devtools.html) has a RabbitMQ driver and you're not limited to process on a single platform, and high-availability out of the box (as opposed to Redis, although [Sentinel](http://redis.io/topics/sentinel-spec) is quite a progress - hurray!).
11
12
13 Here are [great background slides](https://speakerdeck.com/u/hungryblank/p/rails-underground-2009-rabbitmq) given by Paolo Negri over Rails Underground 2009 about [RabbitMQ](http://www.rabbitmq.com/).
9eada04 @jondot adding dash
authored
14
15 ## Quick Start
16
3269e68 @jondot Update README.md
authored
17 Add this line to your application's Gemfile:
18
19 gem 'frenzy_bunnies'
20
21 And then execute:
22
23 $ bundle
24
25 Or install it yourself as:
26
27 $ gem install frenzy_bunnies
28
29 Then, you basically just need to define a worker in its own class, and then
9eada04 @jondot adding dash
authored
30 decide if you want to use the Frenzy Bunnies runner
31 `frenzy_bunnies` to run it, or do it programmatically via the
32 `FrenzyBunnies::Context` API.
33
34 ```ruby
35 class FeedWorker
36 include FrenzyBunnies::Worker
37 from_queue 'new.feeds', :prefetch => 20, :threads => 13, :durable => true
38
39 def work(msg)
40 puts msg
41 ack!
42 end
43 end
44 ```
45
46 You indicate that a class is a worker by `include
47 FrenzyBunnies::Worker`. Set up a queue with `from_queue` and implement a
48 `work(msg)` method.
49
50 You should indicate successful processing with
51 `ack!`, otherwise it will be rejected and lost (per RabbitMQ semantics,
52 in future versions, they'll add a feature where rejected messages goes
53 to an error queue).
54
55 ### Running with CLI
56
3269e68 @jondot Update README.md
authored
57 Running a worker with the command-line executable is easy
9eada04 @jondot adding dash
authored
58
59 $ frenzy_bunnies start_workers worker_file.rb
60
61 Where `worker_file.rb` is a file containing all of your worker(s)
62 definition. FrenzyBunnies will require the file and immediately start
63 handing work to your workers.
64
65 ### Running Programatically
66
67 Assuming that workers are already `require`d in your code, their classes
68 should be visible by the moment you write this code:
69
70 ```ruby
71 f = FrenzyBunnies::Context.new
3c5e075 @jondot thread state
authored
72 f.run FeedWorker, FeedDownloader
9eada04 @jondot adding dash
authored
73 ```
74
3c5e075 @jondot thread state
authored
75 In the listing above, `f.run` accepts your worker _classes_, and will run your workers immediately.
9eada04 @jondot adding dash
authored
76
77
78 ## Web Dashboard
79
3269e68 @jondot Update README.md
authored
80 When FrenzyBunnies run, it will automatically create a web dashboard for you, on `localhost:11333` by default.
81
f6bc561 @jondot Fixing some sad paths.
authored
82
83 Currently, the dashboard displays your job statistics (passed vs. failed), JVM
84 health (heap usage) and threads overview.
85
86
87 <img src="https://raw.github.com/jondot/frenzy_bunnies/master/fb-cap.png"/><br/>
88
3269e68 @jondot Update README.md
authored
89
90 Changing the bound address is easy to do through the many options you can pass to the running `Context`:
91
92 ```ruby
93 f = FrenzyBunnies::Context.new :web_host=>'0.0.0.0', :web_port=>11222
94 ```
95
96
9eada04 @jondot adding dash
authored
97 context definitions
98
99 ## In Detail
100
3269e68 @jondot Update README.md
authored
101 ### Worker Configuration
9eada04 @jondot adding dash
authored
102
3c5e075 @jondot thread state
authored
103 In your worker class, say `from_queue 'queue_name'` and pass any of these options:
9eada04 @jondot adding dash
authored
104
3269e68 @jondot Update README.md
authored
105 ```ruby
106 :prefetch # default 10. number of messages to prefetch each time
107 :durable # default false. durability of the queue
108 :timeout_job_after # default 5. reject the message if not processed for number of seconds
109 :threads # default none. number of threads in the threadpool. leave empty to let the threadpool manage it.
110 ```
9eada04 @jondot adding dash
authored
111
3c5e075 @jondot thread state
authored
112 Example:
113
114
115 ```ruby
116 class FeedWorker
117 include FrenzyBunnies::Worker
118 from_queue 'new.feeds', :prefetch => 20, :threads => 13, :durable => true
119
120 ...
121 ```
d11864c @jondot ii
authored
122
3269e68 @jondot Update README.md
authored
123 ### General Configuration
d11864c @jondot ii
authored
124
3269e68 @jondot Update README.md
authored
125 Global / running configuration can be set through the running context `FrenzyBunnies::Context`, pass any of these as options (shown with defaults).
d11864c @jondot ii
authored
126
3269e68 @jondot Update README.md
authored
127 ```ruby
128 :host # default 'localhost'
129 :exchange # default 'frenzy_bunnies'
130 :heartbeat # default 5
131 :web_host # default 'localhost'
132 :web_port # default 11333
133 :web_threadfilter # default /^pool-.*/
134 :env # default ''
135 ```
d11864c @jondot ii
authored
136
137
3269e68 @jondot Update README.md
authored
138 Example:
d11864c @jondot ii
authored
139
3269e68 @jondot Update README.md
authored
140 ```ruby
141 FrenzyBunnies::Context.new :exchange=> 'foo'
142 ```
d11864c @jondot ii
authored
143
3269e68 @jondot Update README.md
authored
144 ### AMQP Queue Wiring Under the Hood
d11864c @jondot ii
authored
145
3269e68 @jondot Update README.md
authored
146 If you're interested with the mechanics, in order to mimic a background-job / work-queue
147 semantics, the following is the AMQP wireup used within this library:
d11864c @jondot ii
authored
148
3269e68 @jondot Update README.md
authored
149 * Durable per configuration
150 * The exchange is created and named by default `frenzy_bunnies`
3c5e075 @jondot thread state
authored
151 * Each worker is bound to an AMQP queue named `my_queue_environment` with the environment postfix appended automatically.
3269e68 @jondot Update README.md
authored
152 * The routing key on the exchange is of the same name and bound to the queue.
d11864c @jondot ii
authored
153
3c5e075 @jondot thread state
authored
154 # Contributing
155
156 Fork, implement, add tests, pull request, get my everlasting thanks and a respectable place here :).
157
158
159 # Copyright
160
161 Copyright (c) 2012 [Dotan Nahum](http://gplus.to/dotan) [@jondot](http://twitter.com/jondot). See MIT-LICENSE for further details.
d11864c @jondot ii
authored
162
3269e68 @jondot Update README.md
authored
163
Something went wrong with that request. Please try again.