Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 166 lines (103 sloc) 5.456 kb
d11864c @jondot ii
authored
1 # FrenzyBunnies
2
43f38a3 @danielfarrell Move to march_hare
danielfarrell authored
3 A lightweight background workers library based on JRuby and the very efficient `march_hare` RabbitMQ driver for very fast and
a8a25be @jondot description wording
authored
4 efficient processing.
9eada04 @jondot adding dash
authored
5
6 Unlike other background job processing libraries, a Frenzy Bunnies worker is offering its work to a native JVM-based thread pool, where threads are allocated and cached.
7
a8a25be @jondot description wording
authored
8 This firstly means that the processing model isn't process-per-worker (saving memory) and it also isn't fixed-thread-per-worker based allowing workers to be pooled(saving memory even further).
9eada04 @jondot adding dash
authored
9
3269e68 @jondot Update README.md
authored
10 RabbitMQ is a really awesome queue solution for background jobs as well as more real-time messaging processing. Within its strengths are its [performance](http://www.rabbitmq.com/blog/2012/04/17/rabbitmq-performance-measurements-part-1/), portability - [almost every worthy server-side language and platform](http://www.rabbitmq.com/devtools.html) has a RabbitMQ driver and you're not limited to process on a single platform, and high-availability out of the box (as opposed to Redis, although [Sentinel](http://redis.io/topics/sentinel-spec) is quite a progress - hurray!).
11
12
13 Here are [great background slides](https://speakerdeck.com/u/hungryblank/p/rails-underground-2009-rabbitmq) given by Paolo Negri over Rails Underground 2009 about [RabbitMQ](http://www.rabbitmq.com/).
9eada04 @jondot adding dash
authored
14
15 ## Quick Start
16
3269e68 @jondot Update README.md
authored
17 Add this line to your application's Gemfile:
18
19 gem 'frenzy_bunnies'
20
21 And then execute:
22
23 $ bundle
24
25 Or install it yourself as:
26
27 $ gem install frenzy_bunnies
28
29 Then, you basically just need to define a worker in its own class, and then
9eada04 @jondot adding dash
authored
30 decide if you want to use the Frenzy Bunnies runner
31 `frenzy_bunnies` to run it, or do it programmatically via the
32 `FrenzyBunnies::Context` API.
33
34 ```ruby
35 class FeedWorker
36 include FrenzyBunnies::Worker
37 from_queue 'new.feeds', :prefetch => 20, :threads => 13, :durable => true
38
39 def work(msg)
40 puts msg
41 ack!
42 end
43 end
44 ```
45
46 You indicate that a class is a worker by `include
47 FrenzyBunnies::Worker`. Set up a queue with `from_queue` and implement a
48 `work(msg)` method.
49
50 You should indicate successful processing with
51 `ack!`, otherwise it will be rejected and lost (per RabbitMQ semantics,
52 in future versions, they'll add a feature where rejected messages goes
53 to an error queue).
54
55 ### Running with CLI
56
3269e68 @jondot Update README.md
authored
57 Running a worker with the command-line executable is easy
9eada04 @jondot adding dash
authored
58
59 $ frenzy_bunnies start_workers worker_file.rb
60
61 Where `worker_file.rb` is a file containing all of your worker(s)
62 definition. FrenzyBunnies will require the file and immediately start
63 handing work to your workers.
64
65 ### Running Programatically
66
67 Assuming that workers are already `require`d in your code, their classes
68 should be visible by the moment you write this code:
69
70 ```ruby
71 f = FrenzyBunnies::Context.new
3c5e075 @jondot thread state
authored
72 f.run FeedWorker, FeedDownloader
9eada04 @jondot adding dash
authored
73 ```
74
3c5e075 @jondot thread state
authored
75 In the listing above, `f.run` accepts your worker _classes_, and will run your workers immediately.
9eada04 @jondot adding dash
authored
76
77
78 ## Web Dashboard
79
3269e68 @jondot Update README.md
authored
80 When FrenzyBunnies run, it will automatically create a web dashboard for you, on `localhost:11333` by default.
81
f6bc561 @jondot Fixing some sad paths.
authored
82
83 Currently, the dashboard displays your job statistics (passed vs. failed), JVM
84 health (heap usage) and threads overview.
85
86
87 <img src="https://raw.github.com/jondot/frenzy_bunnies/master/fb-cap.png"/><br/>
88
3269e68 @jondot Update README.md
authored
89
90 Changing the bound address is easy to do through the many options you can pass to the running `Context`:
91
92 ```ruby
93 f = FrenzyBunnies::Context.new :web_host=>'0.0.0.0', :web_port=>11222
94 ```
95
96
9eada04 @jondot adding dash
authored
97 context definitions
98
99 ## In Detail
100
3269e68 @jondot Update README.md
authored
101 ### Worker Configuration
9eada04 @jondot adding dash
authored
102
3c5e075 @jondot thread state
authored
103 In your worker class, say `from_queue 'queue_name'` and pass any of these options:
9eada04 @jondot adding dash
authored
104
3269e68 @jondot Update README.md
authored
105 ```ruby
f56b358 @danielfarrell Allow exchange to be set on worker
danielfarrell authored
106 :exchange # default frenzy_bunnies. name of exchange.
107 :exchange_type # default :direct. type of exchange used.
108 :routing_key # default queue_name. allows for other routing keys, useful for topic exchanges.
3269e68 @jondot Update README.md
authored
109 :prefetch # default 10. number of messages to prefetch each time
110 :durable # default false. durability of the queue
111 :timeout_job_after # default 5. reject the message if not processed for number of seconds
112 :threads # default none. number of threads in the threadpool. leave empty to let the threadpool manage it.
113 ```
9eada04 @jondot adding dash
authored
114
3c5e075 @jondot thread state
authored
115 Example:
116
117
118 ```ruby
119 class FeedWorker
120 include FrenzyBunnies::Worker
121 from_queue 'new.feeds', :prefetch => 20, :threads => 13, :durable => true
122
123 ...
124 ```
d11864c @jondot ii
authored
125
3269e68 @jondot Update README.md
authored
126 ### General Configuration
d11864c @jondot ii
authored
127
3269e68 @jondot Update README.md
authored
128 Global / running configuration can be set through the running context `FrenzyBunnies::Context`, pass any of these as options (shown with defaults).
d11864c @jondot ii
authored
129
3269e68 @jondot Update README.md
authored
130 ```ruby
131 :host # default 'localhost'
132 :heartbeat # default 5
133 :web_host # default 'localhost'
134 :web_port # default 11333
135 :web_threadfilter # default /^pool-.*/
136 :env # default ''
137 ```
d11864c @jondot ii
authored
138
139
3269e68 @jondot Update README.md
authored
140 Example:
d11864c @jondot ii
authored
141
3269e68 @jondot Update README.md
authored
142 ```ruby
f56b358 @danielfarrell Allow exchange to be set on worker
danielfarrell authored
143 FrenzyBunnies::Context.new :heartbeat => 10
3269e68 @jondot Update README.md
authored
144 ```
d11864c @jondot ii
authored
145
3269e68 @jondot Update README.md
authored
146 ### AMQP Queue Wiring Under the Hood
d11864c @jondot ii
authored
147
3269e68 @jondot Update README.md
authored
148 If you're interested with the mechanics, in order to mimic a background-job / work-queue
149 semantics, the following is the AMQP wireup used within this library:
d11864c @jondot ii
authored
150
3269e68 @jondot Update README.md
authored
151 * Durable per configuration
152 * The exchange is created and named by default `frenzy_bunnies`
3c5e075 @jondot thread state
authored
153 * Each worker is bound to an AMQP queue named `my_queue_environment` with the environment postfix appended automatically.
3269e68 @jondot Update README.md
authored
154 * The routing key on the exchange is of the same name and bound to the queue.
d11864c @jondot ii
authored
155
3c5e075 @jondot thread state
authored
156 # Contributing
157
158 Fork, implement, add tests, pull request, get my everlasting thanks and a respectable place here :).
159
160
161 # Copyright
162
163 Copyright (c) 2012 [Dotan Nahum](http://gplus.to/dotan) [@jondot](http://twitter.com/jondot). See MIT-LICENSE for further details.
d11864c @jondot ii
authored
164
3269e68 @jondot Update README.md
authored
165
Something went wrong with that request. Please try again.