Skip to content
Newer
Older
100644 72 lines (51 sloc) 1.62 KB
c27c3d7 @alindeman Initial shot at a README
authored Feb 3, 2012
1 # Rector
2
3ed0ea0 @alindeman Adds build status to README
authored Feb 6, 2012
3 [![Build Status](https://secure.travis-ci.org/alindeman/rector.png)](http://travis-ci.org/alindeman/rector)
4
c27c3d7 @alindeman Initial shot at a README
authored Feb 4, 2012
5 Rector allows coordination of a number of jobs spawned with a mechanism
6 like Resque (though any job manager will do). If you are able to parallelize
7 the processing of a task, yet all these tasks are generating metrics,
8 statistics, or other data that need to be combined, Rector might be for you.
9
f21e84f @alindeman Sets up spec environment
authored Feb 6, 2012
10 ## Requirements
11
12 * Ruby >= 1.9.2 (or 1.9 mode of JRuby or Rubinius)
13
c27c3d7 @alindeman Initial shot at a README
authored Feb 4, 2012
14 ## Configuration
15
16 Rector currently supports Redis as a backend for job coordination and
17 data storage.
18
19 ### Redis Server
20
21 ```ruby
afcb256 @alindeman Rector can be configured
authored Feb 6, 2012
22 Rector.configure do |c|
3542019 @alindeman We will not try to support strings yet
authored Feb 6, 2012
23 c.redis = Redis.new(:host => "10.0.1.1", :port => 6380)
afcb256 @alindeman Rector can be configured
authored Feb 6, 2012
24 end
c27c3d7 @alindeman Initial shot at a README
authored Feb 4, 2012
25 ```
26
27 ## Job Creation (Master)
28
29 Rector requires that some process be designated as the "master" process.
30 This is usually the process that is also responsible for spawning the
31 worker jobs.
32
33 ```ruby
34 job = Rector::Job.new
35
36 # e.g., processing files in parallel
37 files.each do |file|
38 worker = job.workers.create
39
40 # e.g., using Resque for job management; Rector doesn't really care
41 Resque.enqueue(WordCounterJob, worker.id, file)
42 end
43
44 # wait for all the workers to complete
45 job.join
46
47 # get aggregated data from all the jobs
48 job.data.each do |word, count|
49 puts "#{word} was seen #{count} times across all files"
50 end
50ddd7a @alindeman Jobs can be cleaned up
authored Feb 6, 2012
51
52 job.cleanup
c27c3d7 @alindeman Initial shot at a README
authored Feb 4, 2012
53 ```
54
55 ## Job Processing (Workers)
56
57 ```ruby
58 class ProcessFileJob
59 def self.perform(worker_id, file)
60 worker = Rector::Worker.new(worker_id)
61
62 words = File.read(file).split(/\W/)
63 words.reject(&:blank?).each do |word|
bf32311 @alindeman README correction
authored Feb 6, 2012
64 worker.data[word] ||= 0
65 worker.data[word] += 1
c27c3d7 @alindeman Initial shot at a README
authored Feb 4, 2012
66 end
67
68 worker.finish
69 end
70 end
71 ```
Something went wrong with that request. Please try again.