Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 188 lines (137 sloc) 7.484 kB
cbc065f @yabawock Updated README.md
authored
1 # QuScheduler
8edae10 @yabawock Add documentation
authored
2
3 ## Description
4
5 qu-scheduler is an extension to [Qu](http://github.com/bkeepers/qu)
6 that adds support for queueing jobs in the future.
7
8 Currently qu-scheduler only works with qu-redis and requires Redis 2.0 or newer.
9
10 Job scheduling is supported in two different ways: Recurring (scheduled) and
11 Delayed.
12
13 Scheduled jobs are like cron jobs, recurring on a regular basis. Delayed
14 jobs are Qu jobs that you want to run at some point in the future.
15 The syntax is pretty explanatory:
16
17 Qu.enqueue_in(5.days, SendFollowupEmail) # run a job in 5 days
18 # or
19 Qu.enqueue_at(5.days.from_now, SomeJob) # run SomeJob at a specific time
20
21 ## Installation
22
23 # Rails 3.x: add it to your Gemfile
24 gem 'qu-scheduler'
25
cbc065f @yabawock Updated README.md
authored
26 There is only a single thing `qu-scheduler` needs to know about in order
27 to do it's jobs: the schedule. The easiest way to configure this is via
28 the rake task. By default, `qu-scheduler` depends on the "qu:setup" rake
29 task. Since you probably already have this task, lets just put our
8edae10 @yabawock Add documentation
authored
30 configuration there. `qu-scheduler` pretty much inherits everything else
31 from Qu.
32
33 # Qu tasks
34 require 'qu/tasks'
35 require 'qu-scheduler/tasks'
36
37 namespace :qu do
38 task :setup do
cbc065f @yabawock Updated README.md
authored
39 require 'qu-redis'
8edae10 @yabawock Add documentation
authored
40 require 'qu-scheduler'
41
42 # If you want to be able to dynamically change the schedule,
43 # uncomment this line. A dynamic schedule can be updated via the
44 # Qu::Scheduler.set_schedule (and remove_schedule) methods.
45 # When dynamic is set to true, the scheduler process looks for
46 # schedule changes and applies them on the fly.
47 # Note: This feature is still under development
48 # Qu::Scheduler.dynamic = true
49
50 # The schedule doesn't need to be stored in a YAML, it just needs to
51 # be a hash. YAML is usually the easiest.
52 Qu.schedule = YAML.load_file(Rails.root.join('config', 'your_resque_schedule.yml'))
53
54 # If your don't depend on your application environment you need to
55 # require your jobs here. Qu determines the queue exclusively from the
56 # class, so we need to have access to them.
57 require 'jobs'
58 end
59 end
60
61 The scheduler process is just a rake task which is responsible for both
62 queueing jobs from the schedule and polling the delayed queue for jobs
63 ready to be pushed on to the work queues. For obvious reasons, this process
64 never exits.
65
66 $ bundle exec rake qu:scheduler
67
68 NOTE: You DO NOT want to run more than one instance of the scheduler. Doing
69 so will result in the same job being queued multiple times. You only need one
70 instance of the scheduler running per application, regardless of number of servers.
71
72 If the scheduler process goes down for whatever reason, the delayed items
73 that should have fired during the outage will fire once the scheduler process
74 is started back up again (even if it is on a new machine). Missed scheduled
75 jobs, however, will not fire upon recovery of the scheduler process.
76
77 ## Delayed jobs
78
79 Delayed jobs are one-off jobs that you want to be put into a queue at some point
80 in the future. The classic example is sending email:
81
82 Qu.enqueue_in(5.days, SendFollowUpEmail, current_user.id)
83
84 This will store the job for 5 days in the Qu delayed queue at which time
85 the scheduler process will pull it from the delayed queue and put it in the
86 appropriate work queue for the given job. It will then be processed as soon as
87 a worker is available (just like any other Qu job).
88
89 NOTE: The job does not fire **exactly** at the time supplied. Rather, once that
90 time is in the past, the job moves from the delayed queue to the actual work
91 queue and will be completed as workers as free to process it.
92
93 Also supported is `Qu.enqueue_at` which takes a timestamp to queue the job.
94
95 The delayed queue is stored in redis and is persisted in the same way the
96 standard Qu jobs are persisted (redis writing to disk). Delayed jobs differ
97 from scheduled jobs in that if your scheduler process is down or workers are
98 down when a particular job is supposed to be processed, they will simply "catch up"
99 once they are started again. Jobs are guaranteed to run (provided they make it
100 into the delayed queue) after their given queue_at time has passed.
101
102 Your jobs can specify one or more `before_schedule` and `after_schedule` hooks,
103 to be run before or after scheduling. If *any* of your `before_schedule` hooks
104 returns `false`, the job will *not* be scheduled and your `after_schedule` hooks
105 will *not* be run.
106
107 One other thing to note is that insertion into the delayed queue is O(log(n))
108 since the jobs are stored in a redis sorted set (zset). I can't imagine this
109 being an issue for someone since redis is stupidly fast even at log(n), but full
110 disclosure is always best.
111
112 ### Removing Delayed jobs
113
114 If you have the need to cancel a delayed job, you can do it like this:
115
116 # after you've enqueued a job like:
117 Qu.enqueue_at(5.days.from_now, SendFollowUpEmail, current_user.id)
118 # remove the job with exactly the same parameters:
119 Qu.remove_delayed(SendFollowUpEmail, current_user.id)
120
121 ## Scheduled Jobs (Recurring Jobs)
122
123 Scheduled (or recurring) jobs are logically no different than a standard cron
124 job. They are jobs that run based on a fixed schedule which is set at
125 startup.
126
127 The schedule is a list of job classes with arguments and a schedule frequency
128 (in crontab syntax). The schedule is just a hash, but is most likely stored in
129 a YAML like this:
130
131 queue_documents_for_indexing:
132 cron: "0 0 * * *"
133 # you can use rufus-scheduler "every" syntax in place of cron if you prefer
134 # every: 1hr
135 klass: QueueDocuments
136 args:
137 description: "This job queues all content for indexing in solr"
138
139 clear_leaderboards_contributors:
140 cron: "30 6 * * 1"
141 klass: ClearLeaderboards
142 args: contributors
143 description: "This job resets the weekly leaderboard for contributions"
144
145 NOTE: Six parameter cron's are also supported (as they supported by
146 rufus-scheduler which powers the resque-scheduler process). This allows you
147 to schedule jobs per second (ie: "30 * * * * *" would fire a job every 30
148 seconds past the minute).
149
150 A big shout out to [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler)
151 for handling the heavy lifting of the actual scheduling engine.
152
153 ## Running in the background
154
155 (Only supported with ruby >= 1.9). There are scenarios where it's helpful for
156 the resque worker to run itself in the background (usually in combination with
157 PIDFILE). Use the BACKGROUND option so that rake will return as soon as the
158 worker is started.
159
160 $ PIDFILE=./qu-scheduler.pid BACKGROUND=yes bundle exec rake qu:scheduler
161
162 ## Note on Patches / Pull Requests
163
164 * Fork the project.
165 * Make your feature addition or bug fix.
166 * Add tests for it. This is important so I don't break it in a future version unintentionally.
167 * Commit, do not mess with rakefile, version, or history.
168 (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
169 * Send me a pull request. Bonus points for topic branches.
170
171 ## Credits
172
6c7e1fc @yabawock Update README.md
authored
173 This work is a port of [resque-scheduler](https://github.com/bvandenbos/resque-scheduler) by Ben VandenBos.
8edae10 @yabawock Add documentation
authored
174 Modified to work with the Qu queueing library by Morton Jonuschat.
175
176 ## Maintainers
177
178 * [Morton Jonuschat](https://github.com/yabawock)
179
180 ## License
181
182 MIT License
183
184 ## Copyright
185
6c7e1fc @yabawock Update README.md
authored
186 Copyright 2011 Morton Jonuschat
187 Some parts copyright 2010 Ben VandenBos
Something went wrong with that request. Please try again.