Permalink
Browse files

Explicitly order the resque jobs from fast and important -> slow and …

…unimportant.

This should address the starvation problems from #228 for the foreseable future.
  • Loading branch information...
1 parent c1c6736 commit a241a439df2fb71aebad73bace589214b8e09126 @philc philc committed Apr 27, 2012
Showing with 21 additions and 1 deletion.
  1. +1 −1 Procfile
  2. +20 −0 script/run_resque.rb
View
@@ -2,4 +2,4 @@
# In production, this file is used as a basis for generating Upstart daemon scripts.
web: bin/run_app.bash
cron: bundle exec script/run_clockwork.rb
-resque: bundle exec rake resque:work QUEUE=*
+resque: script/run_resque.rb
View
@@ -0,0 +1,20 @@
+#!/usr/bin/env ruby
+# These queues are listed in priority order, from fast and important to slow and unimportant. This ordering
+# is critical because you don't want unimportant slow jobs to starve the faster jobs. Resque workers will
+# process jobs strictly in this order -- there is no fair round-robin among the queues.
+#
+# If we ever get the point where we want true concurrency (drain more than one slow queue at the same time),
+# then we should run two instances of `rake resque` via Foreman, each with a different set of queues.
+queue_order = %W(
+ db_commit_ingest
+ fetch_commits
+ clone_new_repo
+ delete_old_comments_by_demo_users
+ deliver_review_request_emails
+ deliver_comment_emails
+ batch_comment_emails
+ deliver_commit_emails
+ generate_tagged_diffs
+).join(",")
+
+exec "bash -c 'QUEUE=#{queue_order} bundle exec rake resque:work'"

0 comments on commit a241a43

Please sign in to comment.