Skip to content
Browse files

Merge branch 'master' of github.com:baoshan/fairy

  • Loading branch information...
2 parents f9b132e + 9b8ffa5 commit f9d3a49b38156bf1b9bf917447deb86e6f46265a Wang Ling committed May 22, 2012
Showing with 72 additions and 73 deletions.
  1. +0 −9 .gitmodules
  2. +22 −0 LICENSE
  3. +1 −0 Makefile
  4. +13 −28 README.md
  5. +0 −1 node_modules/.bin/express
  6. +0 −1 node_modules/node-uuid
  7. +0 −1 node_modules/redis
  8. +10 −6 src/fairy.coffee
  9. +26 −27 src/web/fairy.html
View
9 .gitmodules
@@ -1,9 +0,0 @@
-[submodule "node_modules/node-uuid"]
- path = node_modules/node-uuid
- url = git://github.com/broofa/node-uuid.git
-[submodule "node_modules/redis"]
- path = node_modules/redis
- url = git://github.com/mranney/node_redis.git
-[submodule "node_modules/connect"]
- path = node_modules/connect
- url = git://github.com/senchalabs/connect.git
View
22 LICENSE
@@ -0,0 +1,22 @@
+Copyright (c) 2012 Baoshan Sheng
+
+Permission is hereby granted, free of charge, to any person
+obtaining a copy of this software and associated documentation
+files (the "Software"), to deal in the Software without
+restriction, including without limitation the rights to use,
+copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the
+Software is furnished to do so, subject to the following
+conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.
View
1 Makefile
@@ -27,6 +27,7 @@ publish:
@cp -R lib package/lib
@cp -R web package/web
@cp package.json package
+ @cp LICENSE package
@tar czf $(TARBALL) package
@rm -r package
View
41 README.md
@@ -6,8 +6,8 @@ the sequential processing order of tasks belong to a same group.
[Message Groups]: http://activemq.apache.org/message-groups.html
-But, unkile **message groups**, **Fairy** doesn't always route tasks of a
-group to a same worker, which can lead to unwanted waiting time when:
+But, unlike **message groups**, **Fairy** doesn't always route tasks of a
+group to a same worker, which will introduce unwanted waiting time when:
1. Tasks of group `X` and `Y` are appointed to worker `A`.
2. Worker `A` is processing tasks of group `X` **sequentially**.
@@ -22,13 +22,16 @@ task to any worker when there's no **processing** tasks of the same group.
The design philosophy makes **Fairy** ideal for the following requirements:
- 1. Tasks of a same groups need be processed in sequence.
- 2. Each worker processes tasks in serial.
- 3. Worker spawns child process (e.g., a shell script) to handle the real job.
+ + Tasks of a same groups need be processed in sequence.
+ + Each worker processes tasks in serial.
+ + Multiple workers need be instantiated to increase throughput.
**Fairy** takes a different approach than Message Groups. Instead of making all
tasks of a same group be routed to the same consumer, **Fairy** route a task to
-any worker when there's no **processing** tasks of the same group.
+any worker when there's no **processing** tasks of the **same group**.
+
+When the number of workers is much smaller compared to the number of groups,
+**Fairy**'s approach makes sense.
**[Resque]** cannot guarantee the processing order of the tasks although the task
queue is FIFO. The more workers you have, the more possible you'll encountering
@@ -42,7 +45,7 @@ concurrency which breaks the processing order of tasks in the same group.
## Get Started
-The minimium set of APIs you need to learn in order to bootstrap a task queue
+The minimium set of APIs you need to learn in order to implement a task queue
system are:
+ `enqueue` tasks, and
@@ -79,7 +82,8 @@ dispatch tasks to the worker and block tasks of the same group forever!)
## Web Front-End
-**Fairy** comes with a web front-end. Use it as a express/connect middleware:
+**Fairy** comes with a ready-to-use web front-end. Simply insert the middleware into
+the pipeline:
app = require('express').createServer()
fairy_web = require 'fairy/web'
@@ -119,23 +123,4 @@ for complete API explanations.
Copyright (c) 2012 Baoshan Sheng
-Permission is hereby granted, free of charge, to any person
-obtaining a copy of this software and associated documentation
-files (the "Software"), to deal in the Software without
-restriction, including without limitation the rights to use,
-copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the
-Software is furnished to do so, subject to the following
-conditions:
-
-The above copyright notice and this permission notice shall be
-included in all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
-OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
-HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
-WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
-OTHER DEALINGS IN THE SOFTWARE.
+Released under the MIT license.
View
1 node_modules/.bin/express
1 node_modules/node-uuid
@@ -1 +0,0 @@
-Subproject commit 64d22f7fc853e0a66d9cb962694cea6317a46358
1 node_modules/redis
@@ -1 +0,0 @@
-Subproject commit 874a893c2cdeb5372782cce33e2dbdaeacb6541c
View
16 src/fairy.coffee
@@ -6,8 +6,8 @@
#
# [Message Groups]: http://activemq.apache.org/message-groups.html
#
-# But, unkile **message groups**, **Fairy** doesn't always route tasks of a
-# group to a same worker, which can lead to unwanted waiting time when:
+# But, unlike **message groups**, **Fairy** doesn't always route tasks of a
+# group to a same worker, which will introduce unwanted waiting time when:
#
# 1. Tasks of group `X` and `Y` are appointed to worker `A`.
# 2. Worker `A` is processing tasks of group `X` **sequentially**.
@@ -24,6 +24,7 @@
#
# + Tasks of a same groups need be processed in order.
# + Each worker processes tasks sequentially.
+# + Multiple workers need be instantiated to increase throughput.
#
# Copyright © 2012, Baoshan Sheng.
# Released under the MIT License.
@@ -865,12 +866,17 @@ class Queue
workers: (callback) =>
@redis.hvals @key('WORKERS'), (err, res) ->
return callback err if err
- callback null, res.map (entry) ->
+ callback null, res.map((entry) ->
entry = entry.split '|'
host: entry[0]
ip: entry[1]
pid: parseInt entry[2]
since: new Date parseInt entry[3]
+ ).sort (a, b) ->
+ return 1 if a.ip > b.ip
+ return -1 if a.ip < b.ip
+ return 1 if a.pid > b.pid
+ return -1 if a.pid < b.pid
# ### Clear A Queue
@@ -1005,6 +1011,4 @@ class Queue
callback null, result
-# Known Bugs:
-#
-# 1. Clear lead to negative pending tasks.
+# ### Known Bugs:
View
53 src/web/fairy.html
@@ -40,32 +40,31 @@
<div class="table_title">Workers</div>
<div id="s_workers"></div>
- <!--最近处理任务---失败任务汇总---周期最长-->
<div class="tabbable">
<ul class="nav nav-tabs">
- <li class="active"><a href="#tab1" data-toggle="tab">最近处理任务</a></li>
- <li><a href="#tab2" data-toggle="tab">失败任务汇总</a></li>
- <li><a href="#tab3" data-toggle="tab">当前处理任务</a></li>
- <li><a href="#tab4" data-toggle="tab">周期最长</a></li>
+ <li class="active"><a href="#tab1" data-toggle="tab">Recently Finished Tasks</a></li>
+ <li><a href="#tab2" data-toggle="tab">Failed Tasks</a></li>
+ <li><a href="#tab3" data-toggle="tab">Processing Tasks</a></li>
+ <li><a href="#tab4" data-toggle="tab">Slowest Tasks</a></li>
</ul>
<div class="tab-content">
- <div class="tab-pane-title">最近处理任务</div>
+ <div class="tab-pane-title">Recently Finished Tasks</div>
<div class="tab-pane active" id="tab1">
<div id="s_recently_finished_tasks"></div>
</div>
- <div class="tab-pane-title">失败任务汇总</div>
+ <div class="tab-pane-title">Failed Tasks</div>
<div class="tab-pane" id="tab2">
<div id="s_failed_tasks"></div>
</div>
- <div class="tab-pane-title">当前处理任务</div>
+ <div class="tab-pane-title">Processing Tasks</div>
<div class="tab-pane" id="tab3">
<div id="s_processing_tasks"></div>
</div>
- <div class="tab-pane-title">周期最长</div>
+ <div class="tab-pane-title">Slowest Tasks</div>
<div class="tab-pane" id="tab4">
<div id="s_slowest_tasks"></div>
</div>
@@ -89,10 +88,10 @@
<table class="table table-bordered">
<thead>
<tr>
- <th>编号</th>
- <th colspan = <%= recently_finished_tasks.length?recently_finished_tasks[0].params.length:0 %>>参数</th>
- <th>进入堆栈时间</th>
- <th>完成时间</th>
+ <th>ID</th>
+ <th colspan = <%= recently_finished_tasks.length?recently_finished_tasks[0].params.length:0 %>>Params</th>
+ <th>Queued</th>
+ <th>Finished</th>
</tr>
</thead>
<tbody>
@@ -115,11 +114,11 @@
<table class="table table-bordered">
<thead>
<tr>
- <th>编号</th>
- <th colspan = <%= failed_tasks.length?failed_tasks[0].params.length:0 %>>参数</th>
- <th>进入堆栈时间</th>
- <th>完成时间</th>
- <th colspan = <%= failed_tasks.length?failed_tasks[0].reason.length:0%>>失败原因</th>
+ <th>ID</th>
+ <th colspan = <%= failed_tasks.length?failed_tasks[0].params.length:0 %>>Params</th>
+ <th>Queued</th>
+ <th>Finished</th>
+ <th colspan = <%= failed_tasks.length?failed_tasks[0].reason.length:0%>>Failed Reason</th>
</tr>
</thead>
<tbody>
@@ -160,11 +159,11 @@
<table class="table table-bordered">
<thead>
<tr>
- <th>编号</th>
- <th colspan = <%= slowest_tasks.length?slowest_tasks[0].params.length:0 %>>参数</th>
- <th>进入堆栈时间</th>
- <th>开始时间</th>
- <th>完成时间</th>
+ <th>ID</th>
+ <th colspan = <%= slowest_tasks.length?slowest_tasks[0].params.length:0 %>>Params</th>
+ <th>Queued</th>
+ <th>Started</th>
+ <th>Finished</th>
</tr>
</thead>
<% _.each(slowest_tasks, function(item) { %>
@@ -187,10 +186,10 @@
<table class="table table-bordered">
<thead>
<tr>
- <th>编号</th>
- <th colspan = <%= processing_tasks.length?processing_tasks[0].params.length:0 %>>参数</th>
- <th>进入堆栈时间</th>
- <th>开始时间</th>
+ <th>ID</th>
+ <th colspan = <%= processing_tasks.length?processing_tasks[0].params.length:0 %>>Params</th>
+ <th>Queued</th>
+ <th>Started</th>
</tr>
</thead>
<% _.each(processing_tasks, function(item) { %>

0 comments on commit f9d3a49

Please sign in to comment.
Something went wrong with that request. Please try again.