Distributed task execution framework for node, backed by redis.
var bgTask = require('background-task').connect({ taskKey: 'someKey' })
, task;
task = {
someKey : 'someValue',
someOtherKey : ['someValues', 'someMoreValues']
};
bgTask.addTask(task, function(resp){
console.log(resp);
});
$ npm install background-task
Background tasks involve two concepts, creators and workers. Creators have tasks that they need executed, workers know how to execute tasks. Workers listen for events tagged with a specific task and execute them. Should no worker execute the task in a specific time (which can be customized) then a failure is reported back.
Additionally, the amount of active tasks can be limited using a "task key" (which must be a property at the top level of you task object), where only N tasks can be pending on a certain key prior to failures.
connect(options)
-- Creates a new instance of a BackgroundTask, allowing you to specify options.addTask(task, callback[, progress])
-- Schedule a task for background execution, calling progress on task progress and `callback when complete.
Processing for a worker is kicked off by the TASK_AVAILABLE
event,
once you get this event you can either accept the task, or ignore the
task. If multiple workers accept the task then the first to accept
will handle the task.
connect(options)
-- Create a new instance of a BackgroundTask, allowing you to specify options. You must specifyisWorker: true
to register as a background worker.acceptTask(callback)
-- Accepts a task from the queue to start processing, callback must accept two args,id
and a paramater for the task.completeTask(taskId, status, msg)
-- Mark a task as complete.progressTask(taskId, msg)
-- Report task progress.
The options
hash for creators and workers allows you to set:
host
-- redis host.maxTasksPerKey
-- limit the amount of active tasks based on the "task key".password
-- redis password.port
-- redis port.taskKey
-- the task key.task
-- listen for specific tasks only.timeout
-- task timeout (in ms).
node-background-task uses the following events:
TASK_AVAILABLE
-- There is data available for a background worker.TASK_DONE
-- A task has finished and the response is ready, task may not be successful, just complete.TASK_PROGRESS
-- Progress is reported for a task.TASK_ERROR
-- Something went wrong.
- Lightweight
- Backed by redis (Can be portable to other DBs)
- Callback and Event based
- Easy to build distributed architectures
- Workers are event driven, not polling
- Task limiting
We looked at the excellent coffee-resque project to fit an internal need, however we needed a non-polling based worker architecture. There are up and down-sides to this approach, but the balance was correct for the architecture that we're creating at Kinvey.
Copyright 2013 Kinvey, Inc
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.