Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
A redis-backed manager of recurrent jobs, for node.js
JavaScript
branch: master
Failed to load latest commit information.
lib make timebox configurable, and default to 60s
tests add support for task data. persist retry count
.gitignore starting poin
.travis.yml let travis test against 0.8 rather than 0.7
MIT-LICENSE.txt licensed with MIT
README.md
package.json support node.js 0.8.x. bump up version for release
recurrent.js licensed with MIT

README.md

recurrent

Build Status

A redis-backed manager of recurrent jobs, for node.js.

(immature, with minimal testing and probably nasty bugs)

Features

  • multiple queues, backed up by redis
  • tasks have a permanent unique id and mutable associated data
  • failed job runs are retried until they succeed or give up, with exponential back-off
  • job retries and repetitions don't pile up
  • the worker's job implementation determines at the end of each run when (and whether) to repeat
  • as many worker processes as you want spread over multiple cores and machines - but you have to start them

Caveats

  • single point of failure in queue manager (as well as redis itself) - you need to make sure it stays up / restarts quickly and there's exactly 1 instance

Installing

npm install recurrent

Running the manager

Make sure redis is running and then:

var recurrent = require('recurrent');

// will manage a queue q

var m = new recurrent.Manager('q').connect();
# pass arguments for redis.createClient() to connect()

recurrent offers a (currently very incomplete) web UI:

http.createServer(m.webUI).listen(7654);

And then browse to localhost:7654

When the manager is not needed anymore:

m.stop()

Starting a recurrent job

var recurrent = require('recurrent');

// starts a job which will run for the first time in about 30s

var c = new recurrent.Client('q').connect();
# pass arguments for redis.createClient() to connect()

c.add('t1', new Date().getTime() + 30000, {my: 'data'}, function(err) {
  ...
});

Adding again the same taskId will reset execution time.

When the client is not needed any more:

c.stop();

Recurrent job workers

var recurrent = require('recurrent');

function doWork(task, cb) {
  // do nothing for 600s
  console.log('got', task.id, 'with this data: ', task.data);
  setTimeout(function () {
    console.error('completed', task.id);

    // do again in about 5s
    cb(null, new Date().getTime()+ 5000);
  }, 600);
}

var w = new recurrent.Worker('q', doWork).connect();
# pass arguments for redis.createClient to connect()

When the job worker is not needed any more:

w.stop();

Task failure

Workers must not throw.

To signal failure a worker calls back with error. It can specify a retry time:

cb('something went wrong', new Date().getTime()+ 1000); // retry in 1s

Or let recurrent do exponential back-off

cb('something went wrong');

Exponential back-off can be configured per worker:

var w = new recurrent.Worker('q', doWork, {
  minBackOff: 500, //  start backing off at 500 ms
  maxBackOff:5000, // max back-off of 5 s
  backOffMultiplier:1.5 // back-off 50% longer every time
}).connect();

When a task is being retried after failure task.retries contains how many retries have been attempted (including the current one). It will undefined for the first execution, 1 for the first retry and so on.

Something went wrong with that request. Please try again.