Skip to content

pconstr/recurrent

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
lib
 
 
 
 
 
 
 
 
 
 
 
 
 
 

recurrent

Build Status

A redis-backed manager of recurrent jobs, for node.js.

(immature, with minimal testing and probably nasty bugs)

Features

  • multiple queues, backed up by redis
  • tasks have a permanent unique id and mutable associated data
  • failed job runs are retried until they succeed or give up, with exponential back-off
  • job retries and repetitions don't pile up
  • the worker's job implementation determines at the end of each run when (and whether) to repeat
  • as many worker processes as you want spread over multiple cores and machines - but you have to start them

Caveats

  • single point of failure in queue manager (as well as redis itself) - you need to make sure it stays up / restarts quickly and there's exactly 1 instance

Installing

npm install recurrent

Running the manager

Make sure redis is running and then:

var recurrent = require('recurrent');

// will manage a queue q

var m = new recurrent.Manager('q').connect();
# pass arguments for redis.createClient() to connect()

recurrent offers a (currently very incomplete) web UI:

http.createServer(m.webUI).listen(7654);

And then browse to localhost:7654

When the manager is not needed anymore:

m.stop()

Starting a recurrent job

var recurrent = require('recurrent');

// starts a job which will run for the first time in about 30s

var c = new recurrent.Client('q').connect();
# pass arguments for redis.createClient() to connect()

c.add('t1', new Date().getTime() + 30000, {my: 'data'}, function(err) {
  ...
});

Adding again the same taskId will reset execution time.

When the client is not needed any more:

c.stop();

Recurrent job workers

var recurrent = require('recurrent');

function doWork(task, cb) {
  // do nothing for 600s
  console.log('got', task.id, 'with this data: ', task.data);
  setTimeout(function () {
    console.error('completed', task.id);

    // do again in about 5s
    cb(null, new Date().getTime()+ 5000);
  }, 600);
}

var w = new recurrent.Worker('q', doWork).connect();
# pass arguments for redis.createClient to connect()

When the job worker is not needed any more:

w.stop();

Task failure

Workers must not throw.

To signal failure a worker calls back with error. It can specify a retry time:

cb('something went wrong', new Date().getTime()+ 1000); // retry in 1s

Or let recurrent do exponential back-off

cb('something went wrong');

Exponential back-off can be configured per worker:

var w = new recurrent.Worker('q', doWork, {
  minBackOff: 500, //  start backing off at 500 ms
  maxBackOff:5000, // max back-off of 5 s
  backOffMultiplier:1.5 // back-off 50% longer every time
}).connect();

When a task is being retried after failure task.retries contains how many retries have been attempted (including the current one). It will undefined for the first execution, 1 for the first retry and so on.

About

A redis-backed manager of recurrent jobs, for node.js

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published