Split up asynchronous tasks among a configurable number of forked processes
CoffeeScript JavaScript Makefile Shell
Switch branches/tags
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
lib
test
.gitignore
LICENSE
Makefile
README.md
circle.yml
index.js
package.json
publish.sh

README.md

Async-ForkQueue

Async-ForkQueue is based on ForkQueue, but it allows setting a level of concurrency where each forked process will run that many at a time and provides a api for creating worker functions.

Install

  npm install async-forkqueue

API

var Queue = require('async-forkqueue');
var num_workers = 4;
var concurrency = 4;

var queue = new Queue(num_workers, concurrency, module_path);

for (var i = 0; i < 100; i++) {
  queue.push(i);
}

queue.end(callback);

Worker modules

Worker modules are spawned with child_process.fork.

A simple worker is below.

module.exports = function (payload, cb) {
  // Do something with the payload
  cb();
}