Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Simple, reliable request/reply-based parallel computing over TCP.
JavaScript
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
examples
lib
test
.gitignore
.travis.yml
README.md
index.js
package.json

README.md

stacks

Simple, reliable request/reply-based parallel computing over TCP.

Build Status

What is stacks?

Stacks provides a simple request/reply interface to parallel computing. It's simply a queue of requests that clients (seperate worker machines) reply to. It's designed to be reliable and easy-to-use in situations involving long, compute-heavy requests.

Features

  • JSON, BSON, binary support – msgpack protocol
  • Automatic round-robin scheduling
  • Automatic rescheduling of failed requests
  • Typed requests. client.on(type, function(body, reply) { ... interface
  • Configurable per-request-type worker concurrency limits
  • Supports zero or more client workers
  • Automatically saves requests received when no worker is available, and schedules them when the next worker connects
  • Automatically reschedules requests that were pending on clients that disconnect

Installation

To begin using stacks, you must have at least node v0.10 or higher installed.

Install from npm

 $ (sudo) npm install --save stacks

And add it to your project

 var stacks = require('stacks');
 ...

making requests

The master node makes requests to stacks. The requests are automatically handled by clients.

var stacks = require('stacks');
// Listen for clients on port 3000
stacks.listen(3000);

stacks.request('string reverse', 'hello', { maxRetries: 5 }, function (error, reversedString) {
  if (error) {
    console.error('Failed to reverse string after 5 retries. Error:', error);
  } else {
    console.log('Reversed string:', reversedString); // Reversed string: olleh
  }
});

handling requests

var stacks = require('stacks');
// connect to stacks server
stacks.connect(3000, function(client) {

  client.on('string reverse', function(string, reply) {
    var reversedString = string.split("").reverse().join("");
    var error = null;
    reply(error, reversedString);
  }, 15); // handle only 15 concurrent string reverse requests at a time

});

Server API

listen(port, [callback])

Listen for client (worker) connections

  • Number port: Port to listen for client connections on
  • Function callback: Callback function invoked when stacks has begun listening. Takes no arguments

request(type, body, [options], callback)

Make a request on stacks

  • String type: Type of request. Used in client interface to process/reply i.e., client.on(type, function(body, reply) {...
  • Mixed body: The request body that will be received by the client request worker. WARNING:Arrays will be parsed as arguments and cannot be sent as the top-level object in a request.
  • Object options: Request options.
  • Function callback: Callback function invoked when request completes (or fails after options.maxRetries). Takes arguments error, response.

request options

  • Number maxRetries (3): The maximum amount of retries to attempt before invoking request callback with error. (These retries are rescheduled by stacks (round-robin if multiple clients are connected)).

Client API

connect(server, callback)

Connect to stacks server

  • Mixed server: If Number, server is interpreted as port. If Object, server must have the signature { host: '[host-ip-address]', port: 3000 }
  • Function callback: Invoked after client is connected to stacks server. Takes argument client.

disconnect()

Disconnect from stacks server

Client

Instance methods of the client object passed to the #connect() callback.

on(type, worker, [concurrency])

Handle and reply to requests of type type

  • String type: Type of request to handle with concurrency concurrent worker functions
  • Function worker: Function that processes and replies to a request. Takes arguments body– the request body –and reply– the reply function.
  • Number concurrency (1): Limit of concurrent worker functions to use for requests of type type.

License

MIT

Something went wrong with that request. Please try again.