Scalable, fault-tolerant application-layer sharding for Node.js applications
JavaScript Shell
Latest commit 5b21caa Jan 18, 2017 @mennopruijssers mennopruijssers committed on GitHub Retry discover provider (#314)
* Optionally retry discover provider on error
Permalink
Failed to load latest commit information.
benchmarks Update licence headers Apr 14, 2016
examples Update licence headers Apr 14, 2016
lib Retry discover provider (#314) Jan 18, 2017
server Refactor membership to use a localMemer as the source of truth about … Aug 2, 2016
test Retry discover provider (#314) Jan 18, 2017
.gitignore Initial commit Jan 17, 2015
.jshintignore Fix tools recursing into ringpop-common Apr 15, 2016
.jshintrc Initial commit Jan 17, 2015
.travis.yml Have updated to use the legacy implementation of hash32, and have add… Aug 22, 2016
CHANGES.md Update changelog in preparation of 10.17.0 release (#311) Nov 22, 2016
LICENSE Initial commit Jan 17, 2015
MIGRATION.md Fixed a typo in last MIGRATION note Feb 2, 2016
README.md Add node version requirements Jul 13, 2016
client.js Add server/client middleware to patch tombstone state Apr 28, 2016
client_errors.js Update licence headers Apr 14, 2016
client_request.js Add server/client middleware to patch tombstone state Apr 28, 2016
client_request_errors.js Update licence headers Apr 14, 2016
client_request_states.js Update licence headers Apr 14, 2016
config.js [self-evict 2] Gossip self eviction (#300) Oct 18, 2016
discover-providers.js Retry discover provider (#314) Jan 18, 2017
index.js [Feature] Key consistent forwarding (#310) Nov 17, 2016
main.js [self-evict 3] self evict on shutdown in testpop and enable self-evic… Oct 18, 2016
package.json Retry discover provider (#314) Jan 18, 2017
request_response.js Update licence headers Apr 14, 2016
ringpop-handler.js Update licence headers Apr 14, 2016
ringpop_errors.js Update licence headers Apr 14, 2016

README.md

ringpop-node Build Status

Ringpop is a library that brings cooperation and coordination to distributed applications. It maintains a consistent hash ring on top of a membership protocol and provides request forwarding as a routing convenience. It can be used to shard your application in a way that's scalable and fault tolerant.

Requirements

  • Node 0.10 (0.10.32 or higher)

Installation

To install Ringpop for usage as a library:

npm install ringpop

Prepare the current directory for development:

npm install

To be able to run the tests, make sure you have your open file limit restriction on at least 4K:

ulimit -n 4096

Tick Cluster

An example application tools/tick-cluster.js is included in ringpop-common repository. It just launches a ringpop cluster of a given size. Using this application is the quickest way to start a ringpop cluster.

git clone https://github.com/uber/ringpop-common.git
./ringpop-common/tools/tick-cluster.js --interpreter node main.js

Example

Run a 2-node Ringpop cluster from the command-line. Install Ringpop and TChannel, copy/paste the below into your editor and run!

var Ringpop = require('ringpop');
var TChannel = require('tchannel');

function Cluster(opts) {
    this.name = opts.name;
    this.size = opts.size;
    this.basePort = opts.basePort;
    this.bootstrapNodes = [];

    // Create the bootstrap list of nodes that'll
    // be used to seed Ringpop for its join request.
    for (var i = 0; i < this.size; i++) {
        this.bootstrapNodes.push('127.0.0.1:' + (this.basePort + i));
    }
}

Cluster.prototype.launch = function launch(callback) {
    var self = this;
    var done = after(self.size, callback);

    for (var i = 0; i < this.size; i++) {
        var addr = this.bootstrapNodes[i];
        var addrParts = addr.split(':');

        var tchannel = new TChannel();
        var ringpop = new Ringpop({
            app: this.name,
            hostPort: addr,
            channel: tchannel.makeSubChannel({
                serviceName: 'ringpop',
                trace: false
            })
        });
        ringpop.setupChannel();

        // First make sure TChannel is accepting connections.
        tchannel.listen(+addrParts[1], addrParts[0], listenCb(ringpop));
    }


    function listenCb(ringpop) {
        // When TChannel is listening, bootstrap Ringpop. It'll
        // try to join its friends in the bootstrap list.
        return function onListen() {
            ringpop.bootstrap(self.bootstrapNodes, done);
        };
    }
};

// IGNORE THIS! It's a little utility function that invokes
// a callback after a specified number of invocations
// of its shim.
function after(count, callback) {
    var countdown = count;

    return function shim(err) {
        if (typeof callback !== 'function') return;

        if (err) {
            callback(err);
            callback = null;
            return;
        }

        if (--countdown === 0) {
            callback();
            callback = null;
        }
    };
}

if (require.main === module) {
    // Launch a Ringpop cluster of arbitrary size.
    var cluster = new Cluster({
        name: 'mycluster',
        size: 2,
        basePort: 3000
    });

    // When all nodes have been bootstrapped, your
    // Ringpop cluster will be ready for use.
    cluster.launch(function onLaunch(err) {
        if (err) {
            console.error('Error: failed to launch cluster');
            process.exit(1);
        }

        console.log('Ringpop cluster is ready!');
    });
}

Documentation

Interested in where to go from here? Read the docs at ringpop.readthedocs.org.