node native addon for rabin fingerprinting data streams
Clone or download


Node native addon module (C/C++) for Rabin fingerprinting data streams.

Build Status Build status

Uses the implementation of Rabin fingerprinting from LBFS.

Rabin fingerprinting is useful for finding the chunks of a file that differ from a previous version. It's one implementation of a technique called "Content-defined chunking", meaning the chunk boundaries are determinstic to the content (as opposed to "fixed-sized chunking").

Theres a JavaScript API and an accompanying command-line tool.

JavaScript API

var createRabin = require('rabin')

createRabin can be used to create multiple fingerprinting streams

var rabin = createRabin()

rabin is a duplex stream. You write raw data in, and buffers chunked by rabin fingerprints will be written out.

JavaScript Example

// require and create an instance
var rabin = require('rabin')()

// pipe some data in
var rs = fs.createReadStream('somefile.dat')

// handle output chunks
rabin.on('data', function (chunk) {
  // chunks are created by taking your input data
  // and splitting on each rabin fingerprint found


$ npm install rabin -g
$ rabin myfile.txt --bits=14 --min=8192 --max=32768 # defaults
average 12500