Content Addressable Blob Store
JavaScript
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bench
.gitignore
cabs.js
chunker.js
cli.js
index.js
package.json
readme.md
test.js

readme.md

cabs

A Content Addressable Blob Store for Node.

NPM

This implements something similar to the object store from git (the .git/objects folder). By default blobs stored by cabs are sha256 hashed and stored in subdirectories to avoid putting too many files in a single directory. The hashing algorithm and depth of the directories are configurable.

write

Cabs.write(path[, hashFunction, limit]);

Pipe in a blob stream (e.g. fs.createReadStream), get back objects for the various pieces stored. Each object has a hash of the block, as well as the starting and ending locations within the stream. You may optionally pass a string representing the hash function to use (default sha256) and number representing the block size (defaults to 5 MB).

To learn more about hashing algorithm tradeoffs read the comments on this issue.

read

Cabs.read(path);

Pipe in the objects from write, returns a readable stream of the blob.

example

var fs = require('fs')
var Cabs = require('cabs')

/** stream a movie into cabs, store hashes in hashes.json **/
fs.createReadStream('movie.avi')
  .pipe(Cabs.write('./storage'))
  .pipe(fs.createWriteStream('hashes.json'));
  
/** later, to retrieve the movie, stream the hashes into cabs **/
fs.createReadStream('hashes.json')
  .pipe(Cabs.read('./storage'))
  .pipe(fs.createWriteStream('movie-copy.avi'));

Low Level Class

you also have access to the base Cabs class located at cabs.Cabs, initialize it with a location and optionally an options object wtih a string representing the hash function to use (Defaults to sha256), the block size limit (defaults to 5 MB) and the depth of folders to use.

var store = new Cabs('./location');
// or
var store = new Cabs({
  path: './location',
  hashFunction: 'sha256',
  limit: 5 * 1024 *1024
  depth: 3
});

store.write(buffer, callback);
//stores buffer, callback is called with the hash

store.read(hash, callback);
//calls the callback with the blob

store.rm(hash, callback);
//removes the file with the given hash

store.destroy(callback);
//deletes all the files related to the store, just a shortcut to rimraf so beware.

store.readStream();
//same as Cabs.read

store.writeStream();
//same as Cabs.write

store.has(hash, callback);
//calls callback with true if it exists, otherwise false

store.check(hash, callback);
//similar to has but throws an error if the file doesn't exist
//or it's hash doesn't match it's address hash

store.writeFile();
//write to a single file on disc.  Will only ever emit a single string
//the hash for the combined file you streamed in.
//unlike writeStream which chunks a big file into multiple smaller ones
//which can be handled in memory, this method buffers to disk.