Skip to content
BLAKE3 hashing for JavaScript: native Node bindings (where available) and WebAssembly
TypeScript Rust Makefile JavaScript HTML
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/workflows
.vscode
rs
ts fix: allow importing as native browser module, tests Feb 12, 2020
.gitignore fix: browser bundle to use pure es modules Jan 24, 2020
.npmignore fix: browser bundle to use pure es modules Jan 24, 2020
.remarkrc refactor: reduce duplication in Node.js code, prep for reader feature Jan 19, 2020
LICENSE Initial Jan 10, 2020
Makefile fix: run js extension addition in build Feb 12, 2020
benchmark.js
browser-async.d.ts fix: allow importing as native browser module, tests Feb 12, 2020
browser-async.js
browser-async.test.html fix: allow importing as native browser module, tests Feb 12, 2020
browser.d.ts fix: browser bundle to use pure es modules Jan 24, 2020
browser.js
changelog.md
package-lock.json 2.1.1 Feb 12, 2020
package.json 2.1.1 Feb 12, 2020
readme.md fix: allow importing as native browser module, tests Feb 12, 2020
test-input.txt Initial Jan 10, 2020
tsconfig.esm.json
tsconfig.json cleanup project a bit, add browser target and pptr tests Jan 12, 2020

readme.md

BLAKE3

BLAKE3 running in JavaScript (node.js and browsers) via native bindings, where available, or WebAssembly.

npm install blake3

Table of Contents

Quickstart

If you're on Node, import the module via

const blake3 = require('blake3');

blake3.hash('foo'); // => Buffer

If you're in the browser, import blake3/browser. This includes a WebAssembly binary, so you probably want to import it asynchronously, like so:

import('blake3/browser').then(blake3 => {
  blake3.hash('foo'); // => Uint8Array
});

The API is very similar in Node.js and browsers, but Node supports and returns Buffers and a wider range of input and output encoding.

More complete example:

const { hash, createHash } = require('blake3');

hash('some string'); // => hash a string to a uint8array

// Update incrementally (Node and Browsers):
const hash = createHash();
stream.on('data', d => hash.update(hash));
stream.on('error', err => {
  // hashes use unmanaged memory in WebAssembly, always free them if you don't digest()!
  hash.dispose();
  throw err;
});
stream.on('end', () => finishedHash(hash.digest()));

// Or, in Node, it's also a transform stream:
createReadStream('file.txt')
  .pipe(createHash())
  .on('data', hash => console.log(hash.toString('hex')));

API

Node.js

The Node API can be imported via require('blake3').

hash(data: BinaryLike, options?: { length: number }): Buffer

Returns a hash for the given data. The data can be a string, buffer, typedarray, array buffer, or array. By default, it creates a hash with the first 32 bytes of data, but this is configurable. It returns a Buffer.

keyedHash(key: Buffer, data: BinaryLike, options?: { length: number }): Buffer

Returns keyed a hash for the given data. The key must be exactly 32 bytes. The data can be a string, buffer, typedarray, array buffer, or array. By default, it creates a hash with the first 32 bytes of data, but this is configurable. It returns a Buffer.

For more information, see the blake3 docs.

deriveKey(context: string, material: BinaryLike, options?: { length: number }): Buffer

The key derivation function. The data can be a string, buffer, typedarray, array buffer, or array. By default, it creates a hash with the first 32 bytes of data, but this is configurable. It returns a Buffer.

For more information, see the blake3 docs.

Hasher

The hasher is a type that lets you incrementally build a hash. It's compatible with Node's crypto hash instance. For instance, it implements a transform stream, so you could do something like:

createReadStream('file.txt')
  .pipe(createHash())
  .on('data', hash => console.log(hash.toString('hex')));
createHash(): Hasher

Creates a new hasher instance using the standard hash function.

createKeyed(key: Buffer): Hasher

Creates a new hasher instance for a keyed hash. For more information, see the blake3 docs.

createDeriveKey(key: Buffer): Hasher

Creates a new hasher instance for the key derivation function. For more information, see the blake3 docs.

hasher.update(data: BinaryLike): this

Adds data to a hash. The data can be a string, buffer, typedarray, array buffer, or array. This will throw if called after digest() or dispose().

hasher.digest(encoding?: string, options?: { length: number, dispose: boolean })): Buffer | string

Returns the hash of the data. If an encoding is given, a string will be returned. Otherwise, a Buffer is returned. Optionally, you can specify the requested byte length of the hash.

If dispose: false is given in the options, the hash will not automatically be disposed of, allowing you to continue updating it after obtaining the current reader.

hasher.reader(options?: { dispose: boolean }): HashReader

Returns a HashReader for the current hash.

If dispose: false is given in the options, the hash will not automatically be disposed of, allowing you to continue updating it after obtaining the current reader.

hasher.dispose()

Disposes of unmanaged resources. You should always call this if you don't call digest() to free umanaged (WebAssembly-based) memory.

HashReader

The hash reader can be returned from hashing functions. Up to 264-1 bytes of data can be read from BLAKE3 hashes; this structure lets you read those. Note that, like hash, this is an object which needs to be manually disposed of.

reader.position: bigint

A property which gets or sets the position of the reader in the output stream. A RangeError is thrown if setting this to a value less than 0 or greater than 264-1. Note that this is a bigint, not a standard number.

reader.position += 32n; // advance the reader 32 bytes
reader.readInto(target: Buffer): void

Reads bytes into the target array, filling it up and advancing the reader's position. A RangeError is thrown if reading this data puts the reader past 264-1 bytes.

reader.read(bytes: number): Buffer

Reads and returns the given number of bytes from the reader, and advances the position. A RangeError is thrown if reading this data puts the reader past 264-1 bytes.

reader.toString([encoding]): string

Converts first 32 bytes of the hash to a string with the given encoding. Defaults to hex encoding.

reader.toBuffer(): Buffer

Converts first 32 bytes of the hash to a Buffer.

reader.dispose()

Disposes of unmanaged resources. You should always call this to free umanaged (WebAssembly-based) memory, or you application will leak memory.

using(disposable: IDisposable, fn: disposable => T): T

A helper method that takes a disposable, and automatically calls the dispose method when the function returns, or the promise returned from the function is settled.

// read and auto-dispose the first 64 bytes
const first64Bytes = using(hash.reader(), reader => reader.toBuffer(64));

// you can also return promises/use async methods:
using(hash.reader(), async reader => {
  do {
    await send(reader.read(64));
  } while (needsMoreData());
});

Browser

The browser API can be imported via import('blake3/browser'), which works well with Webpack.

If you aren't using a bundler or using a more "pure" bundler like Parcel, you can import blake3/browser-async which exports a function to asynchronously load the WebAssembly code and resolves to the package contents.

import load from 'blake3/browser-async';

load().then(blake3 => {
  console.log(blake3.hash('hello world'));
});

hash(data: BinaryLike, options?: { length: number }): Hash

Returns a hash for the given data. The data can be a string, typedarray, array buffer, or array. By default, it creates a hash with the first 32 bytes of data, but this is configurable. It returns a Hash instance.

keyedHash(key: Buffer, data: BinaryLike, options?: { length: number }): Hash

Returns keyed a hash for the given data. The key must be exactly 32 bytes. The data can be a string, typedarray, array buffer, or array. By default, it creates a hash with the first 32 bytes of data, but this is configurable. It returns a Hash instance.

For more information, see the blake3 docs.

deriveKey(context: string, material: BinaryLike, options?: { length: number }): Hash

The key derivation function. The data can be a string, typedarray, array buffer, or array. By default, it creates a hash with the first 32 bytes of data, but this is configurable. It returns a Hash instance.

For more information, see the blake3 docs.

Hash

A Hash is the type returned from hash functions and the hasher in the browser. It's a Uint8Array with a few additional helper methods.

hash.equals(other: Uint8Array)

Returns whether this hash equals the other hash, via a constant-time equality check.

hash.toString(encoding: 'hex' | 'base64' | 'utf8'): string

Hasher

The hasher is a type that lets you incrementally build a hash. For instance, you can hash a fetched page like:

const res = await fetch('https://example.com');
const body = await res.body;

const hasher = blake3.createHash();
const reader = body.getReader();

while (true) {
  const { done, value } = await reader.read();
  if (done) {
    break;
  }

  hasher.update(value);
}

console.log('Hash of', res.url, 'is', hasher.digest('hex'));

Converts the hash to a string with the given encoding.

createHash(): Hasher

Creates a new hasher instance using the standard hash function.

createKeyed(key: Buffer): Hasher

Creates a new hasher instance for a keyed hash. For more information, see the blake3 docs.

createDeriveKey(key: Buffer): Hasher

Creates a new hasher instance for the key derivation function. For more information, see the blake3 docs.

hasher.update(data: BinaryLike): this

Adds data to a hash. The data can be a string, buffer, typedarray, array buffer, or array. This will throw if called after digest() or dispose().

hasher.digest(encoding?: 'hex' | 'base64' | 'utf8', options?: { length: number, dispose: boolean })): Hash | string

Returns the hash of the data. If an encoding is given, a string will be returned. Otherwise, a Hash is returned. Optionally, you can specify the requested byte length of the hash.

If dispose: false is given in the options, the hash will not automatically be disposed of, allowing you to continue updating it after obtaining the current reader.

hasher.reader(options?: { dispose: boolean }): HashReader

Returns a HashReader for the current hash.

If dispose: false is given in the options, the hash will not automatically be disposed of, allowing you to continue updating it after obtaining the current reader.

hasher.dispose()

Disposes of unmanaged resources. You should always call this if you don't call digest() to free umanaged (WebAssembly-based) memory.

HashReader

The hash reader can be returned from hashing functions. Up to 264-1 bytes of data can be read from BLAKE3 hashes; this structure lets you read those. Note that, like hash, this is an object which needs to be manually disposed of.

reader.position: bigint

A property which gets or sets the position of the reader in the output stream. A RangeError is thrown if setting this to a value less than 0 or greater than 264-1. Note that this is a bigint, not a standard number.

reader.position += 32n; // advance the reader 32 bytes
reader.readInto(target: Uint8Array): void

Reads bytes into the target array, filling it up and advancing the reader's position. A RangeError is thrown if reading this data puts the reader past 264-1 bytes.

reader.read(bytes: number): Hash

Reads and returns the given number of bytes from the reader, and advances the position. A RangeError is thrown if reading this data puts the reader past 264-1 bytes.

reader.toString(encoding?: string): string

Converts first 32 bytes of the hash to a string with the given encoding. Defaults to hex encoding.

reader.toArray(): Uint8Array

Converts first 32 bytes of the hash to an array.

reader.dispose()

Disposes of unmanaged resources. You should always call this to free umanaged (WebAssembly-based) memory, or you application will leak memory.

using(disposable: IDisposable, fn: disposable => T): T

A helper method that takes a disposable, and automatically calls the dispose method when the function returns, or the promise returned from the function is settled.

// read and auto-dispose the first 64 bytes
const first64Bytes = using(hash.reader(), reader => reader.toArray(64));

// you can also return promises/use async methods:
using(hash.reader(), async reader => {
  do {
    await send(reader.read(64));
  } while (needsMoreData());
});

Speed

Native Node.js bindings are a work in progress.

You can run benchmarks by installing npm install -g @c4312/matcha, then running matcha benchmark.js. These are the results running on Node 12 on my MacBook. Blake3 is significantly faster than Node's built-in hashing.

  337,000 ops/sec > 64B#md5
  302,000 ops/sec > 64B#sha1
  276,000 ops/sec > 64B#sha256
  752,000 ops/sec > 64B#blake3

    11,700 ops/sec > 64KB#md5
    16,100 ops/sec > 64KB#sha1
     7,550 ops/sec > 64KB#sha256
    52,800 ops/sec > 64KB#blake3

       124 ops/sec > 6MB#md5
       175 ops/sec > 6MB#sha1
      80.2 ops/sec > 6MB#sha256
       518 ops/sec > 6MB#blake3

Contributing

This build is a little esoteric due to the mixing of languages. We use a Makefile to coodinate things.

To get set up, you'll need the following. Windows users are recommended to use WSL, no effort has been made to make this repo Windows-compatible.

  • A recent version of Node.js, such as 12.x
  • A make command
  • Rust installed locally
  • wasm-pack installed (cargo install wasm-pack once you have rust)
  • wasm-opt to create production releases, part of Binaryen

Then, run make prepare to install local dependencies.

Finally, make will create a build for you; you can run make MODE=release for a production release, and certainly should if you want to benchmark it.

  • Rust code is compiled from src/lib.rs to pkg/browser and pkg/node
  • TypeScript code is compiled from ts/*.ts into dist

Publishing

In case I get hit by a bus or get other contributors, these are the steps for publishing:

  1. Get all your code ready to go in master, pushed up to Github.
  2. Run make prepare-binaries. This will update the branch generate-binary, which kicks off a build via Github actions to create .node binaries for every relevant Node.js version.
  3. When the build completes, it'll generate a zip file of artifacts. Download those.
  4. Back on master, run npm version <type> to update the version in git. git push --tags.
  5. On Github, upload the contents of the artifacts folder to the release for the newly tagged version.
  6. Run npm publish.
You can’t perform that action at this time.