Skip to content
Permalink
Browse files

doc: add documentation for brotli support

PR-URL: #24938
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Ali Ijaz Sheikh <ofrobots@google.com>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
  • Loading branch information...
addaleax committed Dec 9, 2018
1 parent 9a627a4 commit 1c5a99797b5c3ab08551e66e03a1a0fac7be1019
Showing with 211 additions and 13 deletions.
  1. +5 −5 doc/api/errors.md
  2. +204 −8 doc/api/zlib.md
  3. +2 −0 tools/doc/type-parser.js
@@ -624,16 +624,16 @@ An attempt was made to register something that is not a function as an
The type of an asynchronous resource was invalid. Note that users are also able
to define their own types if using the public embedder API.

<a id="ERR_BROTLI_COMPRESSION_FAILED"></a>
### ERR_BROTLI_COMPRESSION_FAILED

Data passed to a Brotli stream was not successfully compressed.

<a id="ERR_BROTLI_INVALID_PARAM"></a>
### ERR_BROTLI_INVALID_PARAM

An invalid parameter key was passed during construction of a Brotli stream.

<a id="ERR_BROTLI_COMPRESSION_FAILED">
### ERR_BROTLI_COMPRESSION_FAILED

Data passed to a Brotli stream was not successfully compressed.

<a id="ERR_BUFFER_CONTEXT_NOT_AVAILABLE"></a>
### ERR_BUFFER_CONTEXT_NOT_AVAILABLE

@@ -5,7 +5,7 @@
> Stability: 2 - Stable
The `zlib` module provides compression functionality implemented using Gzip and
Deflate/Inflate. It can be accessed using:
Deflate/Inflate, as well as Brotli. It can be accessed using:

```js
const zlib = require('zlib');
@@ -54,8 +54,8 @@ and/or unrecoverable and catastrophic memory fragmentation.

## Compressing HTTP requests and responses

The `zlib` module can be used to implement support for the `gzip` and `deflate`
content-encoding mechanisms defined by
The `zlib` module can be used to implement support for the `gzip`, `deflate`
and `br` content-encoding mechanisms defined by
[HTTP](https://tools.ietf.org/html/rfc7230#section-4.2).

The HTTP [`Accept-Encoding`][] header is used within an http request to identify
@@ -76,12 +76,15 @@ const fs = require('fs');
const request = http.get({ host: 'example.com',
path: '/',
port: 80,
headers: { 'Accept-Encoding': 'gzip,deflate' } });
headers: { 'Accept-Encoding': 'br,gzip,deflate' } });
request.on('response', (response) => {
const output = fs.createWriteStream('example.com_index.html');
switch (response.headers['content-encoding']) {
// Or, just use zlib.createUnzip() to handle both cases
case 'br':
response.pipe(zlib.createBrotliDecompress()).pipe(output);
break;
// Or, just use zlib.createUnzip() to handle both of the following cases:
case 'gzip':
response.pipe(zlib.createGunzip()).pipe(output);
break;
@@ -117,6 +120,9 @@ http.createServer((request, response) => {
} else if (/\bgzip\b/.test(acceptEncoding)) {
response.writeHead(200, { 'Content-Encoding': 'gzip' });
raw.pipe(zlib.createGzip()).pipe(response);
} else if (/\bbr\b/.test(acceptEncoding)) {
response.writeHead(200, { 'Content-Encoding': 'br' });
raw.pipe(zlib.createBrotliCompress()).pipe(response);
} else {
response.writeHead(200, {});
raw.pipe(response);
@@ -136,6 +142,7 @@ const buffer = Buffer.from('eJzT0yMA', 'base64');
zlib.unzip(
buffer,
// For Brotli, the equivalent is zlib.constants.BROTLI_OPERATION_FLUSH.
{ finishFlush: zlib.constants.Z_SYNC_FLUSH },
(err, buffer) => {
if (!err) {
@@ -156,6 +163,8 @@ decompressed result is valid.

<!--type=misc-->

### For zlib-based streams

From `zlib/zconf.h`, modified to Node.js's usage:

The memory requirements for deflate are (in bytes):
@@ -194,6 +203,16 @@ fewer calls to `zlib` because it will be able to process more data on
each `write` operation. So, this is another factor that affects the
speed, at the cost of memory usage.

### For Brotli-based streams

There are equivalents to the zlib options for Brotli-based streams, although
these options have different ranges than the zlib ones:

- zlib’s `level` option matches Brotli’s `BROTLI_PARAM_QUALITY` option.
- zlib’s `windowBits` option matches Brotli’s `BROTLI_PARAM_LGWIN` option.

See [below][Brotli parameters] for more details on Brotli-specific options.

## Flushing

Calling [`.flush()`][] on a compression stream will make `zlib` return as much
@@ -231,6 +250,8 @@ added: v0.5.8

<!--type=misc-->

### zlib constants

All of the constants defined in `zlib.h` are also defined on
`require('zlib').constants`. In the normal course of operations, it will not be
necessary to use these constants. They are documented so that their presence is
@@ -281,6 +302,77 @@ Compression strategy.
* `zlib.constants.Z_FIXED`
* `zlib.constants.Z_DEFAULT_STRATEGY`

### Brotli constants
<!-- YAML
added: REPLACEME
-->

There are several options and other constants available for Brotli-based
streams:

#### Flush operations

The following values are valid flush operations for Brotli-based streams:

* `zlib.constants.BROTLI_OPERATION_PROCESS` (default for all operations)
* `zlib.constants.BROTLI_OPERATION_FLUSH` (default when calling `.flush()`)
* `zlib.constants.BROTLI_OPERATION_FINISH` (default for the last chunk)
* `zlib.constants.BROTLI_OPERATION_EMIT_METADATA`
* This particular operation may be hard to use in a Node.js context,
as the streaming layer makes it hard to know which data will end up
in this frame. Also, there is currently no way to consume this data through
the Node.js API.

#### Compressor options

There are several options that can be set on Brotli encoders, affecting
compression efficiency and speed. Both the keys and the values can be accessed
as properties of the `zlib.constants` object.

The most important options are:

* `BROTLI_PARAM_MODE`
* `BROTLI_MODE_GENERIC` (default)
* `BROTLI_MODE_TEXT`, adjusted for UTF-8 text
* `BROTLI_MODE_FONT`, adjusted for WOFF 2.0 fonts
* `BROTLI_PARAM_QUALITY`
* Ranges from `BROTLI_MIN_QUALITY` to `BROTLI_MAX_QUALITY`,
with a default of `BROTLI_DEFAULT_QUALITY`.
* `BROTLI_PARAM_SIZE_HINT`
* Integer value representing the expected input size;
defaults to `0` for an unknown input size.

The following flags can be set for advanced control over the compression
algorithm and memory usage tuning:

* `BROTLI_PARAM_LGWIN`
* Ranges from `BROTLI_MIN_WINDOW_BITS` to `BROTLI_MAX_WINDOW_BITS`,
with a default of `BROTLI_DEFAULT_WINDOW`, or up to
`BROTLI_LARGE_MAX_WINDOW_BITS` if the `BROTLI_PARAM_LARGE_WINDOW` flag
is set.
* `BROTLI_PARAM_LGBLOCK`
* Ranges from `BROTLI_MIN_INPUT_BLOCK_BITS` to `BROTLI_MAX_INPUT_BLOCK_BITS`.
* `BROTLI_PARAM_DISABLE_LITERAL_CONTEXT_MODELING`
* Boolean flag that decreases compression ratio in favour of
decompression speed.
* `BROTLI_PARAM_LARGE_WINDOW`
* Boolean flag enabling “Large Window Brotli” mode (not compatible with the
Brotli format as standardized in [RFC 7932][]).
* `BROTLI_PARAM_NPOSTFIX`
* Ranges from `0` to `BROTLI_MAX_NPOSTFIX`.
* `BROTLI_PARAM_NDIRECT`
* Ranges from `0` to `15 << NPOSTFIX` in steps of `1 << NPOSTFIX`.

#### Decompressor options

These advanced options are available for controlling decompression:

* `BROTLI_DECODER_PARAM_DISABLE_RING_BUFFER_REALLOCATION`
* Boolean flag that affects internal memory allocation patterns.
* `BROTLI_DECODER_PARAM_LARGE_WINDOW`
* Boolean flag enabling “Large Window Brotli” mode (not compatible with the
Brotli format as standardized in [RFC 7932][]).

## Class: Options
<!-- YAML
added: v0.11.1
@@ -298,7 +390,7 @@ changes:

<!--type=misc-->

Each class takes an `options` object. All options are optional.
Each zlib-based class takes an `options` object. All options are optional.

Note that some options are only relevant when compressing, and are
ignored by the decompression classes.
@@ -317,6 +409,47 @@ ignored by the decompression classes.
See the description of `deflateInit2` and `inflateInit2` at
<https://zlib.net/manual.html#Advanced> for more information on these.

## Class: BrotliOptions
<!-- YAML
added: REPLACEME
-->

<!--type=misc-->

Each Brotli-based class takes an `options` object. All options are optional.

* `flush` {integer} **Default:** `zlib.constants.BROTLI_OPERATION_PROCESS`
* `finishFlush` {integer} **Default:** `zlib.constants.BROTLI_OPERATION_FINISH`
* `chunkSize` {integer} **Default:** `16 * 1024`
* `params` {Object} Key-value object containing indexed [Brotli parameters][].

For example:

```js
const stream = zlib.createBrotliCompress({
chunkSize: 32 * 1024,
params: {
[zlib.constants.BROTLI_PARAM_MODE]: zlib.constants.BROTLI_MODE_TEXT,
[zlib.constants.BROTLI_PARAM_QUALITY]: 4,
[zlib.constants.BROTLI_PARAM_SIZE_HINT]: fs.statSync(inputFile).size
}
});
```

## Class: zlib.BrotliCompress
<!-- YAML
added: REPLACEME
-->

Compress data using the Brotli algorithm.

## Class: zlib.BrotliDecompress
<!-- YAML
added: REPLACEME
-->

Decompress data using the Brotli algorithm.

## Class: zlib.Deflate
<!-- YAML
added: v0.5.8
@@ -389,9 +522,13 @@ added: v0.5.8
Decompress either a Gzip- or Deflate-compressed stream by auto-detecting
the header.

## Class: zlib.Zlib
## Class: zlib.ZlibBase
<!-- YAML
added: v0.5.8
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/24939
description: This class was renamed from `Zlib` to `ZlibBase`.
-->

Not exported by the `zlib` module. It is documented here because it is the base
@@ -440,7 +577,8 @@ Close the underlying handle.
added: v0.5.8
-->

* `kind` **Default:** `zlib.constants.Z_FULL_FLUSH`
* `kind` **Default:** `zlib.constants.Z_FULL_FLUSH` for zlib-based streams,
`zlib.constants.BROTLI_OPERATION_FLUSH` for Brotli-based streams.
* `callback` {Function}

Flush pending data. Don't call this frivolously, premature flushes negatively
@@ -460,6 +598,8 @@ added: v0.11.4
* `strategy` {integer}
* `callback` {Function}

This function is only available for zlib-based streams, i.e. not Brotli.

Dynamically update the compression level and compression strategy.
Only applicable to deflate algorithm.

@@ -478,6 +618,24 @@ added: v7.0.0

Provides an object enumerating Zlib-related constants.

## zlib.createBrotliCompress([options])
<!-- YAML
added: REPLACEME
-->

* `options` {brotli options}

Creates and returns a new [`BrotliCompress`][] object.

## zlib.createBrotliDecompress([options])
<!-- YAML
added: REPLACEME
-->

* `options` {brotli options}

Creates and returns a new [`BrotliDecompress`][] object.

## zlib.createDeflate([options])
<!-- YAML
added: v0.5.8
@@ -560,6 +718,40 @@ with `callback(error, result)`.
Every method has a `*Sync` counterpart, which accept the same arguments, but
without a callback.

### zlib.brotliCompress(buffer[, options], callback)
<!-- YAML
added: REPLACEME
-->
* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}
* `options` {brotli options}
* `callback` {Function}

### zlib.brotliCompressSync(buffer[, options])
<!-- YAML
added: REPLACEME
-->
* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}
* `options` {brotli options}

Compress a chunk of data with [`BrotliCompress`][].

### zlib.brotliDecompress(buffer[, options], callback)
<!-- YAML
added: REPLACEME
-->
* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}
* `options` {brotli options}
* `callback` {Function}

### zlib.brotliDecompressSync(buffer[, options])
<!-- YAML
added: REPLACEME
-->
* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}
* `options` {brotli options}

Decompress a chunk of data with [`BrotliDecompress`][].

### zlib.deflate(buffer[, options], callback)
<!-- YAML
added: v0.6.0
@@ -832,6 +1024,8 @@ Decompress a chunk of data with [`Unzip`][].
[`.flush()`]: #zlib_zlib_flush_kind_callback
[`Accept-Encoding`]: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.3
[`ArrayBuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer
[`BrotliCompress`]: #zlib_class_zlib_brotlicompress
[`BrotliDecompress`]: #zlib_class_zlib_brotlidecompress
[`Buffer`]: buffer.html#buffer_class_buffer
[`Content-Encoding`]: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11
[`DataView`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView
@@ -845,6 +1039,8 @@ Decompress a chunk of data with [`Unzip`][].
[`Unzip`]: #zlib_class_zlib_unzip
[`stream.Transform`]: stream.html#stream_class_stream_transform
[`zlib.bytesWritten`]: #zlib_zlib_byteswritten
[Brotli parameters]: #zlib_brotli_constants
[Memory Usage Tuning]: #zlib_memory_usage_tuning
[RFC 7932]: https://www.rfc-editor.org/rfc/rfc7932.txt
[pool size]: cli.html#cli_uv_threadpool_size_size
[zlib documentation]: https://zlib.net/manual.html#Constants
@@ -38,6 +38,8 @@ const customTypesMap = {
'AsyncHook': 'async_hooks.html#async_hooks_async_hooks_createhook_callbacks',
'AsyncResource': 'async_hooks.html#async_hooks_class_asyncresource',

'brotli options': 'zlib.html#zlib_class_brotlioptions',

'Buffer': 'buffer.html#buffer_class_buffer',

'ChildProcess': 'child_process.html#child_process_class_childprocess',

0 comments on commit 1c5a997

Please sign in to comment.
You can’t perform that action at this time.