Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow all ArrayBufferView types as Buffers #12223

Closed
wants to merge 8 commits into from
Closed

Allow all ArrayBufferView types as Buffers #12223

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a8abe9c
src: allow ArrayBufferView as instance of Buffer
TimothyGu Apr 4, 2017
dae94ff
test: add common.getArrayBufferViews(buf)
TimothyGu Apr 4, 2017
c58b945
crypto: support all ArrayBufferView types
TimothyGu Apr 4, 2017
07a399b
zlib: support all ArrayBufferView types
TimothyGu Apr 4, 2017
1459f27
fixup crypto
TimothyGu Apr 5, 2017
039208e
fixup zlib
TimothyGu Apr 5, 2017
072e898
fixup test
TimothyGu Apr 5, 2017
1315240
fixup util
TimothyGu Apr 5, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
127 changes: 67 additions & 60 deletions doc/api/crypto.md
View file
Open in desktop

Large diffs are not rendered by default.

69 changes: 56 additions & 13 deletions doc/api/zlib.md
View file
Open in desktop
Expand Up @@ -300,7 +300,7 @@ ignored by the decompression classes.
* `level` {integer} (compression only)
* `memLevel` {integer} (compression only)
* `strategy` {integer} (compression only)
* `dictionary` {Buffer|Uint8Array} (deflate/inflate only, empty dictionary by
* `dictionary` {Buffer|TypedArray|DataView} (deflate/inflate only, empty dictionary by
default)

See the description of `deflateInit2` and `inflateInit2` at
Expand Down Expand Up @@ -477,9 +477,9 @@ Returns a new [Unzip][] object with an [options][].

<!--type=misc-->

All of these take a [Buffer][], [Uint8Array][], or string as the first
argument, an optional second argument to supply options to the `zlib` classes
and will call the supplied callback with `callback(error, result)`.
All of these take a [`Buffer`][], [`TypedArray`][], [`DataView`][], or string as
the first argument, an optional second argument to supply options to the `zlib`
classes and will call the supplied callback with `callback(error, result)`.

Every method has a `*Sync` counterpart, which accept the same arguments, but
without a callback.
Expand All @@ -488,6 +488,9 @@ without a callback.
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
Expand All @@ -496,19 +499,25 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|Uint8Array|string}
- `buffer` {Buffer|TypedArray|DataView|string}

Compress a chunk of data with [Deflate][].

### zlib.deflateRaw(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
Expand All @@ -517,19 +526,25 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|Uint8Array|string}
- `buffer` {Buffer|TypedArray|DataView|string}

Compress a chunk of data with [DeflateRaw][].

### zlib.gunzip(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
Expand All @@ -538,19 +553,25 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|Uint8Array|string}
- `buffer` {Buffer|TypedArray|DataView|string}

Decompress a chunk of data with [Gunzip][].

### zlib.gzip(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
Expand All @@ -559,19 +580,25 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|Uint8Array|string}
- `buffer` {Buffer|TypedArray|DataView|string}

Compress a chunk of data with [Gzip][].

### zlib.inflate(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
Expand All @@ -580,19 +607,25 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|Uint8Array|string}
- `buffer` {Buffer|TypedArray|DataView|string}

Decompress a chunk of data with [Inflate][].

### zlib.inflateRaw(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
Expand All @@ -601,19 +634,25 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|Uint8Array|string}
- `buffer` {Buffer|TypedArray|DataView|string}

Decompress a chunk of data with [InflateRaw][].

### zlib.unzip(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
Expand All @@ -622,12 +661,15 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: REPLACEME
description: The `buffer` parameter can be any TypedArray or DataView now.
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12001
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|Uint8Array|string}
- `buffer` {Buffer|TypedArray|DataView|string}

Decompress a chunk of data with [Unzip][].

Expand All @@ -644,5 +686,6 @@ Decompress a chunk of data with [Unzip][].
[InflateRaw]: #zlib_class_zlib_inflateraw
[Unzip]: #zlib_class_zlib_unzip
[`.flush()`]: #zlib_zlib_flush_kind_callback
[Buffer]: buffer.html
[Uint8Array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array
[`Buffer`]: buffer.html#buffer_class_buffer
[`DataView`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView
[`TypedArray`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray
5 changes: 2 additions & 3 deletions lib/crypto.js
View file
Open in desktop
Expand Up @@ -42,7 +42,6 @@ const timingSafeEqual = binding.timingSafeEqual;
const Buffer = require('buffer').Buffer;
const stream = require('stream');
const util = require('util');
const { isUint8Array } = process.binding('util');
const LazyTransform = require('internal/streams/lazy_transform');

const DH_GENERATOR = 2;
Expand Down Expand Up @@ -414,9 +413,9 @@ function DiffieHellman(sizeOrKey, keyEncoding, generator, genEncoding) {

if (typeof sizeOrKey !== 'number' &&
typeof sizeOrKey !== 'string' &&
!isUint8Array(sizeOrKey)) {
!ArrayBuffer.isView(sizeOrKey)) {
throw new TypeError('First argument should be number, string, ' +
'Uint8Array or Buffer');
'Buffer, TypedArray, or DataView');
}

if (keyEncoding) {
Expand Down
27 changes: 13 additions & 14 deletions lib/zlib.js
View file
Open in desktop
Expand Up @@ -24,7 +24,6 @@
const Buffer = require('buffer').Buffer;
const internalUtil = require('internal/util');
const Transform = require('_stream_transform');
const { isUint8Array } = process.binding('util');
const binding = process.binding('zlib');
const assert = require('assert').ok;
const kMaxLength = require('buffer').kMaxLength;
Expand Down Expand Up @@ -79,9 +78,9 @@ function isInvalidStrategy(strategy) {
}

function zlibBuffer(engine, buffer, callback) {
// Streams do not support non-Buffer Uint8Arrays yet. Convert it to a
// Streams do not support non-Buffer ArrayBufferViews yet. Convert it to a
// Buffer without copying.
if (isUint8Array(buffer) &&
if (ArrayBuffer.isView(buffer) &&
Object.getPrototypeOf(buffer) !== Buffer.prototype) {
buffer = Buffer.from(buffer.buffer, buffer.byteOffset, buffer.byteLength);
}
Expand All @@ -99,7 +98,7 @@ function zlibBuffer(engine, buffer, callback) {
var chunk;
while (null !== (chunk = engine.read())) {
buffers.push(chunk);
nread += chunk.length;
nread += chunk.byteLength;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chunk is engine output, so it’s always a Buffer, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, but I'd rather err on the side of replacing too many Buffer.lengths rather than missing an ArrayBufferView.byteLength, since Buffer.length === Buffer.byteLength.

}
engine.once('readable', flow);
}
Expand Down Expand Up @@ -129,9 +128,9 @@ function zlibBuffer(engine, buffer, callback) {
function zlibBufferSync(engine, buffer) {
if (typeof buffer === 'string')
buffer = Buffer.from(buffer);
else if (!isUint8Array(buffer))
throw new TypeError('"buffer" argument must be a string, Buffer, or ' +
'Uint8Array');
else if (!ArrayBuffer.isView(buffer))
throw new TypeError('"buffer" argument must be a string, Buffer, ' +
'TypedArray, or DataView');

var flushFlag = engine._finishFlushFlag;

Expand Down Expand Up @@ -214,9 +213,9 @@ class Zlib extends Transform {
throw new TypeError('Invalid strategy: ' + opts.strategy);

if (opts.dictionary) {
if (!isUint8Array(opts.dictionary)) {
if (!ArrayBuffer.isView(opts.dictionary)) {
throw new TypeError(
'Invalid dictionary: it should be a Buffer or an Uint8Array');
'Invalid dictionary: it should be a Buffer, TypedArray, or DataView');
}
}

Expand Down Expand Up @@ -309,9 +308,9 @@ class Zlib extends Transform {
var flushFlag;
var ws = this._writableState;
var ending = ws.ending || ws.ended;
var last = ending && (!chunk || ws.length === chunk.length);
var last = ending && (!chunk || ws.length === chunk.byteLength);

if (chunk !== null && !isUint8Array(chunk))
if (chunk !== null && !ArrayBuffer.isView(chunk))
return cb(new TypeError('invalid input'));

if (!this._handle)
Expand All @@ -328,7 +327,7 @@ class Zlib extends Transform {
flushFlag = this._flushFlag;
// once we've flushed the last of the queue, stop flushing and
// go back to the normal behavior.
if (chunk.length >= ws.length) {
if (chunk.byteLength >= ws.length) {
this._flushFlag = this._opts.flush || constants.Z_NO_FLUSH;
}
}
Expand All @@ -337,7 +336,7 @@ class Zlib extends Transform {
}

_processChunk(chunk, flushFlag, cb) {
var availInBefore = chunk && chunk.length;
var availInBefore = chunk && chunk.byteLength;
var availOutBefore = this._chunkSize - this._offset;
var inOff = 0;

Expand Down Expand Up @@ -417,7 +416,7 @@ class Zlib extends Transform {
self.push(out);
} else {
buffers.push(out);
nread += out.length;
nread += out.byteLength;
}
}

Expand Down