New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stream: support readable/writableHighWaterMark for Duplex streams #14636

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
7 participants
@guymguym
Contributor

guymguym commented Aug 4, 2017

Fixes: #14555

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines
Affected core subsystem(s)

stream

@gibfahn

This comment has been minimized.

Show comment
Hide comment
@gibfahn
Member

gibfahn commented Aug 5, 2017

@guymguym

This comment has been minimized.

Show comment
Hide comment
@guymguym

guymguym Aug 5, 2017

Contributor

@mcollina Thanks for the review! I fixed the comments and squashed.

Contributor

guymguym commented Aug 5, 2017

@mcollina Thanks for the review! I fixed the comments and squashed.

@lpinca

This comment has been minimized.

Show comment
Hide comment
@guymguym

This comment has been minimized.

Show comment
Hide comment
@guymguym

guymguym Aug 6, 2017

Contributor

@lpinca I'm trying to understand if the failure is related to my change, and I suspect it doesn't.
Let me know if I'm missing anything.

Contributor

guymguym commented Aug 6, 2017

@lpinca I'm trying to understand if the failure is related to my change, and I suspect it doesn't.
Let me know if I'm missing anything.

@lpinca

This comment has been minimized.

Show comment
Hide comment
@lpinca

lpinca Aug 6, 2017

Member

@guymguym you are correct, it's not related.

Member

lpinca commented Aug 6, 2017

@guymguym you are correct, it's not related.

@cjihrig

cjihrig approved these changes Aug 7, 2017

LGTM with a few comments.

@guymguym

This comment has been minimized.

Show comment
Hide comment
@guymguym

guymguym Aug 7, 2017

Contributor

@cjihrig Thanks, fixed and squashed.

Contributor

guymguym commented Aug 7, 2017

@cjihrig Thanks, fixed and squashed.

@mcollina

LGTM

@mcollina

This comment has been minimized.

Show comment
Hide comment
@mcollina

mcollina Aug 8, 2017

Member

Landed as c3c045a.

Member

mcollina commented Aug 8, 2017

Landed as c3c045a.

@mcollina mcollina closed this Aug 8, 2017

@mcollina

This comment has been minimized.

Show comment
Hide comment
@mcollina

mcollina Aug 8, 2017

Member

Congratulations for your first contribution to Node.js!

Member

mcollina commented Aug 8, 2017

Congratulations for your first contribution to Node.js!

@mcollina

This comment has been minimized.

Show comment
Hide comment
@mcollina

mcollina Aug 8, 2017

Member

I've added the semver-minor flag as it's a new feature.

Member

mcollina commented Aug 8, 2017

I've added the semver-minor flag as it's a new feature.

mcollina added a commit that referenced this pull request Aug 8, 2017

stream: support readable/writableHWM for Duplex
This commits adds support for readableHighWaterMark and
writableHighWaterMark in Duplex stream, so that they can be set without
accessing the internal state.

Fixes: #14555
PR-URL: #14636
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
@refack

A few comments a bit late. If we get some agreement I'll open a PR.

@@ -1752,6 +1752,10 @@ constructor and implement *both* the `readable._read()` and
* `writableObjectMode` {boolean} Defaults to `false`. Sets `objectMode`
for writable side of the stream. Has no effect if `objectMode`
is `true`.
* `readableHighWaterMark` {number} Sets `highWaterMark` for the readable side

This comment has been minimized.

@refack

refack Aug 8, 2017

Member

Suggestions:
Add a link to [#buffering]
Also in the first * (options) add links to [#constructor-new-streamwritableoptions] and [#new-streamreadableoptions]

@refack

refack Aug 8, 2017

Member

Suggestions:
Add a link to [#buffering]
Also in the first * (options) add links to [#constructor-new-streamwritableoptions] and [#new-streamreadableoptions]

@@ -1752,6 +1752,10 @@ constructor and implement *both* the `readable._read()` and
* `writableObjectMode` {boolean} Defaults to `false`. Sets `objectMode`
for writable side of the stream. Has no effect if `objectMode`
is `true`.
* `readableHighWaterMark` {number} Sets `highWaterMark` for the readable side
of the stream. Has no effect if `highWaterMark` is provided.

This comment has been minimized.

@refack

refack Aug 8, 2017

Member

I'd phrase it as The general `highWaterMark` option overrides this one. IMHO a positive-active sentence is a little bit clearer than a negative-passive one.

@refack

refack Aug 8, 2017

Member

I'd phrase it as The general `highWaterMark` option overrides this one. IMHO a positive-active sentence is a little bit clearer than a negative-passive one.

This comment has been minimized.

@guymguym

guymguym Sep 10, 2017

Contributor

I agree that positive is better. I just copied this format from readable/writableObjectMode.

@guymguym

guymguym Sep 10, 2017

Contributor

I agree that positive is better. I just copied this format from readable/writableObjectMode.

// However, some cases require setting options to different
// values for the readable and the writable sides of the duplex stream.
// These options can be provided separately as readableXXX and writableXXX.
var isDuplex = stream instanceof Stream.Duplex;

This comment has been minimized.

@refack

refack Aug 8, 2017

Member

@mcollina isn't instanceof discouraged?

@refack

refack Aug 8, 2017

Member

@mcollina isn't instanceof discouraged?

This comment has been minimized.

@mcollina

mcollina Aug 8, 2017

Member

yes, but this was already there, this PR just moves it.

@mcollina

mcollina Aug 8, 2017

Member

yes, but this was already there, this PR just moves it.

This comment has been minimized.

@guymguym

guymguym Sep 10, 2017

Contributor

In my opinion it is best to allow it also for Readable/Writable without checking for Duplex, because this is an inherited behavior that behaves different on the base class vs the child class, and this is more confusing than helping. Not sure if this will be agreed by any developer reading the docs though. I wonder what you guys think.

@guymguym

guymguym Sep 10, 2017

Contributor

In my opinion it is best to allow it also for Readable/Writable without checking for Duplex, because this is an inherited behavior that behaves different on the base class vs the child class, and this is more confusing than helping. Not sure if this will be agreed by any developer reading the docs though. I wonder what you guys think.

});
// test undefined, null, NaN
[undefined, null, NaN].forEach((v) => {

This comment has been minimized.

@refack

refack Aug 8, 2017

Member

what happens with ['', 'foo', ()=>{}, Symbol('frog'), {}, []]?

@refack

refack Aug 8, 2017

Member

what happens with ['', 'foo', ()=>{}, Symbol('frog'), {}, []]?

This comment has been minimized.

@guymguym

guymguym Sep 10, 2017

Contributor

@refack The behavior will be unspecified in these cases, since highWaterMark is expected to be a number, but only checked to be a truthy value or 0.

This was also the case before my PR, though I would prefer it to be a specified behavior instead.

Perhaps the behavior should be to ignore non number values or else throw TypeError as a breaking change...

@guymguym

guymguym Sep 10, 2017

Contributor

@refack The behavior will be unspecified in these cases, since highWaterMark is expected to be a number, but only checked to be a truthy value or 0.

This was also the case before my PR, though I would prefer it to be a specified behavior instead.

Perhaps the behavior should be to ignore non number values or else throw TypeError as a breaking change...

@mcollina

This comment has been minimized.

Show comment
Hide comment
@mcollina

mcollina Aug 8, 2017

Member

@refack go ahead and fire a PR.

Member

mcollina commented Aug 8, 2017

@refack go ahead and fire a PR.

addaleax added a commit that referenced this pull request Aug 10, 2017

stream: support readable/writableHWM for Duplex
This commits adds support for readableHighWaterMark and
writableHighWaterMark in Duplex stream, so that they can be set without
accessing the internal state.

Fixes: #14555
PR-URL: #14636
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

icarter09 added a commit to icarter09/node that referenced this pull request Aug 12, 2017

stream: support readable/writableHWM for Duplex
This commits adds support for readableHighWaterMark and
writableHighWaterMark in Duplex stream, so that they can be set without
accessing the internal state.

Fixes: nodejs#14555
PR-URL: nodejs#14636
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

addaleax added a commit to addaleax/node that referenced this pull request Aug 13, 2017

addaleax added a commit that referenced this pull request Aug 13, 2017

doc: add missing `changes:` metadata for streams
Ref: #14636
PR-URL: #14810
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>

addaleax added a commit that referenced this pull request Aug 13, 2017

doc: add missing `changes:` metadata for streams
Ref: #14636
PR-URL: #14810
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>

addaleax added a commit that referenced this pull request Aug 13, 2017

2017-08-15, Version 8.4.0 (Current)
Notable changes

* **Inspector**
  * `require()` is available in the inspector console now.
    [#8837](#8837)

* **N-API**
  * New APIs for creating number values have been introduced.
    [#14573](#14573)

* **Stream**
  * For `Duplex` streams, the high water mark option can now be set
    independently for readable the readable and the writable side.
    [#14636](#14636)

* **Util**
  * `util.format` now supports the `%o` and `%O` specifiers for printing
    objects.
    [#14558](#14558)

@addaleax addaleax referenced this pull request Aug 13, 2017

Merged

v8.4.0 proposal #14811

addaleax added a commit that referenced this pull request Aug 13, 2017

2017-08-15, Version 8.4.0 (Current)
Notable changes

* **Inspector**
  * `require()` is available in the inspector console now.
    [#8837](#8837)

* **N-API**
  * New APIs for creating number values have been introduced.
    [#14573](#14573)

* **Stream**
  * For `Duplex` streams, the high water mark option can now be set
    independently for the readable and the writable side.
    [#14636](#14636)

* **Util**
  * `util.format` now supports the `%o` and `%O` specifiers for printing
    objects.
    [#14558](#14558)

PR-URL: #14811

addaleax added a commit that referenced this pull request Aug 14, 2017

2017-08-15, Version 8.4.0 (Current)
Notable changes

* **HTTP2**
  * Experimental support for the built-in `http2` has been added via the
    `--expose-http2` flag.
    [#14239](#14239)

* **Inspector**
  * `require()` is available in the inspector console now.
    [#8837](#8837)
  * Multiple contexts, as created by the `vm` module, are supported now.
    [#14465](#14465)

* **N-API**
  * New APIs for creating number values have been introduced.
    [#14573](#14573)

* **Stream**
  * For `Duplex` streams, the high water mark option can now be set
    independently for the readable and the writable side.
    [#14636](#14636)

* **Util**
  * `util.format` now supports the `%o` and `%O` specifiers for printing
    objects.
    [#14558](#14558)

PR-URL: #14811

evanlucas added a commit that referenced this pull request Aug 15, 2017

2017-08-15, Version 8.4.0 (Current)
Notable changes

* **HTTP2**
  * Experimental support for the built-in `http2` has been added via the
    `--expose-http2` flag.
    [#14239](#14239)

* **Inspector**
  * `require()` is available in the inspector console now.
    [#8837](#8837)
  * Multiple contexts, as created by the `vm` module, are supported now.
    [#14465](#14465)

* **N-API**
  * New APIs for creating number values have been introduced.
    [#14573](#14573)

* **Stream**
  * For `Duplex` streams, the high water mark option can now be set
    independently for the readable and the writable side.
    [#14636](#14636)

* **Util**
  * `util.format` now supports the `%o` and `%O` specifiers for printing
    objects.
    [#14558](#14558)

PR-URL: #14811

MSLaguana added a commit to nodejs/node-chakracore that referenced this pull request Aug 21, 2017

2017-08-15, Version 8.4.0 (Current)
Notable changes

* **HTTP2**
  * Experimental support for the built-in `http2` has been added via the
    `--expose-http2` flag.
    [#14239](nodejs/node#14239)

* **Inspector**
  * `require()` is available in the inspector console now.
    [#8837](nodejs/node#8837)
  * Multiple contexts, as created by the `vm` module, are supported now.
    [#14465](nodejs/node#14465)

* **N-API**
  * New APIs for creating number values have been introduced.
    [#14573](nodejs/node#14573)

* **Stream**
  * For `Duplex` streams, the high water mark option can now be set
    independently for the readable and the writable side.
    [#14636](nodejs/node#14636)

* **Util**
  * `util.format` now supports the `%o` and `%O` specifiers for printing
    objects.
    [#14558](nodejs/node#14558)

PR-URL: nodejs/node#14811

@MylesBorins MylesBorins referenced this pull request Jan 15, 2018

Closed

Semver Minor Tracking #298

gabrielschulhof added a commit to gabrielschulhof/node that referenced this pull request Mar 12, 2018

2017-08-15, Version 8.4.0 (Current)
Notable changes

* **HTTP2**
  * Experimental support for the built-in `http2` has been added via the
    `--expose-http2` flag.
    [#14239](nodejs#14239)

* **Inspector**
  * `require()` is available in the inspector console now.
    [#8837](nodejs#8837)
  * Multiple contexts, as created by the `vm` module, are supported now.
    [#14465](nodejs#14465)

* **N-API**
  * New APIs for creating number values have been introduced.
    [#14573](nodejs#14573)

* **Stream**
  * For `Duplex` streams, the high water mark option can now be set
    independently for the readable and the writable side.
    [#14636](nodejs#14636)

* **Util**
  * `util.format` now supports the `%o` and `%O` specifiers for printing
    objects.
    [#14558](nodejs#14558)

This applies parts of 1fe0741 that are
relevant to N-API.

PR-URL: nodejs#14811

gabrielschulhof added a commit to gabrielschulhof/node that referenced this pull request Mar 15, 2018

2017-08-15, Version 8.4.0 (Current)
Notable changes

* **HTTP2**
  * Experimental support for the built-in `http2` has been added via the
    `--expose-http2` flag.
    [#14239](nodejs#14239)

* **Inspector**
  * `require()` is available in the inspector console now.
    [#8837](nodejs#8837)
  * Multiple contexts, as created by the `vm` module, are supported now.
    [#14465](nodejs#14465)

* **N-API**
  * New APIs for creating number values have been introduced.
    [#14573](nodejs#14573)

* **Stream**
  * For `Duplex` streams, the high water mark option can now be set
    independently for the readable and the writable side.
    [#14636](nodejs#14636)

* **Util**
  * `util.format` now supports the `%o` and `%O` specifiers for printing
    objects.
    [#14558](nodejs#14558)

This applies parts of 1fe0741 that are
relevant to N-API.

PR-URL: nodejs#14811
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment