Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http: fix stalled pipeline bug #3342

Closed
wants to merge 1 commit into from
Closed

Conversation

@indutny
Copy link
Member

@indutny indutny commented Oct 13, 2015

This is a two-part fix:

  • Fix pending data notification in OutgoingMessage to notify server
    about flushed data too
  • unconsume socket from http parser on pause, to prevent parser from
    reporting PAUSED error. Previously, no data events were emitted
    after socket.pause() call. Now with all things happening in C++,
    socket.pause() has no effect on internal socket while it is
    consumed. Switching to old mode is enough for now, we should revisit
    it somewhere later.

Fix: #3332

R=@bnoordhuis @jasnell @trevnorris

cc @nodejs/collaborators

@indutny
Copy link
Member Author

@indutny indutny commented Oct 13, 2015

The test may be failing on windows, because it is a bit CPU intensive.

@indutny
Copy link
Member Author

@indutny indutny commented Oct 13, 2015

Going to sleep now, will reply to all comments tomorrow.

@thefourtheye
thefourtheye reviewed Oct 13, 2015
View changes
lib/_http_outgoing.js Outdated
@@ -663,6 +663,9 @@ OutgoingMessage.prototype._flush = function() {
this.output = [];
this.outputEncodings = [];
this.outputCallbacks = [];
if (this._onPendingData !== null)

This comment has been minimized.

@thefourtheye

thefourtheye Oct 13, 2015
Contributor

Wouldn't a better check be if (typeof this._onPendingData === 'function')?

This comment has been minimized.

@indutny

indutny Oct 13, 2015
Author Member

I'll move the checks together and replaced it with typeof, thanks

@indutny
Copy link
Member Author

@indutny indutny commented Oct 13, 2015

Pushed one fix, any other comments?

@jasnell
Copy link
Member

@jasnell jasnell commented Oct 13, 2015

LGTM assuming CI is green

jasnell added a commit that referenced this pull request Oct 13, 2015
* Includes fixes for two regressions
  - Assertion error in WeakCallback  - see [#3329](#3329)
  - Undefined timeout regression - see [#3331](#3331)

* Document an additional known issue with pipelined requests
  - See: #3332 and #3342
jasnell added a commit that referenced this pull request Oct 13, 2015
* Includes fixes for two regressions
  - Assertion error in WeakCallback  - see [#3329](#3329)
  - Undefined timeout regression - see [#3331](#3331)

* Document an additional known issue with pipelined requests
  - See: #3332 and #3342
jasnell added a commit that referenced this pull request Oct 13, 2015
* Includes fixes for two regressions
  - Assertion error in WeakCallback  - see [#3329](#3329)
  - Undefined timeout regression - see [#3331](#3331)

* Document an additional known issue with pipelined requests
  - See: #3332 and #3342
@trevnorris
Copy link
Contributor

@trevnorris trevnorris commented Oct 13, 2015

Fix pending data notification in OutgoingMessage to notify server about flushed data too

The "too" at the end makes me feel like there's more to the sentence.

@@ -135,7 +135,7 @@ OutgoingMessage.prototype._send = function(data, encoding, callback) {
this.outputEncodings.unshift('binary');
this.outputCallbacks.unshift(null);
this.outputSize += this._header.length;
if (this._onPendingData !== null)
if (typeof this._onPendingData === 'function')

This comment has been minimized.

@trevnorris

trevnorris Oct 13, 2015
Contributor

Is checking for typeof function part of a fix, or different cleanup?

This comment has been minimized.

@indutny

indutny Oct 13, 2015
Author Member

It is a cleanup.

@trevnorris
trevnorris reviewed Oct 13, 2015
View changes
test/parallel/test-http-pipeline-regr-3332.js Outdated
}
});
}).listen(8000, function() {
client = net.connect(8000, function() {

This comment has been minimized.

@trevnorris

trevnorris Oct 13, 2015
Contributor

common.PORT?

This comment has been minimized.

@indutny

indutny Oct 13, 2015
Author Member

Yikes, thanks!

This comment has been minimized.

@indutny

indutny Oct 13, 2015
Author Member

Fixed.

@trevnorris
trevnorris reviewed Oct 13, 2015
View changes
test/parallel/test-http-pipeline-regr-3332.js Outdated
client.once('drain', spam);
}

spam();

This comment has been minimized.

@trevnorris

trevnorris Oct 13, 2015
Contributor

Curious. why not just do:

net.connect(common.PORT, spam).resume();
function spam() {
  while (true) {
    var res = this.write('GET / HTTP/1.1\r\n\r\n');
    if (++sent === COUNT)
      return;
    if (!res)
      break;
  }
  this.once('drain', spam);
}

This comment has been minimized.

@indutny

indutny Oct 13, 2015
Author Member

Did it simpler way, thanks for suggestion ;)

@trevnorris
Copy link
Contributor

@trevnorris trevnorris commented Oct 13, 2015

LGTM

@indutny
Copy link
Member Author

@indutny indutny commented Oct 13, 2015

Thanks @trevnorris . I'm working on fixing the test on Windows. For some reason it fails with EALREADY

this.outputCallbacks = [];
if (typeof this._onPendingData === 'function')
this._onPendingData(-this.outputSize);
this.outputSize = 0;

This comment has been minimized.

@trevnorris

trevnorris Oct 13, 2015
Contributor

This part is in OutgoingMessage#_writeRaw() but not in OutgoingMessage#_finish(). Is this already addressed in the git commit message? Don't see any direct reference to why this is necessary for the fix.

This comment has been minimized.

@indutny

indutny Oct 13, 2015
Author Member

You meant _flush?

This comment has been minimized.

@trevnorris

trevnorris Oct 13, 2015
Contributor

Ah oop. Yeah _flush.

This comment has been minimized.

@indutny

indutny Oct 13, 2015
Author Member

I think it is used there too.

@trevnorris
trevnorris reviewed Oct 13, 2015
View changes
lib/_http_server.js Outdated

function unconsume(parser, socket) {
if (socket._handle) {
parser.unconsume(socket._handle._externalStream);

This comment has been minimized.

@trevnorris

trevnorris Oct 13, 2015
Contributor

Is there any need to verify _externalStream exists on _handle?

This comment has been minimized.

@indutny

indutny Oct 13, 2015
Author Member

Actually, let's do it a bit better, and handle it in parser free.

var req = new Array(COUNT + 1).join('GET / HTTP/1.1\r\n\r\n');
client = net.connect(common.PORT, function() {
client.write(req);
});

This comment has been minimized.

@trevnorris

trevnorris Oct 13, 2015
Contributor

Won't having no 'data' event cause all the data to buffer? Don't think that's what you're trying to test, and that would accumulate to ~150MB. On our machines that's nothing big, but I'm thinking about execution time. It's not huge, but is longer than the rest:

ok 56 test-http-pipeline-regr-3332.js
  ---
  duration_ms: 42.108

This comment has been minimized.

@indutny

indutny Oct 13, 2015
Author Member

client.resume() below ;)

This comment has been minimized.

@trevnorris

trevnorris Oct 13, 2015
Contributor

That allows data to be read, but not removed from the internal queue:

$ /usr/bin/time ./node test/parallel/test-http-pipeline-regr-3332.js 
0.43user 0.15system 0:00.54elapsed 107%CPU (0avgtext+0avgdata 123572maxresident)k

~123MB memory usage for this test.

This comment has been minimized.

@trevnorris

trevnorris Oct 14, 2015
Contributor

You're right. Though this means net.connect() doesn't follow standard streaming behavior. But that's not for now.

@trevnorris
trevnorris reviewed Oct 13, 2015
View changes
test/parallel/test-http-pipeline-regr-3332.js Outdated
});

process.on('exit', function() {
assert.equal(received, COUNT);

This comment has been minimized.

@trevnorris

trevnorris Oct 13, 2015
Contributor

I know this isn't the only thing being tested. Mind adding a comment in code about the expected failure? It's not apparent looking at the code from this test.

This comment has been minimized.

@indutny

indutny Oct 13, 2015
Author Member

Makes sense. thank you!

@trevnorris
Copy link
Contributor

@trevnorris trevnorris commented Oct 13, 2015

Overall LGTM but have a few questions.

@@ -644,26 +629,11 @@ OutgoingMessage.prototype._finish = function() {
// to attempt to flush any pending messages out to the socket.
OutgoingMessage.prototype._flush = function() {

This comment has been minimized.

@jasnell
Copy link
Member

@jasnell jasnell commented Oct 14, 2015

@indutny .. thank you... LGTM but can you please go ahead and squash this down :-)

This is a two-part fix:

- Fix pending data notification in `OutgoingMessage` to notify server
  about flushed data too
- Fix pause/resume behavior for the consumed socket. `resume` event is
  emitted on a next tick, and `socket._paused` can already be `true` at
  this time. Pause the socket again to avoid PAUSED error on parser.

Fix: #3332
@indutny indutny force-pushed the indutny:fix/gh-3332 branch to cd9da62 Oct 14, 2015
@indutny
Copy link
Member Author

@indutny indutny commented Oct 14, 2015

@jasnell done, PTAL

@trevnorris
Copy link
Contributor

@trevnorris trevnorris commented Oct 14, 2015

Thanks for working w/ me on all the concerns. LGTM.

@indutny
Copy link
Member Author

@indutny indutny commented Oct 14, 2015

Thank you everyone! Landed in ab03635.

@indutny indutny closed this Oct 14, 2015
indutny added a commit that referenced this pull request Oct 14, 2015
This is a two-part fix:

- Fix pending data notification in `OutgoingMessage` to notify server
  about flushed data too
- Fix pause/resume behavior for the consumed socket. `resume` event is
  emitted on a next tick, and `socket._paused` can already be `true` at
  this time. Pause the socket again to avoid PAUSED error on parser.

Fix: #3332
PR-URL: #3342
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
@indutny indutny deleted the indutny:fix/gh-3332 branch Oct 14, 2015
@jasnell
Copy link
Member

@jasnell jasnell commented Oct 14, 2015

Once this goes out in either a release from master or v5.x I'll get it landed in v4.x

@rvagg rvagg mentioned this pull request Oct 21, 2015
indutny added a commit that referenced this pull request Oct 26, 2015
This is a two-part fix:

- Fix pending data notification in `OutgoingMessage` to notify server
  about flushed data too
- Fix pause/resume behavior for the consumed socket. `resume` event is
  emitted on a next tick, and `socket._paused` can already be `true` at
  this time. Pause the socket again to avoid PAUSED error on parser.

Fix: #3332
PR-URL: #3342
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
@jasnell
Copy link
Member

@jasnell jasnell commented Oct 26, 2015

Landed in v4.x-staging in 87f8ccf

@jasnell jasnell removed the lts-watch-v4.x label Oct 26, 2015
rvagg added a commit to rvagg/io.js that referenced this pull request Oct 28, 2015
Original commit ab03635:

  This is a two-part fix:

  - Fix pending data notification in `OutgoingMessage` to notify server
    about flushed data too
  - Fix pause/resume behavior for the consumed socket. `resume` event is
    emitted on a next tick, and `socket._paused` can already be `true` at
    this time. Pause the socket again to avoid PAUSED error on parser.

  Fix: nodejs#3332
  PR-URL: nodejs#3342
  Reviewed-By: James M Snell <jasnell@gmail.com>
  Reviewed-By: Trevor Norris <trev.norris@gmail.com>
rvagg added a commit to rvagg/io.js that referenced this pull request Oct 29, 2015
Notable changes:

* buffer: (Breaking) Removed both 'raw' and 'raws' encoding types from Buffer,
  these have been deprecated for a long time (Sakthipriyan Vairamani) nodejs#2859.
* console: (Breaking) Values reported by console.time() now have 3 decimals of
  accuracy added (Michaël Zasso) nodejs#3166.
* fs:
  - fs.readFile*(), fs.writeFile*(), and fs.appendFile*() now also accept a file
    descriptor as their first argument (Johannes Wüller) nodejs#3163.
  - (Breaking) In fs.readFile(), if an encoding is specified and the internal
    toString() fails the error is no longer thrown but is passed to the callback
    (Evan Lucas) nodejs#3485.
  - (Breaking) In fs.read() (using the fs.read(fd, length, position, encoding,
    callback) form), if the internal toString() fails the error is no longer
    thrown but is passed to the callback (Evan Lucas) nodejs#3503.
* http:
  - Fixed a bug where pipelined http requests would stall (Fedor Indutny) nodejs#3342.
  - (Breaking) When parsing HTTP, don't add duplicates of the following headers:
    Retry-After, ETag, Last-Modified, Server, Age, Expires. This is in addition
    to the following headers which already block duplicates: Content-Type,
    Content-Length, User-Agent, Referer, Host, Authorization,
    Proxy-Authorization, If-Modified-Since, If-Unmodified-Since, From, Location,
    Max-Forwards (James M Snell) nodejs#3090.
  - (Breaking) The callback argument to OutgoingMessage#setTimeout() must be a
    function or a TypeError is thrown (James M Snell) nodejs#3090.
  - (Breaking) HTTP methods and header names must now conform to the RFC 2616
    "token" rule, a list of allowed characters that excludes control characters
    and a number of separator characters. Specifically, methods and header names
    must now match /^[a-zA-Z0-9_!#$%&'*+.^`|~-]+$/ or a TypeError will be thrown
    (James M Snell) nodejs#2526.
* node:
  - (Breaking) Deprecated the _linklist module (Rich Trott) nodejs#3078.
  - (Breaking) Removed require.paths and require.registerExtension(), both had
    been previously set to throw Error when accessed
    (Sakthipriyan Vairamani) nodejs#2922.
* npm: Upgraded to version 3.3.6 from 2.14.7, see
  https://github.com/npm/npm/releases/tag/v3.3.6 for more details. This is a
  major version bump for npm and it has seen a significant amount of change.
  Please see the original npm v3.0.0 release notes for a list of major changes
  (Rebecca Turner) nodejs#3310.
* src: (Breaking) Bumped NODE_MODULE_VERSION to 47 from 46, this is necessary
  due to the V8 upgrade. Native add-ons will need to be recompiled
  (Rod Vagg) nodejs#3400.
* timers: Attempt to reuse the timer handle for setTimeout().unref(). This fixes
  a long-standing known issue where unrefed timers would perviously hold
  beforeExit open (Fedor Indutny) nodejs#3407.
* tls:
  - Added ALPN Support (Shigeki Ohtsu) nodejs#2564.
  - TLS options can now be passed in an object to createSecurePair()
    (Коренберг Марк) nodejs#2441.
  - (Breaking) The default minimum DH key size for tls.connect() is now 1024
    bits and a warning is shown when DH key size is less than 2048 bits. This a security consideration to prevent "logjam" attacks. A new minDHSize TLS
    option can be used to override the default. (Shigeki Ohtsu) nodejs#1831.
* util:
  - (Breaking) util.p() was deprecated for years, and has now been removed
    (Wyatt Preul) nodejs#3432.
  - (Breaking) util.inherits() can now work with ES6 classes. This is considered
    a breaking change because of potential subtle side-effects caused by a
    change from directly reassigning the prototype of the constructor using
    `ctor.prototype = Object.create(superCtor.prototype, { constructor: { ... } })`
    to using `Object.setPrototypeOf(ctor.prototype, superCtor.prototype)`
    (Michaël Zasso) nodejs#3455.
* v8: (Breaking) Upgraded to 4.6.85.25 from 4.5.103.35 (Ali Ijaz Sheikh) nodejs#3351.
  - Implements the spread operator, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_operator
    for further information.
  - Implements new.target, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new.target
    for further information.
* zlib: Decompression now throws on truncated input (e.g. unexpected end of
  file) (Yuval Brik) nodejs#2595.

PR-URL: nodejs#3466
indutny added a commit that referenced this pull request Oct 29, 2015
This is a two-part fix:

- Fix pending data notification in `OutgoingMessage` to notify server
  about flushed data too
- Fix pause/resume behavior for the consumed socket. `resume` event is
  emitted on a next tick, and `socket._paused` can already be `true` at
  this time. Pause the socket again to avoid PAUSED error on parser.

Fix: #3332
PR-URL: #3342
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
rvagg added a commit that referenced this pull request Oct 29, 2015
Notable changes:

* buffer: (Breaking) Removed both 'raw' and 'raws' encoding types from Buffer,
  these have been deprecated for a long time (Sakthipriyan Vairamani) #2859.
* console: (Breaking) Values reported by console.time() now have 3 decimals of
  accuracy added (Michaël Zasso) #3166.
* fs:
  - fs.readFile*(), fs.writeFile*(), and fs.appendFile*() now also accept a file
    descriptor as their first argument (Johannes Wüller) #3163.
  - (Breaking) In fs.readFile(), if an encoding is specified and the internal
    toString() fails the error is no longer thrown but is passed to the callback
    (Evan Lucas) #3485.
  - (Breaking) In fs.read() (using the fs.read(fd, length, position, encoding,
    callback) form), if the internal toString() fails the error is no longer
    thrown but is passed to the callback (Evan Lucas) #3503.
* http:
  - Fixed a bug where pipelined http requests would stall (Fedor Indutny) #3342.
  - (Breaking) When parsing HTTP, don't add duplicates of the following headers:
    Retry-After, ETag, Last-Modified, Server, Age, Expires. This is in addition
    to the following headers which already block duplicates: Content-Type,
    Content-Length, User-Agent, Referer, Host, Authorization,
    Proxy-Authorization, If-Modified-Since, If-Unmodified-Since, From, Location,
    Max-Forwards (James M Snell) #3090.
  - (Breaking) The callback argument to OutgoingMessage#setTimeout() must be a
    function or a TypeError is thrown (James M Snell) #3090.
  - (Breaking) HTTP methods and header names must now conform to the RFC 2616
    "token" rule, a list of allowed characters that excludes control characters
    and a number of separator characters. Specifically, methods and header names
    must now match /^[a-zA-Z0-9_!#$%&'*+.^`|~-]+$/ or a TypeError will be thrown
    (James M Snell) #2526.
* node:
  - (Breaking) Deprecated the _linklist module (Rich Trott) #3078.
  - (Breaking) Removed require.paths and require.registerExtension(), both had
    been previously set to throw Error when accessed
    (Sakthipriyan Vairamani) #2922.
* npm: Upgraded to version 3.3.6 from 2.14.7, see
  https://github.com/npm/npm/releases/tag/v3.3.6 for more details. This is a
  major version bump for npm and it has seen a significant amount of change.
  Please see the original npm v3.0.0 release notes for a list of major changes
  (Rebecca Turner) #3310.
* src: (Breaking) Bumped NODE_MODULE_VERSION to 47 from 46, this is necessary
  due to the V8 upgrade. Native add-ons will need to be recompiled
  (Rod Vagg) #3400.
* timers: Attempt to reuse the timer handle for setTimeout().unref(). This fixes
  a long-standing known issue where unrefed timers would perviously hold
  beforeExit open (Fedor Indutny) #3407.
* tls:
  - Added ALPN Support (Shigeki Ohtsu) #2564.
  - TLS options can now be passed in an object to createSecurePair()
    (Коренберг Марк) #2441.
  - (Breaking) The default minimum DH key size for tls.connect() is now 1024
    bits and a warning is shown when DH key size is less than 2048 bits. This a security consideration to prevent "logjam" attacks. A new minDHSize TLS
    option can be used to override the default. (Shigeki Ohtsu) #1831.
* util:
  - (Breaking) util.p() was deprecated for years, and has now been removed
    (Wyatt Preul) #3432.
  - (Breaking) util.inherits() can now work with ES6 classes. This is considered
    a breaking change because of potential subtle side-effects caused by a
    change from directly reassigning the prototype of the constructor using
    `ctor.prototype = Object.create(superCtor.prototype, { constructor: { ... } })`
    to using `Object.setPrototypeOf(ctor.prototype, superCtor.prototype)`
    (Michaël Zasso) #3455.
* v8: (Breaking) Upgraded to 4.6.85.25 from 4.5.103.35 (Ali Ijaz Sheikh) #3351.
  - Implements the spread operator, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_operator
    for further information.
  - Implements new.target, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new.target
    for further information.
* zlib: Decompression now throws on truncated input (e.g. unexpected end of
  file) (Yuval Brik) #2595.

PR-URL: #3466
@jasnell jasnell removed the land-on-v4.x label Nov 3, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

4 participants
You can’t perform that action at this time.