Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http2: fix session memory accounting after pausing #30684

Conversation

@mikelehen
Copy link
Contributor

mikelehen commented Nov 27, 2019

The ability to pause input processing was added in 8a4a193 but
introduced a session memory accounting mismatch leading to potential
NGHTTP2_ENHANCE_YOUR_CALM errors.

After pausing
(

// Mark the remainder of the data as available for later consumption.
),
the early return on line 873 skips the
DecrementCurrentSessionMemory(stream_buf_.len) call below (line 878).

When we later finished processing the input chunk
(

if (UNLIKELY(stream_buf_offset_ > 0)) {
),
we were calling DecrementCurrentSessionMemory(stream_buf_offset_) [line
1875] which was a no-op since we just set stream_buf_offset_ to 0 [line
1873].

The correct amount to decrement by is still stream_buf_.len, since
that's the amount we skipped previously (line 878).

Fixes: #29223
Refs: 164ac5b

Checklist
  • make -j4 test (UNIX)
  • tests and/or benchmarks are included
  • commit message follows commit guidelines
The ability to pause input processing was added in 8a4a193 but
introduced a session memory accounting mismatch leading to potential
NGHTTP2_ENHANCE_YOUR_CALM errors.

After pausing
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L871),
the early return on line 873 skips the
DecrementCurrentSessionMemory(stream_buf_.len) call below (line 878).

When we later finished processing the input chunk
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L1858),
we were calling DecrementCurrentSessionMemory(stream_buf_offset_) [line
1875] which was a no-op since we just set stream_buf_offset_ to 0 [line
1873].

The correct amount to decrement by is still stream_buf_.len, since
that's the amount we skipped previously (line 878).

Fixes: #29223
Refs: 164ac5b
@jasnell jasnell requested a review from addaleax Nov 27, 2019
Copy link
Member

addaleax left a comment

LGTM

I’m good with landing this and then we can rebase #30351 against if if we want to?

@lundibundi

This comment has been minimized.

Copy link
Member

lundibundi commented Nov 29, 2019

@addaleax yeah, let's land this first, I'll then update mine.

@addaleax

This comment has been minimized.

Copy link
Member

addaleax commented Nov 29, 2019

@lundibundi Just to be clear, this PR LGTY? It could land pretty soon then :)

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

addaleax added a commit that referenced this pull request Nov 30, 2019
The ability to pause input processing was added in 8a4a193 but
introduced a session memory accounting mismatch leading to potential
NGHTTP2_ENHANCE_YOUR_CALM errors.

After pausing
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L871),
the early return on line 873 skips the
DecrementCurrentSessionMemory(stream_buf_.len) call below (line 878).

When we later finished processing the input chunk
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L1858),
we were calling DecrementCurrentSessionMemory(stream_buf_offset_) [line
1875] which was a no-op since we just set stream_buf_offset_ to 0 [line
1873].

The correct amount to decrement by is still stream_buf_.len, since
that's the amount we skipped previously (line 878).

Fixes: #29223
Refs: 164ac5b

PR-URL: #30684
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@addaleax

This comment has been minimized.

Copy link
Member

addaleax commented Nov 30, 2019

Landed in 18a1796, thanks for the PR!

@addaleax addaleax closed this Nov 30, 2019
addaleax added a commit that referenced this pull request Nov 30, 2019
The ability to pause input processing was added in 8a4a193 but
introduced a session memory accounting mismatch leading to potential
NGHTTP2_ENHANCE_YOUR_CALM errors.

After pausing
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L871),
the early return on line 873 skips the
DecrementCurrentSessionMemory(stream_buf_.len) call below (line 878).

When we later finished processing the input chunk
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L1858),
we were calling DecrementCurrentSessionMemory(stream_buf_offset_) [line
1875] which was a no-op since we just set stream_buf_offset_ to 0 [line
1873].

The correct amount to decrement by is still stream_buf_.len, since
that's the amount we skipped previously (line 878).

Fixes: #29223
Refs: 164ac5b

PR-URL: #30684
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@mikelehen

This comment has been minimized.

Copy link
Contributor Author

mikelehen commented Nov 30, 2019

Awesome! Thanks for the quick reviewing and merging!

@mikelehen

This comment has been minimized.

Copy link
Contributor Author

mikelehen commented Dec 2, 2019

As a minor follow-up, since the original bug affects Node.js 10.x (c152449), I think this fix needs to be back-ported as well. Is that an automatic thing? Or is there some way to "request" the back-port (or submit a PR that will help get it into 10.x)? Thanks!

@MylesBorins

This comment has been minimized.

Copy link
Member

MylesBorins commented Dec 5, 2019

@MylesBorins

This comment has been minimized.

Copy link
Member

MylesBorins commented Dec 5, 2019

FWIW this lands cleanly on both v10.x and v12.x

@BethGriggs could we get this in the next 10.x and 12.x? Considering this is a pretty nasty bug it might make sense to delay both those releases by a week

thoughts?

/cc @nodejs/lts

@addaleax

This comment has been minimized.

Copy link
Member

addaleax commented Dec 5, 2019

I don’t think we need to delay releases for a low-risk patch like this.

targos added a commit that referenced this pull request Dec 5, 2019
The ability to pause input processing was added in 8a4a193 but
introduced a session memory accounting mismatch leading to potential
NGHTTP2_ENHANCE_YOUR_CALM errors.

After pausing
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L871),
the early return on line 873 skips the
DecrementCurrentSessionMemory(stream_buf_.len) call below (line 878).

When we later finished processing the input chunk
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L1858),
we were calling DecrementCurrentSessionMemory(stream_buf_offset_) [line
1875] which was a no-op since we just set stream_buf_offset_ to 0 [line
1873].

The correct amount to decrement by is still stream_buf_.len, since
that's the amount we skipped previously (line 878).

Fixes: #29223
Refs: 164ac5b

PR-URL: #30684
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@MylesBorins

This comment has been minimized.

Copy link
Member

MylesBorins commented Dec 5, 2019

@addaleax are you suggesting we get this in the next release or that we should land without waiting 2 weeks?

@MylesBorins MylesBorins mentioned this pull request Dec 5, 2019
@addaleax

This comment has been minimized.

Copy link
Member

addaleax commented Dec 5, 2019

@MylesBorins I’m suggesting including it in the upcoming releases, without any kind of waiting time; this is a very straightforward bug fix, and I’d treat it like that.

BethGriggs added a commit that referenced this pull request Dec 6, 2019
The ability to pause input processing was added in 8a4a193 but
introduced a session memory accounting mismatch leading to potential
NGHTTP2_ENHANCE_YOUR_CALM errors.

After pausing
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L871),
the early return on line 873 skips the
DecrementCurrentSessionMemory(stream_buf_.len) call below (line 878).

When we later finished processing the input chunk
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L1858),
we were calling DecrementCurrentSessionMemory(stream_buf_offset_) [line
1875] which was a no-op since we just set stream_buf_offset_ to 0 [line
1873].

The correct amount to decrement by is still stream_buf_.len, since
that's the amount we skipped previously (line 878).

Fixes: #29223
Refs: 164ac5b

PR-URL: #30684
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@BethGriggs BethGriggs mentioned this pull request Dec 9, 2019
Sebastien-Ahkrin added a commit to Sebastien-Ahkrin/node that referenced this pull request Dec 11, 2019
The ability to pause input processing was added in 8a4a193 but
introduced a session memory accounting mismatch leading to potential
NGHTTP2_ENHANCE_YOUR_CALM errors.

After pausing
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L871),
the early return on line 873 skips the
DecrementCurrentSessionMemory(stream_buf_.len) call below (line 878).

When we later finished processing the input chunk
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L1858),
we were calling DecrementCurrentSessionMemory(stream_buf_offset_) [line
1875] which was a no-op since we just set stream_buf_offset_ to 0 [line
1873].

The correct amount to decrement by is still stream_buf_.len, since
that's the amount we skipped previously (line 878).

Fixes: nodejs#29223
Refs: nodejs@164ac5b

PR-URL: nodejs#30684
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
MylesBorins added a commit that referenced this pull request Dec 17, 2019
The ability to pause input processing was added in 8a4a193 but
introduced a session memory accounting mismatch leading to potential
NGHTTP2_ENHANCE_YOUR_CALM errors.

After pausing
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L871),
the early return on line 873 skips the
DecrementCurrentSessionMemory(stream_buf_.len) call below (line 878).

When we later finished processing the input chunk
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L1858),
we were calling DecrementCurrentSessionMemory(stream_buf_offset_) [line
1875] which was a no-op since we just set stream_buf_offset_ to 0 [line
1873].

The correct amount to decrement by is still stream_buf_.len, since
that's the amount we skipped previously (line 878).

Fixes: #29223
Refs: 164ac5b

PR-URL: #30684
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
MylesBorins added a commit that referenced this pull request Dec 17, 2019
The ability to pause input processing was added in 8a4a193 but
introduced a session memory accounting mismatch leading to potential
NGHTTP2_ENHANCE_YOUR_CALM errors.

After pausing
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L871),
the early return on line 873 skips the
DecrementCurrentSessionMemory(stream_buf_.len) call below (line 878).

When we later finished processing the input chunk
(https://github.com/nodejs/node/blob/f36331c1bfa4c4c202346b05dc3bd672f653e4df/src/node_http2.cc#L1858),
we were calling DecrementCurrentSessionMemory(stream_buf_offset_) [line
1875] which was a no-op since we just set stream_buf_offset_ to 0 [line
1873].

The correct amount to decrement by is still stream_buf_.len, since
that's the amount we skipped previously (line 878).

Fixes: #29223
Refs: 164ac5b

PR-URL: #30684
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@BethGriggs BethGriggs mentioned this pull request Dec 23, 2019
BethGriggs added a commit that referenced this pull request Jan 7, 2020
- http2: fix session memory accounting after pausing (Michael Lehenbauer)
  [#30684](#30684)
- n-api: correct bug in napi_get_last_error (Octavian Soldea)
  [#28702](#28702)

PR-URL:
@BethGriggs BethGriggs mentioned this pull request Jan 7, 2020
BethGriggs added a commit that referenced this pull request Jan 7, 2020
- http2: fix session memory accounting after pausing (Michael Lehenbauer)
  [#30684](#30684)
- n-api: correct bug in napi_get_last_error (Octavian Soldea)
  [#28702](#28702)

PR-URL: #31248
BethGriggs added a commit that referenced this pull request Jan 8, 2020
- http2: fix session memory accounting after pausing (Michael Lehenbauer)
  [#30684](#30684)
- n-api: correct bug in napi_get_last_error (Octavian Soldea)
  [#28702](#28702)

PR-URL: #31248
BethGriggs added a commit that referenced this pull request Jan 8, 2020
- http2: fix session memory accounting after pausing (Michael Lehenbauer)
  [#30684](#30684)
- n-api: correct bug in napi_get_last_error (Octavian Soldea)
  [#28702](#28702)
- tools: update tzdata to 2019c (Myles Borins)
  [#30479](#30479)

PR-URL: #31248
BethGriggs added a commit that referenced this pull request Jan 9, 2020
- http2: fix session memory accounting after pausing (Michael Lehenbauer)
  [#30684](#30684)
- n-api: correct bug in napi_get_last_error (Octavian Soldea)
  [#28702](#28702)
- tools: update tzdata to 2019c (Myles Borins)
  [#30479](#30479)

PR-URL: #31248
BethGriggs added a commit that referenced this pull request Jan 9, 2020
- http2: fix session memory accounting after pausing (Michael Lehenbauer)
  [#30684](#30684)
- n-api: correct bug in napi_get_last_error (Octavian Soldea)
  [#28702](#28702)
- tools: update tzdata to 2019c (Myles Borins)
  [#30479](#30479)

PR-URL: #31248
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.