Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP/2: Prevent memory leak when trying to create new streams on a connection that received a GOAWAY. #9674

Merged
merged 3 commits into from Oct 17, 2019

Conversation

@millems
Copy link
Contributor

millems commented Oct 15, 2019

Motivation:

In #8692, Http2FrameCodec was updated to keep track of all "being initialized" streams, allocating memory before initialization begins, and releasing memory after initialization completes successfully.

In some instances where stream initialization fails (e.g. because this connection has received a GOAWAY frame), this memory is never released.

Modifications:

This change updates the Http2FrameCodec to use a separate promise for monitoring the success of sending HTTP2 headers. When sending of headers fails, we now make sure to release memory allocated for stream initialization.

Result:

After this change, failures in writing HTTP2 Headers (e.g. because this connection has received a GOAWAY frame) will no longer leak memory.

…nnection that received a GOAWAY.

Motivation:

In #8692, `Http2FrameCodec` was
updated to keep track of all "being initialized" streams, allocating
memory before initialization begins, and releasing memory after
initialization completes successfully.

In some instances where stream initialization fails (e.g. because this
connection has received a GOAWAY frame), this memory is never released.

Modifications:

This change updates the `Http2FrameCodec` to use a separate promise
for monitoring the success of sending HTTP2 headers. When sending of
headers fails, we now make sure to release memory allocated for stream
initialization.

Result:

After this change, failures in writing HTTP2 Headers (e.g. because this
connection has received a GOAWAY frame) will no longer leak memory.
@netty-bot

This comment has been minimized.

Copy link

netty-bot commented Oct 15, 2019

Can one of the admins verify this patch?

@normanmaurer

This comment has been minimized.

Copy link
Member

normanmaurer commented Oct 16, 2019

@millems can you please also sign our icla: https://netty.io/s/icla

@normanmaurer

This comment has been minimized.

Copy link
Member

normanmaurer commented Oct 16, 2019

@netty-bot test this please

@njhill

This comment has been minimized.

Copy link
Member

njhill commented Oct 16, 2019

@millems probably I'm missing something but why is a separate promise needed? Couldn't we just add a listener to the existing promise?

@millems

This comment has been minimized.

Copy link
Contributor Author

millems commented Oct 16, 2019

@njhill Using a separate promise was intended to be defensive, and only performing the cleanup if the send-headers failed.

I was concerned about the following scenario:

  1. sendHeaders succeeds
  2. Something fails the original promise for an unrelated issue
  3. The frameStreamToInitializeMap entry is removed because of that failure,
  4. The onStreamAdded is still invoked, because the sendHeaders suceeded and the stream was created
  5. The onStreamAdded doesn't initialize the stream correctly because the original promise was failed.

Let me know if this concern is unfounded, and I can remove the additional future.

@millems

This comment has been minimized.

Copy link
Contributor Author

millems commented Oct 16, 2019

@normanmaurer Will review comments and address. CLA was reviewed and signed yesterday. Let me know if it's not on record and I can go again.

@normanmaurer

This comment has been minimized.

Copy link
Member

normanmaurer commented Oct 16, 2019

@millems I think it will be good enough to just attach to the "original" promise as @njhill suggested

@normanmaurer

This comment has been minimized.

Copy link
Member

normanmaurer commented Oct 16, 2019

@millems all good... I somehow missed it 🤦‍♂

1. Assert channel.finishAndReleaseAll() returns false.
2. Attach failure detection to original future, instead of a separate future.
3. Use ChannelListener instead of GenericFutureListener
4. Note that numInitializingStreams is for testing.
@millems

This comment has been minimized.

Copy link
Contributor Author

millems commented Oct 16, 2019

Addressed change requests.

@normanmaurer

This comment has been minimized.

Copy link
Member

normanmaurer commented Oct 16, 2019

@netty-bot test this please

@normanmaurer

This comment has been minimized.

Copy link
Member

normanmaurer commented Oct 16, 2019

@netty-bot test this please

@normanmaurer normanmaurer added this to the 4.1.43.Final milestone Oct 17, 2019
Copy link
Member

normanmaurer left a comment

@millems thanks a lot for your work... :)

@normanmaurer normanmaurer merged commit bbc34d0 into netty:4.1 Oct 17, 2019
3 checks passed
3 checks passed
pull request validation (centos6-java11) Build finished.
Details
pull request validation (centos6-java12) Build finished.
Details
pull request validation (centos6-java8) Build finished.
Details
normanmaurer added a commit that referenced this pull request Oct 17, 2019
…nnection that received a GOAWAY. (#9674)

Motivation:

In #8692, `Http2FrameCodec` was
updated to keep track of all "being initialized" streams, allocating
memory before initialization begins, and releasing memory after
initialization completes successfully.

In some instances where stream initialization fails (e.g. because this
connection has received a GOAWAY frame), this memory is never released.

Modifications:

This change updates the `Http2FrameCodec` to use a separate promise
for monitoring the success of sending HTTP2 headers. When sending of
headers fails, we now make sure to release memory allocated for stream
initialization.

Result:

After this change, failures in writing HTTP2 Headers (e.g. because this
connection has received a GOAWAY frame) will no longer leak memory.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.