Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify behavior after max_connection_time timeout #34384

Closed
ktalg opened this issue May 28, 2024 · 5 comments · Fixed by #34565
Closed

Clarify behavior after max_connection_time timeout #34384

ktalg opened this issue May 28, 2024 · 5 comments · Fixed by #34565
Labels
area/http question Questions that are neither investigations, bugs, nor enhancements stale stalebot believes this issue/PR has not been touched recently

Comments

@ktalg
Copy link
Contributor

ktalg commented May 28, 2024

Title: One line description
when max_connection_time and drain_timeout reached,the http2 connection wouldn't be closed if there's any active stream, which does not match what doc says

Description:

according to:
https://www.envoyproxy.io/docs/envoy/v1.19.5/api-v3/config/core/v3/protocol.proto#config-core-v3-httpprotocoloptions

When max_connection_duration is reached the connection will be closed. Drain sequence will occur prior to closing the connection if if’s applicable.

But recently I ran a test and looked at the client trace logs:

2024-05-28T07:10:57.597366Z     debug   envoy http2     [C107] updating connection-level initial window size to 268435456
2024-05-28T07:10:57.597367Z     debug   envoy client    [C107] connecting
2024-05-28T07:10:57.597369Z     debug   envoy connection        [C107] connecting to 172.16.29.45:50051
...
2024-05-28T07:10:58.601368Z     trace   envoy http2     [C107] about to recv frame type=7, flags=0, stream_id=0
2024-05-28T07:10:58.601371Z     trace   envoy http2     [C107] track inbound frame type=7 flags=0 length=8 padding_length=0
2024-05-28T07:10:58.601374Z     trace   envoy http2     [C107] recv frame type=7
2024-05-28T07:10:58.601377Z     debug   envoy pool      [C107] remote goaway
2024-05-28T07:10:58.601383Z     trace   envoy http2     [C107] dispatched 17 bytes
2024-05-28T07:11:03.602011Z     trace   envoy connection        [C107] socket event: 3
2024-05-28T07:11:03.602027Z     trace   envoy connection        [C107] write ready
2024-05-28T07:11:03.602032Z     trace   envoy connection        [C107] read ready. dispatch_buffered_data=false
2024-05-28T07:11:03.602041Z     trace   envoy connection        [C107] read returns: 17
2024-05-28T07:11:03.602049Z     trace   envoy connection        [C107] read error: Resource temporarily unavailable
2024-05-28T07:11:03.602056Z     trace   envoy http2     [C107] dispatching 17 bytes
2024-05-28T07:11:03.602061Z     trace   envoy http2     [C107] about to recv frame type=7, flags=0, stream_id=0
2024-05-28T07:11:03.602063Z     trace   envoy http2     [C107] track inbound frame type=7 flags=0 length=8 padding_length=0
2024-05-28T07:11:03.602065Z     trace   envoy http2     [C107] recv frame type=7
...
2024-05-28T07:11:27.626089Z     trace   envoy http2     [C107] dispatched 27 bytes
2024-05-28T07:11:33.836496Z     debug   envoy client    [C107] request reset
2024-05-28T07:11:33.836504Z     debug   envoy pool      [C107] destroying stream: 0 remaining
2024-05-28T07:11:33.836511Z     debug   envoy connection        [C107] closing data_to_write=0 type=1
2024-05-28T07:11:33.836512Z     debug   envoy connection        [C107] closing socket: 1
2024-05-28T07:11:33.836544Z     trace   envoy connection        [C107] raising connection event 1
2024-05-28T07:11:33.836569Z     debug   envoy client    [C107] disconnect. resetting 0 pending requests
2024-05-28T07:11:33.836575Z     debug   envoy pool      [C107] client disconnected, failure reason:

After the second goaway, the server does not actively close the connection (stream exists) until the client closes (30 seconds later).
Is this behavior correct? Or maybe I missed something.

@ktalg ktalg added the triage Issue requires triage label May 28, 2024
@zuercher
Copy link
Member

The docs you linke are quite old, v1.19.5. The current docs say, for a downstream connection, that if there are active streams the drain sequence is initiated. (https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/protocol.proto#config-core-v3-httpprotocoloptions)

@zuercher zuercher added question Questions that are neither investigations, bugs, nor enhancements area/http and removed triage Issue requires triage labels May 28, 2024
@ktalg
Copy link
Contributor Author

ktalg commented May 31, 2024

The docs you linke are quite old, v1.19.5. The current docs say, for a downstream connection, that if there are active streams the drain sequence is initiated. (https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/protocol.proto#config-core-v3-httpprotocoloptions)

Thank you. The latest version of the documentation state:

max_connection_duration:

If the connection is a downstream connection and there are any active streams, the drain sequence will kick in, and the connection will be force-closed after the drain period.

and

drain_timeout:

The time that Envoy will wait between sending an HTTP/2 “shutdown notification” (GOAWAY frame with max stream ID) and a final GOAWAY frame.
Draining occurs both when a connection hits the idle timeout or during general server draining.

This still does not resolve my confusion. If a TCP connection has continuously active streams, after both max_connection_duration and drain_timeout have expired, will the stream be be force-closed, or will Envoy send two GOAWAY frames and then wait for certain conditions (hits the idle timeout or during general server draining) before closing the connection?

@phlax
Copy link
Member

phlax commented May 31, 2024

@ktalg if you were able to raise a PR to clarify the docs i would be happy to review

Copy link

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label Jun 30, 2024
Copy link

github-actions bot commented Jul 7, 2024

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jul 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/http question Questions that are neither investigations, bugs, nor enhancements stale stalebot believes this issue/PR has not been touched recently
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants