Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a bug where a connection is not closed immediatley when HTTP/2 PING write fails #2699

Merged
merged 1 commit into from May 12, 2020

Conversation

ikhoon
Copy link
Contributor

@ikhoon ikhoon commented May 11, 2020

Motivation:

If a connection is closed unexpectedly due to a network problem,
the remote peer does not receive closing signals such as GOAWAY frame and
the connection is alive until cleaned up due to request timeout, idle timeout, or PING failure.

When PING write fails, KeepAliveHandler tries to close the connection, which takes as long as the idle timeout.
Because the idle timeout is configured as gracefulShutdownTimeoutMillis for Netty Http2ConnectionHandler.

final long timeout = config.idleTimeoutMillis();
if (timeout > 0) {
gracefulShutdownTimeoutMillis(timeout);
} else {
// Timeout disabled
gracefulShutdownTimeoutMillis(-1);
}

Modifications:

  • Close a connection immediately if KeepAliveHandler is closing.

Result:
A dead connection is cleaned up immediately when PING write fails.

…G write fail

Motivation:

If a connection is closed unexpectedly by network shutdown,
the opposite does not receive close signals such as GOAWAY frame and
the connection is alive until cleaning up by request timeout, idle timeout, or PING fail.

When PING writes fail, KeepAliveHandler tries to close the connection, which takes for the idle timeout.
Because it is configured as `gracefulShutdownTimeoutMillis` for Netty Http2ConnectionHandler.
https://github.com/line/armeria/blob/fec9fb833275eba59eb9afaf25fc8b7c28132f80/core/src/main/java/com/linecorp/armeria/server/Http2ServerConnectionHandler.java#L56-L62

Modifications:

* Close a connection immediately if KeepAliveHandler is closing.

Result:
Clean up a dead connection immediately when PING write fails.
@ikhoon ikhoon added the defect label May 11, 2020
@ikhoon ikhoon added this to the 0.99.5 milestone May 11, 2020
@codecov
Copy link

codecov bot commented May 12, 2020

Codecov Report

Merging #2699 into master will decrease coverage by 0.00%.
The diff coverage is 71.42%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #2699      +/-   ##
============================================
- Coverage     72.68%   72.68%   -0.01%     
- Complexity    11312    11315       +3     
============================================
  Files          1001     1001              
  Lines         44297    44302       +5     
  Branches       5494     5498       +4     
============================================
+ Hits          32197    32200       +3     
+ Misses         9306     9304       -2     
- Partials       2794     2798       +4     
Impacted Files Coverage Δ Complexity Δ
...p/armeria/client/Http2ClientConnectionHandler.java 78.04% <66.66%> (+1.12%) 13.00 <3.00> (+1.00)
...p/armeria/server/Http2ServerConnectionHandler.java 79.48% <66.66%> (+1.10%) 14.00 <3.00> (+1.00)
...corp/armeria/internal/common/KeepAliveHandler.java 79.72% <100.00%> (-1.27%) 27.00 <2.00> (+2.00) ⬇️
...meria/common/stream/RegularFixedStreamMessage.java 83.01% <0.00%> (-5.67%) 13.00% <0.00%> (-1.00%)
.../armeria/client/endpoint/dns/DnsEndpointGroup.java 76.56% <0.00%> (-4.69%) 12.00% <0.00%> (-3.00%)
...java/com/linecorp/armeria/server/DefaultRoute.java 94.73% <0.00%> (-0.66%) 71.00% <0.00%> (-1.00%)
...inecorp/armeria/server/HttpResponseSubscriber.java 77.95% <0.00%> (-0.40%) 55.00% <0.00%> (ø%)
...armeria/server/healthcheck/HealthCheckService.java 84.21% <0.00%> (+0.43%) 53.00% <0.00%> (+1.00%)
...p/armeria/common/stream/AbstractStreamMessage.java 83.76% <0.00%> (+1.70%) 19.00% <0.00%> (ø%)
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a73a6d0...0b27381. Read the comment docs.

Copy link
Member

@minwoox minwoox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@trustin trustin changed the title Fix a bug where a connection is not close immediatley when HTTP/2 PING write fail Fix a bug where a connection is not closed immediatley when HTTP/2 PING write fails May 12, 2020
@trustin trustin merged commit 424fb98 into line:master May 12, 2020
@trustin
Copy link
Collaborator

trustin commented May 12, 2020

Good job, @ikhoon 😄

@ikhoon ikhoon deleted the ping-write-fail branch May 13, 2020 08:02
fmguerreiro pushed a commit to fmguerreiro/armeria that referenced this pull request Sep 19, 2020
…NG write fails (line#2699)

Motivation:

If a connection is closed unexpectedly due to a network problem,
the remote peer does not receive closing signals such as GOAWAY frame and
the connection is alive until cleaned up due to request timeout, idle timeout, or PING failure.

When PING write fails, KeepAliveHandler tries to close the connection, which takes as long as the idle timeout.
Because the idle timeout is configured as `gracefulShutdownTimeoutMillis` for Netty Http2ConnectionHandler.
https://github.com/line/armeria/blob/fec9fb833275eba59eb9afaf25fc8b7c28132f80/core/src/main/java/com/linecorp/armeria/server/Http2ServerConnectionHandler.java#L56-L62

Modifications:

* Close a connection immediately if KeepAliveHandler is closing.

Result:
A dead connection is cleaned up immediately when PING write fails.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants