Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: net/http: Allow configurable max connection age #43905

Open
dmarkhas opened this issue Jan 25, 2021 · 4 comments
Open

proposal: net/http: Allow configurable max connection age #43905

dmarkhas opened this issue Jan 25, 2021 · 4 comments
Labels
Projects
Milestone

Comments

@dmarkhas
Copy link

@dmarkhas dmarkhas commented Jan 25, 2021

Currently, the http.Transport implementation will reuse an HTTP connection if keepalives are enabled and the connection has not been in an idle state for too long.
This means that for high throughput connections that are always active and rarely/never exceed the IdleConnTimeout limit, the connection will keep getting reused (as long as it is not broken).

This poses a problem in terms of service discovery and client-side load balancing - since address resolution only happens in the dialer's DialContext, this leaves no chance for the client to re-resolve the target address since the same connection is constantly reused without ever "re-dialing".

I propose to add a MaxConnectionAge flag to http.Transport that would limit the lifetime of a persistent connection, and would not reuse it beyond this lifetime.

This would be in line with the behavior of go-grpc which also enforces a MaxConnectionAge limit.

@seankhliao seankhliao changed the title net/http: Allow configurable max connection age proposal: net/http: Allow configurable max connection age Jan 25, 2021
@gopherbot gopherbot added this to the Proposal milestone Jan 25, 2021
@networkimprov
Copy link

@networkimprov networkimprov commented Feb 4, 2021

@RobGraham
Copy link

@RobGraham RobGraham commented Feb 11, 2021

Similar to #23427 which offers a work around (hack). However re-resolving DNS and reusing connection issues have been apparent for many years.

@szuecs
Copy link

@szuecs szuecs commented Feb 15, 2021

Maybe the TTL based DNS resolution could be a solution #23427 (comment).
#23427 (comment) summarized 3 possible solutions.

I am pretty sure that in practice you never max out the number of connections, so you always have idle conns that will be closed in a "hack" as I showed in #23427. We run this in a high traffic scenario since some years without an issue that we could observe (does not mean that it does not exist in practice).

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals Feb 17, 2021
@raidancampbell
Copy link

@raidancampbell raidancampbell commented Jul 29, 2021

I have a different issue that would be resolved by the same configuration. For reasons outside my control, I have a firewall that kills TCP connections after 15 minutes regardless of their activity. As mentioned above MaxConnectionAge solves this server-side in gRPC.

Having a similar configuration would be an easy client-side fix for me in HTTP, but I'll readily admit this problem is not likely common enough to warrant a change to net/http.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Proposals
Incoming
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants