Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: client closes persistent connections when context deadline is exceeded without reuse #45559

Open
akhilesh-godi opened this issue Apr 14, 2021 · 3 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@akhilesh-godi
Copy link

akhilesh-godi commented Apr 14, 2021

What version of Go are you using (go version)?

$ go version
1.14.4

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/godia/Library/Caches/go-build"
GOENV="/Users/godia/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/godia/dev"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/r0/ggt8ftqn467cckf8r_y58vqr0000gp/T/go-build603942499=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Ran into a scenario while using the http client, when there is a surge in the number of requests and in certain requests when context deadline is exceeded at the client side.

What did you expect to see?

Persistent connection remains intact when not explicitly closed due to request context timeouts.

What did you see instead?

Persistent connections should be reused. But a lot of new dials happen due to closing of persistent connections which could have been potentially avoided.

Relevant code in question: https://github.com/golang/go/blob/master/src/net/http/transport.go#L2219

@mknyszek mknyszek changed the title HTTP client closes persistent connections when context deadline is exceeded without reuse net/http: client closes persistent connections when context deadline is exceeded without reuse Apr 14, 2021
@mknyszek mknyszek added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 14, 2021
@mknyszek mknyszek added this to the Backlog milestone Apr 14, 2021
@mknyszek
Copy link
Contributor

mknyszek commented Apr 14, 2021

@networkimprov
Copy link

networkimprov commented May 15, 2021

@akhilesh-godi have you tried this in Go 1.16 or 1.15? Support for 1.14 has ended.

cc @neild @fraenkel

@fraenkel
Copy link
Contributor

fraenkel commented May 16, 2021

This is the correct behavior.
Lets say you sent a request and the deadline hits. The response is never read, now what do you do?
Unless the request/response is completed, it is more correct to just throw out the connection.

If you have a specific case in mind where it can safely be used, you need to provide an example so we can examine further.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

4 participants