Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: client closes persistent connections when context deadline is exceeded without reuse #45559

Open
akhilesh-godi opened this issue Apr 14, 2021 · 3 comments

Comments

@akhilesh-godi
Copy link

@akhilesh-godi akhilesh-godi commented Apr 14, 2021

What version of Go are you using (go version)?

$ go version
1.14.4

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/godia/Library/Caches/go-build"
GOENV="/Users/godia/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/godia/dev"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/r0/ggt8ftqn467cckf8r_y58vqr0000gp/T/go-build603942499=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Ran into a scenario while using the http client, when there is a surge in the number of requests and in certain requests when context deadline is exceeded at the client side.

What did you expect to see?

Persistent connection remains intact when not explicitly closed due to request context timeouts.

What did you see instead?

Persistent connections should be reused. But a lot of new dials happen due to closing of persistent connections which could have been potentially avoided.

Relevant code in question: https://github.com/golang/go/blob/master/src/net/http/transport.go#L2219

@mknyszek mknyszek changed the title HTTP client closes persistent connections when context deadline is exceeded without reuse net/http: client closes persistent connections when context deadline is exceeded without reuse Apr 14, 2021
@mknyszek mknyszek added this to the Backlog milestone Apr 14, 2021
@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Apr 14, 2021

Loading

@networkimprov
Copy link

@networkimprov networkimprov commented May 15, 2021

@akhilesh-godi have you tried this in Go 1.16 or 1.15? Support for 1.14 has ended.

cc @neild @fraenkel

Loading

@fraenkel
Copy link
Contributor

@fraenkel fraenkel commented May 16, 2021

This is the correct behavior.
Lets say you sent a request and the deadline hits. The response is never read, now what do you do?
Unless the request/response is completed, it is more correct to just throw out the connection.

If you have a specific case in mind where it can safely be used, you need to provide an example so we can examine further.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants