net/http: http.Transport leaks context from the request

[juliens]

What version of Go are you using (go version)?

$ go version
go version go1.17 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/juliens/.cache/go-build"
GOENV="/home/juliens/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/juliens/dev/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/juliens/dev/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/juliens/.gvm/gos/go1.17"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/juliens/.gvm/gos/go1.17/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3799944390=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I use the http.Transport to make an HTTP request. I add a not empty struct in the request context.

You can reproduce it like this

What did you expect to see?

After the end of the request, the struct and the context are cleaned by the GC.

What did you see instead?

After the end of the request, the struct and the context are not cleaned by the GC.

Maybe this is the same issue as #43966

[gopherbot]

Change https://golang.org/cl/380674 mentions this issue: net/http: fix memory leak in http.Transport

[mknyszek]

CC @neild

[netrebel]

We seem to be having the same problem. In long running instances, memory utilization is going up every day reaching 80-90% in ~12 days. We started seeing this issue when we upgraded from Go 1.11.3 (no modules) to 1.17 (with go-modules), before the Memory would stay under 30%. Servers have 4GBs.

When I run pprof I see something similar to the comment in the ticket referenced:

      flat  flat%   sum%        cum   cum%
    1.50MB  5.07%  5.07%    12.04MB 40.70%  net/http.(*Transport).dialConn
         0     0%  5.07%    12.04MB 40.70%  net/http.(*Transport).dialConnFor
    8.03MB 27.14% 32.21%     8.03MB 27.14%  bufio.NewWriterSize (inline)
    6.02MB 20.36% 52.57%     6.02MB 20.36%  bufio.NewReaderSize (inline)