Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: TestTransportMaxConnsPerHost still flaky #31982

Open
cuonglm opened this issue May 11, 2019 · 13 comments

Comments

Projects
None yet
7 participants
@cuonglm
Copy link
Contributor

commented May 11, 2019

What version of Go are you using (go version)?

$ go version
go version devel +83f205fa88 Sun May 12 02:55:19 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/cuonglm/.cache/go-build"
GOENV="/home/cuonglm/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/cuonglm/go"
GOPROXY="direct"
GOROOT="/home/cuonglm/sources/go"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/home/cuonglm/sources/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build766721718=/tmp/go-build -gno-record-gcc-switches"

What did you do?

From git root repo
$ cd src/net/http
$ go test -run TestTransportMaxConnsPerHost -count=100 -race

What did you expect to see?

Test passed

What did you see instead?

--- FAIL: TestTransportMaxConnsPerHost (0.07s)
    transport_test.go:661: Too many get connections (http2): 2
    transport_test.go:676: Too many get connections (http2): 3
--- FAIL: TestTransportMaxConnsPerHost (0.03s)
    transport_test.go:661: Too many get connections (http2): 2
    transport_test.go:676: Too many get connections (http2): 3
--- FAIL: TestTransportMaxConnsPerHost (0.08s)
    transport_test.go:661: Too many get connections (http2): 2
    transport_test.go:676: Too many get connections (http2): 3
--- FAIL: TestTransportMaxConnsPerHost (0.06s)
    transport_test.go:661: Too many get connections (http2): 2
    transport_test.go:676: Too many get connections (http2): 3
--- FAIL: TestTransportMaxConnsPerHost (0.03s)
    transport_test.go:661: Too many get connections (http2): 2
    transport_test.go:676: Too many get connections (http2): 3
FAIL
exit status 1
FAIL	net/http	4.433s
@fraenkel

This comment has been minimized.

Copy link
Contributor

commented May 11, 2019

This is a bit interesting because we get the right number of dials and TLS handshakes but the connection returned is supposed not marked reused.

@fraenkel

This comment has been minimized.

Copy link
Contributor

commented May 11, 2019

The problem is that the GotConn for http2 (h2_bundle:6878) is called before we know whether we are creating a new stream (h2_bundle: 7449) which is what determines reuse.

This is not a flaky test but a real bug.

@fraenkel

This comment has been minimized.

Copy link
Contributor

commented May 11, 2019

Using the nextStreamID is flawed anyhow. If you allow HTTP, we start at 3 which means its immediately reused.

@fraenkel

This comment has been minimized.

Copy link
Contributor

commented May 11, 2019

If we move the GotConn call into getClientConn(h2_bundle:749), we can distinguish between a new connection and a used connection.

@cuonglm cuonglm changed the title net/http: TestTransportMaxConnsPerHost still flakes net/http: TestTransportMaxConnsPerHost still flaky May 12, 2019

@anakin

This comment has been minimized.

Copy link

commented May 12, 2019

➜ http go test -run TestTransportMaxConnsPerHost -count=100 -race
PASS
ok net/http 1.641s

@anakin

This comment has been minimized.

Copy link

commented May 12, 2019

by the way,
➜ http go version
go version go1.12.5 darwin/amd64

@gopherbot

This comment has been minimized.

Copy link

commented May 12, 2019

Change https://golang.org/cl/176720 mentions this issue: http2: track reused connections

@fraenkel

This comment has been minimized.

Copy link
Contributor

commented May 12, 2019

I submitted a fix which does highlight the issue with http2. I did some other testing and found some of the other ClientTrace calls like TLSHandshakreXXX are missing.

@andybons andybons added this to the Unplanned milestone May 13, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented May 14, 2019

The darwin-amd64-race builder caught this too:
https://build.golang.org/log/06b60fdfdf4a88e04ae4181d80ddd39db9aaafa0

@bcmills

This comment has been minimized.

Copy link
Member

commented May 14, 2019

@bcmills bcmills modified the milestones: Unplanned, Go1.12.6, Go1.13 May 14, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented May 14, 2019

gopherbot pushed a commit to golang/net that referenced this issue May 14, 2019

http2: track reused connections
nextStreamID was used as a means to determine if the connection was
being reused. Multiple requests can see a new connection because the
nextStreamID is updated after a ClientTrace reports it is being reused.

Updates golang/go#31982

Change-Id: Iaa4b62b217f015423cddb99fd86de75a352f8320
Reviewed-on: https://go-review.googlesource.com/c/net/+/176720
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@gopherbot

This comment has been minimized.

Copy link

commented May 14, 2019

Change https://golang.org/cl/177037 mentions this issue: net/http: update vendored, bundled x/net/http2

@bradfitz

This comment has been minimized.

Copy link
Member

commented May 16, 2019

Whoops, I'd forgotten to submit 177037. Done. Hopefully it's better now.

gopherbot pushed a commit that referenced this issue May 16, 2019

net/http: update vendored, bundled x/net/http2
For:

    http2: track reused connections
    https://golang.org/cl/176720 (updates #31982)

Some x/sys/unix updates come along for the ride too.

I filed #32031 for making the bundling process less difficult and
error-prone in the future.

Change-Id: Ic822080991ffa2d50352c5f613e45648a327cf16
Reviewed-on: https://go-review.googlesource.com/c/go/+/177037
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.