net/http: http.Client.Do() sometimes throws INTERNAL_ERROR when using high number of parallel HTTP/2 requests (regression between 1.12 and 1.13.4)

[rayvbr]

What version of Go are you using (go version)?

go version go1.13.4 darwin/amd64

(also reproduced with 1.13.6)

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GO111MODULE="off"
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/k7/jx2bs5q170z4lvdzbsdt426r0000gn/T/go-build634112063=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Create several http.Clients, and call a high number of simultaneous multipart byterange requests on each of them. Make sure the server supports HTTP/2.

What did you expect to see?

No errors being thrown. Which is the case on Go 1.12

What did you see instead?

On Go 1.13.4 and 1.13.6, http.Do() sometimes returns an error of the form stream error: stream ID 523; INTERNAL_ERROR. It does seem to happen more often when there is some network congestion, but I've also observed it on very good networks. Typically, I'd say it happens once every 200 requests or so.

[cagedmantis]

/cc @bradfitz

[networkimprov]

cc @fraenkel

[fraenkel]

We are going to need a test case or at the very least some http/2 debug to determine why the INTERNAL_ERROR occurred.

Does this problem also occur with tip?

[bcmills]

I wonder if this is the root cause of #51323 (INTERNAL_ERROR failures observed on the build dashboard)..?