x/net/http2: RoundTrip hangs when the response status code > 299

[chungthuang]

What version of Go are you using (go version)?

$ go version
go version go1.15.5 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/cf000197/Library/Caches/go-build"
GOENV="/Users/cf000197/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/cf000197/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/cf000197/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.15.5/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.15.5/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/cf000197/go/issues/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/t1/fvz_vb0n6g9fpqwmdcl2yfbh0000gn/T/go-build998981477=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I followed #13444 to create bidirectional streams. It works great when the server returns 200, but on the client side we are seeing roundtrip hanging when the response status > 299, after upgrading to latest http2. The commit that introduced the issue is golang/net@ff519b6. Even though abortRequestBodyWrite is called when response status > 299, Read in https://github.com/golang/net/blame/master/http2/transport.go#L1333 is blocked indefinitely.
Here is a program to reproduce the issue.

package main

import (
	"fmt"
	"io"
	"io/ioutil"
	"log"
	"net"
	"net/http"
	"os"
	"time"

	"golang.org/x/net/http2"
)

func main() {
	pr, pw := io.Pipe()

	// Host doesn't matter in this case because we already have a connection to the server
	req, err := http.NewRequest(http.MethodPut, "https://http2.golang.org", ioutil.NopCloser(pr))
	if err != nil {
		log.Fatal(err)
	}

	clientConn, serverConn := net.Pipe()
	if err != nil {
		log.Fatal(err)
	}
	go func() {
		server := http2.Server{}
		server.ServeConn(serverConn, &http2.ServeConnOpts{
			Handler: http.HandlerFunc(errorEndpoint),
		})
	}()
	go func() {
		transport := http2.Transport{}
		http2ClientConn, err := transport.NewClientConn(clientConn)
		res, err := http2ClientConn.RoundTrip(req)
		if err != nil {
			log.Fatal(err)
		}
		log.Printf("Got: %#v", res)
		go streamRequest(pw)
		n, err := io.Copy(os.Stdout, res.Body)
		log.Fatalf("copied %d, %v", n, err)
	}()
	select {}
}

func streamRequest(pw io.Writer) {
	for {
		time.Sleep(1 * time.Second)
		fmt.Fprintf(pw, "It is now %v\n", time.Now())
	}
}

func errorEndpoint(w http.ResponseWriter, r *http.Request) {
	w.WriteHeader(http.StatusInternalServerError)
	w.Write([]byte(http.StatusText(http.StatusInternalServerError)))
}

Go module that causes the roundtrip to hang:

module main

go 1.15

require (
	golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 // indirect
	golang.org/x/net v0.0.0-20210119194325-5f4716e94777 // indirect
	golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 // indirect
	golang.org/x/text v0.3.3 // indirect
)

Go module that was working:

module main

go 1.15

require (
	golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 // indirect
	golang.org/x/net v0.0.0-20200904194848-62affa334b73 // indirect
	golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 // indirect
	golang.org/x/text v0.3.3 // indirect
)

What did you expect to see?

Roundtrip returns with a 500 response.

What did you see instead?

Roundtrip hangs indefinitely

[fraenkel]

I believe this is a bug in your code.
You create a roundtrip with a body made from a pipe. You never actually start sending any data on it so the RoundTrip is stuck waiting on a read from the pipe.

[chungthuang]

Hi @fraenkel, my client retries only if roundtrip returns an error. To avoid implementing a rewindable reader, I intentionally wait for roundtrip to return before sending data. This was working in golang.org/x/net v0.0.0-20200904194848.

[ipostelnik]

The RoundTrip code relies on being able to read the body to proceed in case of a non-200 status, this is not a reasonable expectation. It's possible that client will wait to see some of the response before writing the body.