net/http: race in http2Transport

[vadmeste]

What version of Go are you using (go version)?

go version go1.12.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/vadmeste/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/vadmeste/work/gospace"
GOPROXY=""
GORACE=""
GOROOT="/home/vadmeste/work/go"
GOTMPDIR=""
GOTOOLDIR="/home/vadmeste/work/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build457736797=/tmp/go-build -gno-record-gcc-switches"

What did you do?

1. Generate a self signed certificate:

$ openssl req -newkey rsa:2048 -nodes -keyout private.key -x509 -days 365 -out public.crt -subj "/C=US/ST=state/L=location/O=organization/CN=localhost"

2. Ask the OS to trust the generated self signed certificate

$ # Tested under Ubuntu
$ sudo cp public.crt /usr/local/share/ca-certificates/
$ sudo update-ca-certificates

3. Run the following test code: https://play.golang.org/p/mxV6_0TW-Ji

$ go run -race testcode.go

What did you expect to see?

No race detector warning

What did you see instead?

WARNING: DATA RACE
Write at 0x00c00180d1b8 by goroutine 17:
  main.main.func2()
      /home/vadmeste/work/go/src/bytes/buffer.go:102 +0x186

Previous write at 0x00c00180d1b8 by goroutine 203:
  bytes.(*Buffer).Read()
      /home/vadmeste/work/go/src/bytes/buffer.go:310 +0x266
  io/ioutil.(*nopCloser).Read()
      <autogenerated>:1 +0x87
  net/http.(*http2clientStream).writeRequestBody()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:7664 +0x64d
  net/http.(*http2Transport).getBodyWriterState.func1()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:8878 +0x13d

Goroutine 17 (running) created at:
  main.main()
      /home/vadmeste/x/testcode.go:48 +0x277

Goroutine 203 (running) created at:
  net/http.http2bodyWriterState.scheduleBodyWrite()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:8925 +0x12f
  net/http.(*http2ClientConn).roundTrip()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:7465 +0x90f
  net/http.(*http2Transport).RoundTripOpt()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:6867 +0x23d
  net/http.http2noDialH2RoundTripper.RoundTrip()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:6829 +0x4c
  net/http.(*Transport).roundTrip()
      /home/vadmeste/work/go/src/net/http/transport.go:430 +0x14d9
  net/http.(*Transport).RoundTrip()
      /home/vadmeste/work/go/src/net/http/roundtrip.go:17 +0x42
  net/http.send()
      /home/vadmeste/work/go/src/net/http/client.go:250 +0x6a9
  net/http.(*Client).send()
      /home/vadmeste/work/go/src/net/http/client.go:174 +0x1cb
  net/http.(*Client).do()
      /home/vadmeste/work/go/src/net/http/client.go:641 +0x4ef
  main.main.func2()
      /home/vadmeste/work/go/src/net/http/client.go:509 +0x4e0

[dgryski]

@bradfitz @tombergan

[fraenkel]

@bradfitz This might be the same issue with http. There is no guarantee that the bodyWriter is complete when roundtrip exits.

[odeke-em]

Here is a standalone adapted from @vadmeste's original program but here one doesn't have to manually create certificates but also it is reduced down

package main

import (
	"bytes"
	"fmt"
	"io"
	"io/ioutil"
	"log"
	"net/http"
	"net/http/httptest"

	"golang.org/x/net/http2"
)

func main() {
	cst := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "Pong")
	}))
	if err := http2.ConfigureServer(cst.Config, new(http2.Server)); err != nil {
		log.Fatalf("Failed to configure http2 server: %v", err)
	}
	cst.TLS = cst.Config.TLSConfig

	tr := &http.Transport{TLSClientConfig: cst.Config.TLSConfig}
	if err := http2.ConfigureTransport(tr); err != nil {
		log.Fatalf("Failed to configure http2 transport: %v", err)
	}
	tr.TLSClientConfig.InsecureSkipVerify = true
	cst.StartTLS()
	defer cst.Close()

	// Then finally create the HTTP client with the HTTP/2 transport.
	client := &http.Client{Transport: tr}

	for i := 0; i < 3; i++ {
		// Fill buf with data
		buf := bytes.NewBuffer(bytes.Repeat([]byte("A"), 128*1024))

		// Make a POST http call and read and discard response
		req, err := http.NewRequest("POST", cst.URL, buf)
		resp, err := client.Do(req)
		if err != nil {
			log.Fatalln(err)
		}
                if _, err := io.Copy(ioutil.Discard, resp.Body); err != nil {
			log.Fatalln(err)
		}
		resp.Body.Close()

		// Accessing buf here causes a RACE WARNING
		buf.Reset()
		fmt.Printf(".")
	}
}