Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: race in http2Transport #31192

Open
vadmeste opened this issue Apr 1, 2019 · 3 comments
Milestone

Comments

@vadmeste
Copy link

@vadmeste vadmeste commented Apr 1, 2019

What version of Go are you using (go version)?

go version go1.12.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/vadmeste/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/vadmeste/work/gospace"
GOPROXY=""
GORACE=""
GOROOT="/home/vadmeste/work/go"
GOTMPDIR=""
GOTOOLDIR="/home/vadmeste/work/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build457736797=/tmp/go-build -gno-record-gcc-switches"

What did you do?

  1. Generate a self signed certificate:
$ openssl req -newkey rsa:2048 -nodes -keyout private.key -x509 -days 365 -out public.crt -subj "/C=US/ST=state/L=location/O=organization/CN=localhost"
  1. Ask the OS to trust the generated self signed certificate
$ # Tested under Ubuntu
$ sudo cp public.crt /usr/local/share/ca-certificates/
$ sudo update-ca-certificates
  1. Run the following test code: https://play.golang.org/p/mxV6_0TW-Ji
$ go run -race testcode.go

What did you expect to see?

No race detector warning

What did you see instead?

WARNING: DATA RACE
Write at 0x00c00180d1b8 by goroutine 17:
  main.main.func2()
      /home/vadmeste/work/go/src/bytes/buffer.go:102 +0x186

Previous write at 0x00c00180d1b8 by goroutine 203:
  bytes.(*Buffer).Read()
      /home/vadmeste/work/go/src/bytes/buffer.go:310 +0x266
  io/ioutil.(*nopCloser).Read()
      <autogenerated>:1 +0x87
  net/http.(*http2clientStream).writeRequestBody()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:7664 +0x64d
  net/http.(*http2Transport).getBodyWriterState.func1()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:8878 +0x13d

Goroutine 17 (running) created at:
  main.main()
      /home/vadmeste/x/testcode.go:48 +0x277

Goroutine 203 (running) created at:
  net/http.http2bodyWriterState.scheduleBodyWrite()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:8925 +0x12f
  net/http.(*http2ClientConn).roundTrip()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:7465 +0x90f
  net/http.(*http2Transport).RoundTripOpt()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:6867 +0x23d
  net/http.http2noDialH2RoundTripper.RoundTrip()
      /home/vadmeste/work/go/src/net/http/h2_bundle.go:6829 +0x4c
  net/http.(*Transport).roundTrip()
      /home/vadmeste/work/go/src/net/http/transport.go:430 +0x14d9
  net/http.(*Transport).RoundTrip()
      /home/vadmeste/work/go/src/net/http/roundtrip.go:17 +0x42
  net/http.send()
      /home/vadmeste/work/go/src/net/http/client.go:250 +0x6a9
  net/http.(*Client).send()
      /home/vadmeste/work/go/src/net/http/client.go:174 +0x1cb
  net/http.(*Client).do()
      /home/vadmeste/work/go/src/net/http/client.go:641 +0x4ef
  main.main.func2()
      /home/vadmeste/work/go/src/net/http/client.go:509 +0x4e0
@dgryski

This comment has been minimized.

Copy link
Contributor

@dgryski dgryski commented Apr 1, 2019

@bradfitz bradfitz changed the title x/net/http2 client call causing a race detector warning x/net/http2: client call causing a race detector warning Apr 2, 2019
vadmeste added a commit to vadmeste/minio that referenced this issue Apr 2, 2019
A race is detected between a bytes.Buffer generated with cmd/rpc.Pool
and http2 module. An issue is raised in golang (golang/go#31192).

Meanwhile, this commit disables Pool in RPC code and it generates a
new 1kb of bytes.Buffer for each RPC call.
kannappanr added a commit to minio/minio that referenced this issue Apr 2, 2019
A race is detected between a bytes.Buffer generated with cmd/rpc.Pool
and http2 module. An issue is raised in golang (golang/go#31192).

Meanwhile, this commit disables Pool in RPC code and it generates a
new 1kb of bytes.Buffer for each RPC call.
@andybons andybons added this to the Go1.13 milestone Apr 3, 2019
@bcmills bcmills changed the title x/net/http2: client call causing a race detector warning net/http: race in http2Transport Apr 9, 2019
@fraenkel

This comment has been minimized.

Copy link
Contributor

@fraenkel fraenkel commented Apr 10, 2019

@bradfitz This might be the same issue with http. There is no guarantee that the bodyWriter is complete when roundtrip exits.

@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@odeke-em

This comment has been minimized.

Copy link
Member

@odeke-em odeke-em commented Oct 12, 2019

Here is a standalone adapted from @vadmeste's original program but here one doesn't have to manually create certificates but also it is reduced down

package main

import (
	"bytes"
	"fmt"
	"io"
	"io/ioutil"
	"log"
	"net/http"
	"net/http/httptest"

	"golang.org/x/net/http2"
)

func main() {
	cst := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "Pong")
	}))
	if err := http2.ConfigureServer(cst.Config, new(http2.Server)); err != nil {
		log.Fatalf("Failed to configure http2 server: %v", err)
	}
	cst.TLS = cst.Config.TLSConfig

	tr := &http.Transport{TLSClientConfig: cst.Config.TLSConfig}
	if err := http2.ConfigureTransport(tr); err != nil {
		log.Fatalf("Failed to configure http2 transport: %v", err)
	}
	tr.TLSClientConfig.InsecureSkipVerify = true
	cst.StartTLS()
	defer cst.Close()

	// Then finally create the HTTP client with the HTTP/2 transport.
	client := &http.Client{Transport: tr}

	for i := 0; i < 3; i++ {
		// Fill buf with data
		buf := bytes.NewBuffer(bytes.Repeat([]byte("A"), 128*1024))

		// Make a POST http call and read and discard response
		req, err := http.NewRequest("POST", cst.URL, buf)
		resp, err := client.Do(req)
		if err != nil {
			log.Fatalln(err)
		}
                if _, err := io.Copy(ioutil.Discard, resp.Body); err != nil {
			log.Fatalln(err)
		}
		resp.Body.Close()

		// Accessing buf here causes a RACE WARNING
		buf.Reset()
		fmt.Printf(".")
	}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.