net/http: data race in closing and reading from expectContinueReader ()

[Ewg-sha]

What version of Go are you using (go version)?

Go version: 1.10.3

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"

What did you do?

I need to cancel receiving of request body in http server's handle. I use background go routine which blocks on context's channel and closes request body once the channel is closed: request.Body.Close() (go/src/net/http/server.go:878 +0x40)

What did you expect to see?

Receiving of the request stops, no race.

What did you see instead?

==================
WARNING: DATA RACE
Write at 0x00c42000c1d8 by goroutine 53:
  net/http.(*expectContinueReader).Close()
      /local/home/bond/tools/go/src/net/http/server.go:878 +0x40
  prj/test/transfer.(*httpTransfer).Receive.func1()
    /wks/bond/prj/test/src/prj/test/transfer/server.go:193 +0x61
  prj/test/transfer.(*pendingRequest).CancelReceive()
    /wks/bond/prj/test/src/prj/test/transfer/server.go:372 +0x8d
  prj/test/transfer.(*serverTestSuite).TestReceiveAndCancel.func3()
    /wks/bond/prj/test/src/prj/test/transfer/transfer_test.go:232 +0x46
  github.com/stretchr/testify/assert.didPanic.func1()
      /home/bond/build/gopath/src/github.com/stretchr/testify/assert/assertions.go:791 +0x68
  github.com/stretchr/testify/assert.didPanic()
      /home/bond/build/gopath/src/github.com/stretchr/testify/assert/assertions.go:793 +0x5d
  github.com/stretchr/testify/assert.NotPanics()
      /home/bond/build/gopath/src/github.com/stretchr/testify/assert/assertions.go:833 +0x45
  prj/test/transfer.(*serverTestSuite).TestReceiveAndCancel()
    /wks/bond/prj/test/src/prj/test/transfer/transfer_test.go:232 +0x531
  runtime.call32()
      /local/home/bond/tools/go/src/runtime/asm_amd64.s:573 +0x3a
  reflect.Value.Call()
      /local/home/bond/tools/go/src/reflect/value.go:308 +0xc0
  github.com/stretchr/testify/suite.Run.func2()
      /home/bond/build/gopath/src/github.com/stretchr/testify/suite/suite.go:102 +0x2ff
  testing.tRunner()
      /local/home/bond/tools/go/src/testing/testing.go:777 +0x16d

Previous read at 0x00c42000c1d8 by goroutine 76:
  net/http.(*expectContinueReader).Read()
      /local/home/bond/tools/go/src/net/http/server.go:862 +0x40
  mime/multipart.(*stickyErrorReader).Read()
      /local/home/bond/tools/go/src/mime/multipart/multipart.go:124 +0x97
  bufio.(*Reader).fill()
      /local/home/bond/tools/go/src/bufio/bufio.go:100 +0x19f
  bufio.(*Reader).Peek()
      /local/home/bond/tools/go/src/bufio/bufio.go:132 +0x50
  mime/multipart.partReader.Read()
      /local/home/bond/tools/go/src/mime/multipart/multipart.go:177 +0x358
  mime/multipart.(*Part).Read()
      /local/home/bond/tools/go/src/mime/multipart/multipart.go:157 +0x75
  io.copyBuffer()
      /local/home/bond/tools/go/src/io/io.go:400 +0x172
  io.Copy()
      /local/home/bond/tools/go/src/io/io.go:362 +0x74
  prj/test/transfer.(*pendingRequest).Receive.func1()
    /wks/bond/prj/test/src/prj/test/transfer/server.go:350 +0x93
==================

[meirf]

I'm going to try to come up with code that matches your description, but it would really speed things up if you can submit standalone runnable code.

A complete runnable program is good.
A link on play.golang.org is best.

thanks!

[Ewg-sha]

expectContinueReader struct has a race between methods Close and Read when accessing unprotected boolean "closed"

func (ecr *expectContinueReader) Close() error {
	ecr.closed = true
	return ecr.readCloser.Close()
}

and

func (ecr *expectContinueReader) Read(p []byte) (n int, err error) {
	if ecr.closed {
		return 0, ErrBodyReadAfterClose
	}
        ...
}

Is this race expected? Is there any safe way to abort http transfer on server side in net/http package?

[meirf]

Thanks for the code. (I now see how obvious this was from your first comment.)

Can easily recreate this race by inserting

go func() {
    r.Body.Close()
}()

into TestServerExpect's httptest.NewServer (in server_test.go) and then running go test -race -run=TestServerExpect

I suppose an obvious band aid is to use a mutex. If sync.Mutex locking without contention implementation is fast (e.g. futex) then sync.Mutex should be fine.

I use background go routine which blocks on context's channel and closes request body once the channel is closed

Can you elaborate more on what this context is? Is it per request or supposed to live as long as the server?

Is there any safe way to abort http transfer on server side in net/http package?

I assume you've seen Server.ReadTimeout, Server.ReadHeaderTimeout and Server.Close(), but these aren't enough. Hopefully your answer to the question above will give us a better understanding.