net/http: document that it's illegal to reuse a Request concurrently from multiple goroutines #21780
Description
Incorrect errRequestCanceled errors occur when reusing http.Request structs.
There is a small race in reusing Transport's reqCanceler map, where a finishing request can override an active request dial's canceler. I suspect this is what #19653 was running into, and Brad hinted at this.
The problem:
The code centers around transport.go's RoundTrip block of code here, L400-L413.
For two requests a and b that execute back to back:
a sees a body's EOF: L1622-L1624,
b begins, enters RoundTrip, tries to getConn, no idle conns exist, b sets the request's canceler (L940, important), and a large select happens: L948-L986
a's persistConn running gets notified of the bodyEOF, sets its transport's request canceler for this request to nil (L1656) overriding the canceler that was just set by B, and goes to tryPutIdleConn (assuming other conditions are successful): L1655-L1661
a''s tryPutIdleConn sees a waitingDialer: L704-L706
b's dial select receives the persistConn that just ran a: L976-L986
b's RoundTrip getConn returns successfully and goes into pconn.roundTrip (L400-L413, the main block)
roundTrip tries to replace the transports request canceler for b with the persistConn's cancelRequest: L1889-L1891
replaceReqCancel fails because the prior replacement deleted the request from the transport's reqCanceler map, and if the request does not exist in the map, the canceler for it cannot be replaced: L856-L865
This causes roundTrip to return errRequestCanceled: L1893
Also note that this is only the http1 stack and is harder to notice against URLs that have redirects.
What version of Go are you using (go version)?
1.8.3, 1.9, tip
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/twmb/go"
GORACE=""
GOROOT="/home/twmb/go/go"
GOTOOLDIR="/home/twmb/go/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build465265467=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
What did you do?
https://play.golang.org/p/BP7YCP2D4J
go test -bench . -benchtime 10s (should happen quickly)
What did you expect to see?
No panic.
What did you see instead?
A panic.