Description
Incorrect errRequestCanceled
errors occur when reusing http.Request
structs.
There is a small race in reusing Transport
's reqCanceler
map, where a finishing request can override an active request dial's canceler. I suspect this is what #19653 was running into, and Brad hinted at this.
The problem:
The code centers around transport.go's RoundTrip
block of code here, L400-L413.
For two requests a and b that execute back to back:
a sees a body's EOF: L1622-L1624,
b begins, enters RoundTrip
, tries to getConn
, no idle conns exist, b sets the request's canceler (L940, important), and a large select happens: L948-L986
a's persistConn
running gets notified of the bodyEOF
, sets its transport's request canceler for this request to nil (L1656) overriding the canceler that was just set by B, and goes to tryPutIdleConn
(assuming other conditions are successful): L1655-L1661
a''s tryPutIdleConn
sees a waitingDialer
: L704-L706
b's dial select receives the persistConn
that just ran a: L976-L986
b's RoundTrip
getConn
returns successfully and goes into pconn.roundTrip
(L400-L413, the main block)
roundTrip
tries to replace the transports request canceler for b with the persistConn
's cancelRequest
: L1889-L1891
replaceReqCancel
fails because the prior replacement deleted the request from the transport's reqCanceler
map, and if the request does not exist in the map, the canceler for it cannot be replaced: L856-L865
This causes roundTrip
to return errRequestCanceled
: L1893
Also note that this is only the http1 stack and is harder to notice against URLs that have redirects.
What version of Go are you using (go version
)?
1.8.3, 1.9, tip
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env
)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/twmb/go"
GORACE=""
GOROOT="/home/twmb/go/go"
GOTOOLDIR="/home/twmb/go/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build465265467=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
What did you do?
https://play.golang.org/p/BP7YCP2D4J
go test -bench . -benchtime 10s
(should happen quickly)
What did you expect to see?
No panic.
What did you see instead?
A panic.