Description
What version of Go are you using (go version
)?
go 1.11 and 1.14
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env
)?
go env
Output
$ go env set GO111MODULE= set GOARCH=amd64 set GOBIN= set GOCACHE=C:\Users\USER\AppData\Local\go-build set GOENV=C:\Users\USER\AppData\Roaming\go\env set GOEXE=.exe set GOFLAGS= set GOHOSTARCH=amd64 set GOHOSTOS=windows set GOINSECURE= set GONOPROXY= set GONOSUMDB= set GOOS=windows set GOPATH=C:\go set GOPRIVATE= set GOPROXY=https://proxy.golang.org,direct set GOROOT=c:\go set GOSUMDB=sum.golang.org set GOTMPDIR= set GOTOOLDIR=c:\go\pkg\tool\windows_amd64 set GCCGO=gccgo set AR=ar set CC=gcc set CXX=g++ set CGO_ENABLED=1 set GOMOD= set CGO_CFLAGS=-g -O2 set CGO_CPPFLAGS= set CGO_CXXFLAGS=-g -O2 set CGO_FFLAGS=-g -O2 set CGO_LDFLAGS=-g -O2 set PKG_CONFIG=pkg-config set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\USER\AppData\Local\Temp\go-build702297486=/tmp/go-build -gno-record-gcc-switches
What did you do?
Wrote a web crawler using net/http to crawl large networks with lots of different web services of all ages and technologies. The crawler mostly works great, but ocasionally I'm running into nil pointer dereference panics. I cannot reproduce the issue. Looking at the stacktrace it seems to be burried within net/http and independent of my code.
2020-02-29 18:52:11.810 +0100 Critical Scan scope crashed unexpectedly: runtime error: invalid memory address or nil pointer dereference
Stacktrace:
| goroutine 4239417 [running]:
| runtime/debug.Stack(0xa1f, 0xc001186d00, 0x1d)
| C:/Go/src/runtime/debug/stack.go:24 +0xa4
| my-project/log.StacktraceIndented(0xbc8784, 0x1, 0x52, 0x0)
| D:/go/src/my-project/log/logger.go:347 +0x3b
| my-project/scans/webenum.(*Scanner).Run.func1(0xc000481440, 0xc002e0fdb0)
| D:/go/src/my-project/scans/webenum/webenum.go:175 +0x10f
| panic(0xb2a700, 0xa88be0)
| C:/Go/src/runtime/panic.go:967 +0x16b
| net/http.send(0xc000317500, 0xcc32e0, 0xc00107e600, 0xbf8ec712e19c8a1c, 0x67b0b803359d, 0x10b9800, 0xc0005fe868, 0xbf8ec712e19c8a1c, 0x1, 0x0)
| C:/Go/src/net/http/client.go:271 +0x503
| net/http.(*Client).send(0xc0002dabd0, 0xc000317500, 0xbf8ec712e19c8a1c, 0x67b0b803359d, 0x10b9800, 0xc0005fe868, 0x0, 0x1, 0xc0013f0710)
| C:/Go/src/net/http/client.go:176 +0x101
| net/http.(*Client).do(0xc0002dabd0, 0xc000317500, 0x0, 0x0, 0x0)
| C:/Go/src/net/http/client.go:699 +0x451
| net/http.(*Client).Do(...)
| C:/Go/src/net/http/client.go:567
|......
C:/Go/src/net/http/client.go:271
is where the actual panic araises due to a nil pointer dereference, by accessing .Body
on the response of resp, err = rt.RoundTrip(req)
. Obviously, for some reason, resp
is nil, due to some unhanled error. So, execution continues until it ends up with a panic calling resp.Body
. Here is a copy of the source code of client.go lines 252-275 for quick reference:
252: resp, err = rt.RoundTrip(req)
253: if err != nil {
254: stopTimer()
255: if resp != nil {
256: log.Printf("RoundTripper returned a response & error; ignoring response")
257: }
258: if tlsErr, ok := err.(tls.RecordHeaderError); ok {
259: // If we get a bad TLS record header, check to see if the
260: // response looks like HTTP and give a more helpful error.
261: // See golang.org/issue/11111.
262: if string(tlsErr.RecordHeader[:]) == "HTTP/" {
263: err = errors.New("http: server gave HTTP response to HTTPS client")
264: }
265: }
266: return nil, didTimeout, err
267: }
268: if !deadline.IsZero() {
269: resp.Body = &cancelTimerBody{
270: stop: stopTimer,
271: rc: resp.Body,
272: reqDidTimeout: didTimeout,
273: }
274: }
275: return resp, nil, nil
Maybe some package expert has an idea, why rt.RoundTrip(req)
might return nil without error? Obviously, this edge case should not happen, because resp
is required down the road...
What did you expect to see?
I'm expecting anything but panics. Either an error or a normal response.
What did you see instead?
A panic