Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: fatal error: MHeap_AllocLocked - MSpan not free #17412

Closed
brknstrngz opened this issue Oct 12, 2016 · 4 comments

Comments

Projects
None yet
3 participants
@brknstrngz
Copy link

commented Oct 12, 2016

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.7.1 freebsd/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64" GOBIN="" GOEXE="" GOHOSTARCH="amd64" GOHOSTOS="freebsd" GOOS="freebsd" GOPATH="/root/work/go" GORACE="" GOROOT="/usr/local/go" GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64" CC="cc" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -gno-record-gcc-switches" CXX="clang++" CGO_ENABLED="1"
The OS is FreeBSD 11.0-RELEASE.

What did you do?

Sadly I cannot share the code, but it is a very simplistic HTTPI API client that calls multiple API endpoints in parallel. Responses are read into buffers obtained from a sync.Pool for later decoding.

What did you expect to see?

The API client run to completion.

What did you see instead?

fatal error: MHeap_AllocLocked - MSpan not free is thrown by the runtime. Not sure this is related to #15658.

gocrash.txt

@brknstrngz brknstrngz changed the title fatal error: MHeap_AllocLocked - MSpan not free runtime: fatal error: MHeap_AllocLocked - MSpan not free Oct 12, 2016

@bradfitz

This comment has been minimized.

Copy link
Member

commented Oct 12, 2016

Does your program have any data races?

@brknstrngz

This comment has been minimized.

Copy link
Author

commented Oct 12, 2016

As it turns out, it does have a race. It is strange where it happens, though. We use singleflight to serialize access to our API login call.

WARNING: DATA RACE
Write at 0x00c4201ee270 by goroutine 243:
main.(_HTTPApiClient).doAuthenticate()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:360 +0xe34
main.(_HTTPApiClient).(main.doAuthenticate)-fm()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:303 +0x45
globalsign/hvpki/vendor/github.com/golang/groupcache/singleflight.(_Group).Do()
/path/to/go/src/globalsign/hvpki/vendor/github.com/golang/groupcache/singleflight/singleflight.go:56 +0x291
main.(_HTTPApiClient).authenticate()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:303 +0xa3
main.(*HTTPApiClient).requestResource()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:414 +0xdc1
main.runHTTPApiClient.func1()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:266 +0x4d

Previous read at 0x00c4201ee270 by goroutine 70:
runtime.convT2E()
/usr/local/go/src/runtime/iface.go:155 +0x0
main.(*HTTPApiClient).requestResource()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:382 +0x490
main.runHTTPApiClient.func1()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:266 +0x4d

Goroutine 243 (running) created at:
main.runHTTPApiClient()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:271 +0x358
main.main()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:114 +0x61f

Goroutine 70 (running) created at:
main.runHTTPApiClient()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:271 +0x358
main.main()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:114 +0x61f

@brknstrngz

This comment has been minimized.

Copy link
Author

commented Oct 12, 2016

I think I know what is happening, will report back shortly.

@brknstrngz

This comment has been minimized.

Copy link
Author

commented Oct 12, 2016

You were spot on Brad, we did have an unguarded variable access in the singleflight callback. It had flown under the radar for a year and a half or so and only manifested itself when we doubled our goroutine count. Sorry for the noise.

@brknstrngz brknstrngz closed this Oct 12, 2016

@golang golang locked and limited conversation to collaborators Oct 12, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.