runtime/pprof: `found bad pointer in Go heap` during TestGoroutineProfileConcurrency with TestMorestack

[bcmills]

runtime: pointer 0x616e6966 to unallocated span span.base()=0x8f78000 span.limit=0x8f79ff8 span.state=0
runtime: found in object at *(0x8d3a420+0x0)
object=0x8d3a420 s.base()=0x8d3a000 s.limit=0x8d3c000 s.spanclass=5 s.elemsize=16 s.state=mSpanInUse
 *(object+0) = 0x616e6966 <==
 *(object+4) = 0x657a696c
 *(object+8) = 0x8114ed7
 *(object+12) = 0x0
fatal error: found bad pointer in Go heap (incorrect use of unsafe or cgo?)

runtime stack:
runtime.throw({0x81d8da0, 0x3e})
	/workdir/go/src/runtime/panic.go:1047 +0x4d fp=0x8c53e2c sp=0x8c53e18 pc=0x807fbdd
runtime.badPointer(0xef804360, 0x616e6966, 0x8d3a420, 0x0)
	/workdir/go/src/runtime/mbitmap.go:316 +0xa2 fp=0x8c53e4c sp=0x8c53e2c pc=0x8058f12
runtime.findObject(0x616e6966, 0x8d3a420, 0x0)
	/workdir/go/src/runtime/mbitmap.go:359 +0x9e fp=0x8c53e64 sp=0x8c53e4c pc=0x805916e
runtime.scanobject(0x8d3a420, 0x8c2895c)
	/workdir/go/src/runtime/mgcmark.go:1341 +0x183 fp=0x8c53eb4 sp=0x8c53e64 pc=0x8065613
runtime.markrootSpans(0x8c2895c, 0x0)
	/workdir/go/src/runtime/mgcmark.go:390 +0x18e fp=0x8c53ef0 sp=0x8c53eb4 pc=0x80630fe
runtime.markroot(0x8c2895c, 0x4, 0x1)
	/workdir/go/src/runtime/mgcmark.go:192 +0xe9 fp=0x8c53f4c sp=0x8c53ef0 pc=0x80628c9
runtime.gcDrain(0x8c2895c, 0x3)
	/workdir/go/src/runtime/mgcmark.go:1069 +0x4ed fp=0x8c53fb0 sp=0x8c53f4c pc=0x8064ffd
runtime.gcBgMarkWorker.func2()
	/workdir/go/src/runtime/mgc.go:1307 +0xaa fp=0x8c53fdc sp=0x8c53fb0 pc=0x806060a
runtime.systemstack()
	/workdir/go/src/runtime/asm_386.s:370 +0x41 fp=0x8c53fe0 sp=0x8c53fdc pc=0x80af4e1

…

goroutine 72 [wait for GC cycle]:
runtime.gopark(0x81dccd8, 0x82ea18c, 0x17, 0x14, 0x1)
	/workdir/go/src/runtime/proc.go:363 +0x108 fp=0x8ca0f10 sp=0x8ca0efc pc=0x80829e8
runtime.goparkunlock(...)
	/workdir/go/src/runtime/proc.go:369
runtime.gcWaitOnMark(0x37)
	/workdir/go/src/runtime/mgc.go:505 +0x5f fp=0x8ca0f24 sp=0x8ca0f10 pc=0x805dbdf
runtime.GC()
	/workdir/go/src/runtime/mgc.go:448 +0x6e fp=0x8ca0f44 sp=0x8ca0f24 pc=0x805da4e
runtime/pprof.TestGoroutineProfileConcurrency.func5(0x8c830e0)
	/workdir/go/src/runtime/pprof/pprof_test.go:1465 +0x19e fp=0x8ca0f9c sp=0x8ca0f44 pc=0x818a15e
testing.tRunner(0x8c830e0, 0x8c100f0)
	/workdir/go/src/testing/testing.go:1446 +0x113 fp=0x8ca0fe4 sp=0x8ca0f9c pc=0x8114fb3
testing.(*T).Run.func1()
	/workdir/go/src/testing/testing.go:1493 +0x2e fp=0x8ca0ff0 sp=0x8ca0fe4 pc=0x8115dae
runtime.goexit()
	/workdir/go/src/runtime/asm_386.s:1326 +0x1 fp=0x8ca0ff4 sp=0x8ca0ff0 pc=0x80b0871
created by testing.(*T).Run
	/workdir/go/src/testing/testing.go:1493 +0x374

greplogs -l -e runtime\\.throw\(\?:.\*\\n\)\*FAIL\\s+runtime/pprof\\s --since=2022-07-01
2022-08-08T17:36:35-ebf182c/linux-386-longtest

attn @golang/runtime

[mknyszek]

I think it's likely this might be the same bug fixed by https://go.dev/425194, because the span in question being scanned is noscan. It would be even more evidence if this test was using cgo. Putting in WaitingForInfo in case it shows up again.

[mknyszek]

Actually, if there's no cgo then this is new, so I guess we should check that.

[mknyszek]

Also, although the failure is from August 8th, we did confirm in triage that this in the window where @randall77 first landed the bitmap changes, which would cause this issue. That was then reverted and landed again because of a different issue, but that issue could be related. I'll check that as well.

[mknyszek]

https://go.dev/cl/422435 was landed in between the revert and the re-land, which specifically mentions noscan objects being scanned. I think this is exactly that, and this issue should be closed. Keith also landed a more general fix for this kind of situation recently, so it should be less likely to happen anyway, but we'll reopen this issue if we see it again.