runtime,cmd/compile: `unexpected fault address` in `runtime.mapassign`

[gopherbot]

#!watchflakes
post <- `unexpected fault address 0x[0-9a-f]+` && `^\s*runtime\.mapassign`

Issue created automatically to collect these failures.

Example (log):

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x21 pc=0xb91b7a]

goroutine 1 [running]:
cmd/link/internal/loader.(*Loader).OuterSym(...)
	C:/workdir/go/src/cmd/link/internal/loader/loader.go:1695
cmd/link/internal/ld.(*deadcodePass).flood(0xc00006aa00)
	C:/workdir/go/src/cmd/link/internal/ld/deadcode.go:239 +0x265
cmd/link/internal/ld.deadcode(0xc0000c0000)
	C:/workdir/go/src/cmd/link/internal/ld/deadcode.go:336 +0x89
cmd/link/internal/ld.Main(_, {0x20, 0x20, 0x1, 0x7, 0x10, 0x0, {0x0, 0x0}, {0xe248e3, ...}, ...})
	C:/workdir/go/src/cmd/link/internal/ld/main.go:269 +0xef2
main.main()
	C:/workdir/go/src/cmd/link/main.go:72 +0xedb

— watchflakes

[gopherbot]

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/text/internal/triegen.test" && test == ""

2022-10-14 17:33 windows-amd64-race text@1bdb400f go@bfcc3a75 x/text/internal/triegen.test [build] (log)

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x21 pc=0xb91b7a]

goroutine 1 [running]:
cmd/link/internal/loader.(*Loader).OuterSym(...)
	C:/workdir/go/src/cmd/link/internal/loader/loader.go:1695
cmd/link/internal/ld.(*deadcodePass).flood(0xc00006aa00)
	C:/workdir/go/src/cmd/link/internal/ld/deadcode.go:239 +0x265
cmd/link/internal/ld.deadcode(0xc0000c0000)
	C:/workdir/go/src/cmd/link/internal/ld/deadcode.go:336 +0x89
cmd/link/internal/ld.Main(_, {0x20, 0x20, 0x1, 0x7, 0x10, 0x0, {0x0, 0x0}, {0xe248e3, ...}, ...})
	C:/workdir/go/src/cmd/link/internal/ld/main.go:269 +0xef2
main.main()
	C:/workdir/go/src/cmd/link/main.go:72 +0xedb

— watchflakes

[bcmills]

(attn @golang/runtime; CC @golang/windows)

[alexbrainman]

@bcmills

There are couple of different stack traces.

I looked at the first stack trace in the log:

go build golang.org/x/text/internal/number: # golang.org/x/text/internal/number [golang.org/x/text/internal/number.test]
unexpected fault address 0xffffffffffffffff
fatal error: fault
[signal 0xc0000005 code=0x0 addr=0xffffffffffffffff pc=0x4fe86]

goroutine 1 [running]:
runtime.throw({0x9d37da?, 0xc00011c4a8?})
	C:/workdir/go/src/runtime/panic.go:1047 +0x65 fp=0xc00011c470 sp=0xc00011c440 pc=0x79e85
runtime.sigpanic()
	C:/workdir/go/src/runtime/signal_windows.go:270 +0xd0 fp=0xc00011c4b8 sp=0xc00011c470 pc=0x8ed50
runtime.mapassign(0x95cbe0, 0xc000113e00, 0x755a2d?)
	C:/workdir/go/src/runtime/map.go:621 +0x246 fp=0xc00011c540 sp=0xc00011c4b8 pc=0x4fe86
cmd/compile/internal/ssagen.InitTables.func3({0x9dcb21, 0xb}, {0x9dbcfe, 0xb}, {0x9e9cf8, 0x17}, {0x9d2ad8, 0x5}, {0xc000126580, 0xe, ...})
	C:/workdir/go/src/cmd/compile/internal/ssagen/ssa.go:3944 +0x1fa fp=0xc00011c608 sp=0xc00011c540 pc=0x74a25a
cmd/compile/internal/ssagen.InitTables()
	C:/workdir/go/src/cmd/compile/internal/ssagen/ssa.go:4755 +0xb054 fp=0xc00011da98 sp=0xc00011c608 pc=0x72e754
cmd/compile/internal/gc.Main(0xa0cfd8)
	C:/workdir/go/src/cmd/compile/internal/gc/main.go:173 +0xaaa fp=0xc00011df20 sp=0xc00011da98 pc=0x8e672a
main.main()
	C:/workdir/go/src/cmd/compile/main.go:57 +0xdd fp=0xc00011df80 sp=0xc00011df20 pc=0x90e93d
runtime.main()
	C:/workdir/go/src/runtime/proc.go:250 +0x1fe fp=0xc00011dfe0 sp=0xc00011df80 pc=0x7c5fe
runtime.goexit()
	C:/workdir/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00011dfe8 sp=0xc00011dfe0 pc=0xaca81

The crash happens here

go/src/runtime/map.go

Line 621 in bfcc3a7

if b.tophash[i] != top {

Which looks like uninitialised map used in

go/src/cmd/compile/internal/ssagen/ssa.go

Line 3944 in bfcc3a7

intrinsics[intrinsicKey{a, pkg, fn}] = b

I have no idea how to fix that problem.

Alex

[randall77]

This crash happens early on during compiler startup, before it has even read any of the code to be compiled.
In fact, I think the compiler is completely deterministic up to this point, so it is hard to imagine why it would crash on these runs but not every time.
So some unlikely but possible causes: something wrong with runtime/gc, randomness in map hash triggering some rare map bug, or our favorite boogeyman, bad machine memory.

[randall77]

Both failures are with -race, which may implicate the race detector itself or our instrumentation that calls it (e.g. if it messed up live variable stack maps that in turn caused the GC to overcollect).

[prattmic]

The crashes are in the compiler and linker. Just double checking, do we build the toolchain with the race detector enabled on race builders, or just the tests?

[randall77]

Good point, I don't think the compiler/linker is race-enabled when on a race builder. Just the resulting tests.

[mknyszek]

In triage the consensus is to wait for more to happen. @prattmic notes that it's suspicious that 3 compiler/linker processes crashed at once. Maybe something wrong with the machine?