cmd/go: concurrent map writes in `go/build.(*Context).Import`

[prattmic]

Seen on the linux-amd64-noopt builder:

+ go install -buildmode=shared runtime sync/atomic
fatal error: concurrent map writes

goroutine 82 gp=0x17a15184b40 m=4 mp=0x17a15180008 [running]:
runtime.fatal({0x1054e29, 0x15})
	runtime/panic.go:1267 +0x74 fp=0x17a1535a090 sp=0x17a1535a050 pc=0x45a934
internal/runtime/maps.fatal({0x1054e29?, 0x5b?})
	runtime/panic.go:1195 +0x18 fp=0x17a1535a0b0 sp=0x17a1535a090 pc=0x4981d8
runtime.mapassign_faststr(0x15da8f8, 0x17a1524e120, {0x17a154aecb9, 0x6})
	internal/runtime/maps/runtime_faststr.go:271 +0x65 fp=0x17a1535a188 sp=0x17a1535a0b0 pc=0x40f385
go/build.(*Context).Import(0x1734920, {0x17a152945e9, 0x10}, {0x17a150c03c0, 0x4a}, 0x4)
	go/build/build.go:1039 +0x55dd fp=0x17a1535b508 sp=0x17a1535a188 pc=0x6b057d
cmd/go/internal/load.loadPackageData.func2()
	cmd/go/internal/load/pkg.go:985 +0xaaa fp=0x17a1535b8a0 sp=0x17a1535b508 pc=0xc7b2ea
cmd/internal/par.(*ErrCache[...]).Do.func1()
	cmd/internal/par/work.go:119 +0x4b fp=0x17a1535b920 sp=0x17a1535b8a0 pc=0xad954b
cmd/internal/par.(*Cache[...]).Do(0x10c0920, {0x17a152945e9, 0x10}, 0x17a153b7ad0)
	cmd/internal/par/work.go:160 +0x29e fp=0x17a1535ba68 sp=0x17a1535b920 pc=0xada65e
cmd/internal/par.(*ErrCache[...]).Do(0x10bf300, {0x17a152945e9, 0x10}, 0x17a153b7d00)
	cmd/internal/par/work.go:118 +0xd3 fp=0x17a1535bb00 sp=0x17a1535ba68 pc=0xada0d3
cmd/go/internal/load.loadPackageData(0x17a151c2ae0, {0x16c0d98, 0x1756d80}, {0x17a152945e9, 0x10}, {0x7ffc79e36194, 0x7}, {0x17a150c03c0, 0x4a}, {0x17a150c0197, ...}, ...)
	cmd/go/internal/load/pkg.go:913 +0x8a5 fp=0x17a1535be60 sp=0x17a1535bb00 pc=0xc546e5
cmd/go/internal/load.(*preload).preloadImports.func1({0x17a152945e9, 0x10})
	cmd/go/internal/load/pkg.go:1108 +0x154 fp=0x17a1535bfa8 sp=0x17a1535be60 pc=0xc7c2b4
cmd/go/internal/load.(*preload).preloadImports.gowrap1()
	cmd/go/internal/load/pkg.go:1113 +0x33 fp=0x17a1535bfe0 sp=0x17a1535bfa8 pc=0xc7c133
runtime.goexit({})
	runtime/asm_amd64.s:1264 +0x1 fp=0x17a1535bfe8 sp=0x17a1535bfe0 pc=0x49f121
created by cmd/go/internal/load.(*preload).preloadImports in goroutine 20
	cmd/go/internal/load/pkg.go:1107 +0x4f3

goroutine 1 gp=0x17a1508c1e0 m=nil [runnable]:
runtime.gopark(0x16c45e8, 0x173dca0, 0xd, 0x5, 0x4)
	runtime/proc.go:467 +0xf1 fp=0x17a1543b4f0 sp=0x17a1543b4c0 pc=0x498451
runtime.goparkunlock(0x173dca0?, 0xa4?, 0x5d?, 0x17a1571c310?)
	runtime/proc.go:473 +0x25 fp=0x17a1543b520 sp=0x17a1543b4f0 pc=0x45e245
runtime.semacquire1(0x1735da4, 0x0, 0x0, 0x0, 0xd)
	runtime/sema.go:192 +0x191 fp=0x17a1543b578 sp=0x17a1543b520 pc=0x471311
runtime.semacquire(0x43f4f8?)
	runtime/sema.go:143 +0x1e fp=0x17a1543b5b0 sp=0x17a1543b578 pc=0x47115e
runtime.gcMarkDone()
	runtime/mgc.go:1000 +0x1e fp=0x17a1543b630 sp=0x17a1543b5b0 pc=0x43837e
runtime.gcAssistAlloc(0x17a1508c1e0)
	runtime/mgcmark.go:647 +0x31d fp=0x17a1543b6e8 sp=0x17a1543b630 pc=0x43c91d
runtime.deductAssistCredit(0x0?)
	runtime/malloc_stubs.go:155 +0x46 fp=0x17a1543b708 sp=0x17a1543b6e8 pc=0x42e546
runtime.mallocgc(0x1000, 0x161ff38, 0x1)
	runtime/malloc.go:1116 +0xe7 fp=0x17a1543b738 sp=0x17a1543b708 pc=0x496227
runtime.makeslice(0x17a157829a0?, 0x62?, 0x0?)
	runtime/slice.go:117 +0x86 fp=0x17a1543b770 sp=0x17a1543b738 pc=0x49ac26
bufio.NewReaderSize({0x16ba760, 0x17a1553c1b0}, 0x1000)
	bufio/bufio.go:57 +0x129 fp=0x17a1543b818 sp=0x17a1543b770 pc=0x681369
bufio.NewReader({0x16ba760, 0x17a1553c1b0})
	bufio/bufio.go:63 +0x35 fp=0x17a1543b850 sp=0x17a1543b818 pc=0x681435
go/build.newImportReader({0x0, 0x0}, {0x16ba760, 0x17a1553c1b0})
	go/build/read.go:37 +0x58 fp=0x17a1543b930 sp=0x17a1543b850 pc=0x6bb4f8
go/build.readComments({0x16ba760, 0x17a1553c1b0})
	go/build/read.go:256 +0x4a fp=0x17a1543b9a8 sp=0x17a1543b930 pc=0x6c26ea
go/build.(*Context).matchFile(0x1734920, {0x17a156163c0, 0x53}, {0x17a1553e320, 0xe}, 0x17a1555c390, 0x0, 0x17a1555c3c0)
	go/build/build.go:1489 +0x7d7 fp=0x17a1543bc38 sp=0x17a1543b9a8 pc=0x6b7597
go/build.(*Context).Import(0x1734920, {0x17a151e6529, 0x10}, {0x17a150c03c0, 0x4a}, 0x4)
	go/build/build.go:900 +0x2765 fp=0x17a1543cfb8 sp=0x17a1543bc38 pc=0x6ad705
cmd/go/internal/load.loadPackageData.func2()
	cmd/go/internal/load/pkg.go:985 +0xaaa fp=0x17a1543d350 sp=0x17a1543cfb8 pc=0xc7b2ea
cmd/internal/par.(*ErrCache[...]).Do.func1()
	cmd/internal/par/work.go:119 +0x4b fp=0x17a1543d3d0 sp=0x17a1543d350 pc=0xad954b
cmd/internal/par.(*Cache[...]).Do(0x10c0920, {0x17a151e6529, 0x10}, 0x17a1543d580)
	cmd/internal/par/work.go:160 +0x29e fp=0x17a1543d518 sp=0x17a1543d3d0 pc=0xada65e
cmd/internal/par.(*ErrCache[...]).Do(0x10bf300, {0x17a151e6529, 0x10}, 0x17a1543d7b0)
	cmd/internal/par/work.go:118 +0xd3 fp=0x17a1543d5b0 sp=0x17a1543d518 pc=0xada0d3
cmd/go/internal/load.loadPackageData(0x17a151c2ae0, {0x16c0d98, 0x1756d80}, {0x17a151e6529, 0x10}, {0x7ffc79e36194, 0x7}, {0x17a150c03c0, 0x4a}, {0x17a150c0197, ...}, ...)
	cmd/go/internal/load/pkg.go:913 +0x8a5 fp=0x17a1543d910 sp=0x17a1543d5b0 pc=0xc546e5
cmd/go/internal/load.loadImport(0x17a151c2ae0, {0x16c0d98, 0x1756d80}, {0x0, 0x0, 0x0, 0x1, 0x0, 0x0}, 0x0, ...)
	cmd/go/internal/load/pkg.go:722 +0x2dc fp=0x17a1543de60 sp=0x17a1543d910 pc=0xc5261c
cmd/go/internal/load.(*Package).load(0x17a151bcc08, 0x17a151c2ae0, {0x16c0d98, 0x1756d80}, {0x0, 0x0, 0x0, 0x1, 0x0, 0x0}, ...)
	cmd/go/internal/load/pkg.go:2032 +0x3785 fp=0x17a1543e9b0 sp=0x17a1543de60 pc=0xc5d445
cmd/go/internal/load.loadImport(0x17a151c2ae0, {0x16c0d98, 0x1756d80}, {0x0, 0x0, 0x0, 0x1, 0x0, 0x0}, 0x17a15250aa0, ...)
	cmd/go/internal/load/pkg.go:777 +0x7b3 fp=0x17a1543ef00 sp=0x17a1543e9b0 pc=0xc52af3
cmd/go/internal/load.PackagesAndErrors(0x17a151c2ae0, {0x16c0d98, 0x1756d80}, {0x0, 0x0, 0x0, 0x1, 0x0, 0x0}, {0x17a151c4030, ...})
	cmd/go/internal/load/pkg.go:2949 +0xc25 fp=0x17a1543f4d0 sp=0x17a1543ef00 pc=0xc681e5
cmd/go/internal/work.runInstall({0x16c0d98, 0x1756d80}, 0x17244c8, {0x17a151c4030, 0x2, 0x2})
	cmd/go/internal/work/build.go:701 +0x238 fp=0x17a1543f690 sp=0x17a1543f4d0 pc=0xdc65b8
main.invoke(0x17244c8, {0x17a151c4030, 0x2, 0x2})
	cmd/go/main.go:340 +0xc13 fp=0x17a1543fb10 sp=0x17a1543f690 pc=0xf3ce73
main.main()
	cmd/go/main.go:219 +0x1b65 fp=0x17a1543ff70 sp=0x17a1543fb10 pc=0xf3bba5
runtime.main()
	runtime/proc.go:295 +0x275 fp=0x17a1543ffe0 sp=0x17a1543ff70 pc=0x45de15
runtime.goexit({})
	runtime/asm_amd64.s:1264 +0x1 fp=0x17a1543ffe8 sp=0x17a1543ffe0 pc=0x49f121

At a glance this doesn't look possible, as importMap is a locally-created map.

cc @golang/command-line @matloob

[gabyhelp]

Related Issues

• cmd/go: crash in symabis #31922 (closed)
• cmd/compile/internal/types: concurrent map read and map write while compiling #21352 (closed)
• runtime: crash in gcInstallStackBarrier #11057 (closed)
• runtime: garbage collector found invalid heap pointer #21432 (closed)
• x/tools/gopls: concurrent map write panic #32809 (closed)
• cmd/go: "fatal error: concurrent map writes" during go get #35317 (closed)
• runtime: panic (slice bounds out of range) in runtime.gcInstallStackBarrier #11080 (closed)
• runtime: darwin/arm64 panics with slice bounds out of range error #11085 (closed)
• runtime: crashes on OS X 10.9 at TestGCFairness-2 #9564 (closed)
• runtime: trybot windows/amd64 failure in gc shade #18867 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[adonovan]

I agree, it looks impossible. All the possible referents of that map variable are confined to the single goroutine that executes Ctxt.Import. That suggests either a bug in the runtime, or a soft error.

[dmitshur]

@prattmic Do you happen to have a link to the build handy? I tried looking over the last few https://ci.chromium.org/ui/p/golang/builders/luci.golang.ci/gotip-linux-amd64-noopt and didn't spot this failure mode there. (Maybe what you saw was on a pre-submit run?)

One of the things I stood out to me earlier about the noopt builder turned out to be me mixing up GO_GCFLAGS and GOFLAGS, so at least that part isn't actually a problem.

[prattmic]

Oops, sorry I forgot the link: https://ci.chromium.org/ui/p/golang/builders/ci/gotip-linux-amd64-noopt/b8681256315191280049/test-results

This is a post-submit run on gotip-linux-amd64-noopt