Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: "runtime: cannot allocate memory" in workspaces with many non go files #54226

Open
florianl opened this issue Aug 3, 2022 · 6 comments
Labels
GoCommand NeedsInvestigation
Milestone

Comments

@florianl
Copy link
Contributor

florianl commented Aug 3, 2022

What version of Go are you using (go version)?

$ go version
go version go1.19 linux/amd64

Does this issue reproduce with the latest release?

yes. No for go1.18.5.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/flehner/.cache/go-build"
GOENV="/home/flehner/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/flehner/go/pkg/mod"
GONOPROXY="github.com/optimyze"
GONOSUMDB="github.com/optimyze"
GOOS="linux"
GOPATH="/home/flehner/go"
GOPRIVATE="github.com/optimyze"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3567430451=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ go mod tidy

What did you expect to see?

No changes at all or changes to go.mod and/or go.sum.

What did you see instead?

$ go version
go version go1.19 linux/amd64
$ go mod tidy
fatal error: runtime: cannot allocate memory

runtime stack:
runtime.throw({0xa35670?, 0xfffffffffffff800?})
	/usr/local/go/src/runtime/panic.go:1047 +0x5d fp=0x7f224d172d58 sp=0x7f224d172d28 pc=0x43765d
runtime.persistentalloc1(0x1018, 0x7f224d172dc0?, 0xe69838)
	/usr/local/go/src/runtime/malloc.go:1377 +0x24f fp=0x7f224d172da0 sp=0x7f224d172d58 pc=0x40f90f
runtime.persistentalloc.func1()
	/usr/local/go/src/runtime/malloc.go:1330 +0x2e fp=0x7f224d172dd0 sp=0x7f224d172da0 pc=0x40f68e
runtime.systemstack()
	/usr/local/go/src/runtime/asm_amd64.s:492 +0x49 fp=0x7f224d172dd8 sp=0x7f224d172dd0 pc=0x465e89

goroutine 53770 [running]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_amd64.s:459 fp=0xc007c3f718 sp=0xc007c3f710 pc=0x465e20
runtime.persistentalloc(0x40?, 0xe69838?, 0xc007c3f748?)
	/usr/local/go/src/runtime/malloc.go:1329 +0x65 fp=0xc007c3f760 sp=0xc007c3f718 pc=0x40f605
runtime.(*spanSetBlockAlloc).alloc(0xa4?)
	/usr/local/go/src/runtime/mspanset.go:286 +0x76 fp=0xc007c3f788 sp=0xc007c3f760 pc=0x4322d6
runtime.(*spanSet).push(0xe5f2d8, 0x7f222bf2ff60)
	/usr/local/go/src/runtime/mspanset.go:124 +0x10c fp=0xc007c3f7e0 sp=0xc007c3f788 pc=0x431f4c
runtime.(*mcentral).uncacheSpan(0x2?, 0xc005299ec0?)
	/usr/local/go/src/runtime/mcentral.go:234 +0x8d fp=0xc007c3f808 sp=0xc007c3f7e0 pc=0x41942d
runtime.(*mcache).refill(0x7f22aef98a68, 0x1c)
	/usr/local/go/src/runtime/mcache.go:158 +0x9f fp=0xc007c3f848 sp=0xc007c3f808 pc=0x41895f
runtime.(*mcache).nextFree(0x7f22aef98a68, 0x1c)
	/usr/local/go/src/runtime/malloc.go:819 +0x85 fp=0xc007c3f890 sp=0xc007c3f848 pc=0x40e6e5
runtime.mallocgc(0xc0, 0xa09260, 0x1)
	/usr/local/go/src/runtime/malloc.go:1018 +0x4c8 fp=0xc007c3f908 sp=0xc007c3f890 pc=0x40ed48
runtime.newobject(0xc007115760?)
	/usr/local/go/src/runtime/malloc.go:1192 +0x27 fp=0xc007c3f930 sp=0xc007c3f908 pc=0x40f267
cmd/go/internal/modindex.importRaw({0xc000030004, 0x32}, {0xc006625f23, 0x71})
	/usr/local/go/src/cmd/go/internal/modindex/scan.go:217 +0x70a fp=0xc007c3fbc8 sp=0xc007c3f930 pc=0x7ddf8a
cmd/go/internal/modindex.indexPackage({0xc000030004, 0x32}, {0xc006625ef0, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/scan.go:77 +0x6b fp=0xc007c3fc00 sp=0xc007c3fbc8 pc=0x7dd52b
cmd/go/internal/modindex.openIndexPackage.func1()
	/usr/local/go/src/cmd/go/internal/modindex/read.go:226 +0x134 fp=0xc007c3fd08 sp=0xc007c3fc00 pc=0x7d5df4
cmd/go/internal/par.(*Cache).Do(0xe32bc0, {0x97f3e0, 0xc004cdd000}, 0xc008badd90)
	/usr/local/go/src/cmd/go/internal/par/work.go:128 +0xc3 fp=0xc007c3fd50 sp=0xc007c3fd08 pc=0x5bb6e3
cmd/go/internal/modindex.openIndexPackage({0xc000030004, 0x32}, {0xc006625ef0, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:216 +0xd0 fp=0xc007c3fdc8 sp=0xc007c3fd50 pc=0x7d5c50
cmd/go/internal/modindex.GetPackage({0xc000030004, 0x32}, {0xc006625ef0, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:145 +0x156 fp=0xc007c3fe10 sp=0xc007c3fdc8 pc=0x7d5516
cmd/go/internal/modload.scanDir({0xc000030004?, 0x71ebc31?}, {0xc006625ef0, 0xa4}, 0x0?)
	/usr/local/go/src/cmd/go/internal/modload/load.go:2105 +0x4b fp=0xc007c3ff10 sp=0xc007c3fe10 pc=0x816d8b
cmd/go/internal/modload.matchPackages.func4.1.1()
	/usr/local/go/src/cmd/go/internal/modload/search.go:133 +0x4c fp=0xc007c3ff60 sp=0xc007c3ff10 pc=0x8290cc
cmd/go/internal/par.(*Queue).Add.func1()
	/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d fp=0xc007c3ffe0 sp=0xc007c3ff60 pc=0x5bad7d
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc007c3ffe8 sp=0xc007c3ffe0 pc=0x468041
created by cmd/go/internal/par.(*Queue).Add
	/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x225

goroutine 1 [runnable]:
syscall.Syscall6(0xc006648fd0?, 0xb0?, 0x7f22aef9a1d8?, 0xd0?, 0xc000100000?, 0xc005314b60?, 0x0?)
	/usr/local/go/src/syscall/syscall_linux.go:90 +0x36 fp=0xc0001d8420 sp=0xc0001d8398 pc=0x47e036
syscall.fstatat(0xc0001d84f8?, {0xc006648fd0?, 0x9c3b20?}, 0xc005314b98, 0x3?)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:1439 +0x9a fp=0xc0001d8498 sp=0xc0001d8420 pc=0x47d41a
syscall.Lstat(...)
	/usr/local/go/src/syscall/syscall_linux_amd64.go:67
os.lstatNolog.func1(...)
	/usr/local/go/src/os/stat_unix.go:45
os.ignoringEINTR(...)
	/usr/local/go/src/os/file_posix.go:245
os.lstatNolog({0xc006648fd0, 0xb0})
	/usr/local/go/src/os/stat_unix.go:44 +0x5b fp=0xc0001d84e8 sp=0xc0001d8498 pc=0x4e045b
os.Lstat({0xc006648fd0, 0xb0})
	/usr/local/go/src/os/stat.go:22 +0x34 fp=0xc0001d8508 sp=0xc0001d84e8 pc=0x4dfe74
os.(*File).readdir(0xc0072e3600, 0xffffffffffffffff?, 0x2)
	/usr/local/go/src/os/dir_unix.go:128 +0x6fb fp=0xc0001d8628 sp=0xc0001d8508 pc=0x4d7f7b
os.(*File).Readdir(0xc007328d10?, 0xa4?)
	/usr/local/go/src/os/dir.go:41 +0x25 fp=0xc0001d8660 sp=0xc0001d8628 pc=0x4d7285
io/ioutil.ReadDir({0xc007328d10?, 0xc0001d8728?})
	/usr/local/go/src/io/ioutil/ioutil.go:74 +0x45 fp=0xc0001d86e0 sp=0xc0001d8660 pc=0x5946a5
cmd/go/internal/fsys.readDir({0xc007328d10, 0xa4})
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:297 +0x27 fp=0xc0001d8720 sp=0xc0001d86e0 pc=0x5982c7
cmd/go/internal/fsys.ReadDir({0xc007328d10, 0xa4})
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:322 +0x2c5 fp=0xc0001d8ab8 sp=0xc0001d8720 pc=0x5986e5
cmd/go/internal/fsys.walk({0xc007328d10, 0xa4}, {0xb2e510, 0xc005245ee0}, 0xc0001d9100)
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:471 +0xb8 fp=0xc0001d8b60 sp=0xc0001d8ab8 pc=0x5997d8
cmd/go/internal/fsys.walk({0xc00523ea00, 0x96}, {0xb2e510, 0xc005245d40}, 0xc0001d9100)
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:478 +0x1bc fp=0xc0001d8c08 sp=0xc0001d8b60 pc=0x5998dc
cmd/go/internal/fsys.walk({0xc005b6ea00, 0x7c}, {0xb2e510, 0xc004c429c0}, 0xc0001d9100)
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:478 +0x1bc fp=0xc0001d8cb0 sp=0xc0001d8c08 pc=0x5998dc
cmd/go/internal/fsys.walk({0xc002a10b60, 0x6a}, {0xb2e510, 0xc000513860}, 0xc0001d9100)
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:478 +0x1bc fp=0xc0001d8d58 sp=0xc0001d8cb0 pc=0x5998dc
cmd/go/internal/fsys.walk({0xc0002c1620, 0x51}, {0xb2e510, 0xc0003def70}, 0xc0001d9100)
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:478 +0x1bc fp=0xc0001d8e00 sp=0xc0001d8d58 pc=0x5998dc
cmd/go/internal/fsys.walk({0xc0004f4960, 0x49}, {0xb2e510, 0xc00038fd40}, 0xc0001d9100)
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:478 +0x1bc fp=0xc0001d8ea8 sp=0xc0001d8e00 pc=0x5998dc
cmd/go/internal/fsys.walk({0xc000217c80, 0x3e}, {0xb2e510, 0xc0002acea0}, 0xc0001d9100)
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:478 +0x1bc fp=0xc0001d8f50 sp=0xc0001d8ea8 pc=0x5998dc
cmd/go/internal/fsys.walk({0xc000030004, 0x32}, {0xb2e510, 0xc0002332b0}, 0xc0001d9100)
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:478 +0x1bc fp=0xc0001d8ff8 sp=0xc0001d8f50 pc=0x5998dc
cmd/go/internal/fsys.Walk({0xc000030004, 0x32}, 0xc0001d9100)
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:495 +0x94 fp=0xc0001d9048 sp=0xc0001d8ff8 pc=0x599a54
cmd/go/internal/modload.matchPackages.func4({0xc000030004, 0x32}, {0xc0001bc540, 0x1d}, 0x3)
	/usr/local/go/src/cmd/go/internal/modload/search.go:80 +0x231 fp=0xc0001d9180 sp=0xc0001d9048 pc=0x8285f1
cmd/go/internal/modload.matchPackages({0xb2d2d0?, 0xc000188000?}, 0xc00019e2a0, 0xc000215ec0, 0x0, {0xc0001adb20, 0x1, 0xc0002879b0?})
	/usr/local/go/src/cmd/go/internal/modload/search.go:209 +0xc0d fp=0xc0001d9460 sp=0xc0001d9180 pc=0x82808d
cmd/go/internal/modload.LoadPackages.func1(0xe34de0?, 0x0)
	/usr/local/go/src/cmd/go/internal/modload/load.go:324 +0x734 fp=0xc0001d95d8 sp=0xc0001d9460 pc=0x80cb34
cmd/go/internal/modload.LoadPackages.func2(0xc0002331e0?)
	/usr/local/go/src/cmd/go/internal/modload/load.go:351 +0x2c fp=0xc0001d9650 sp=0xc0001d95d8 pc=0x80c2ac
cmd/go/internal/modload.loadFromRoots({0xb2d2d0, 0xc000188000}, {{{0x0, 0x0}, 0xc000215ec0, 0x1, {0x0, 0x0}, 0x1, 0x1, ...}, ...})
	/usr/local/go/src/cmd/go/internal/modload/load.go:1066 +0x5f4 fp=0xc0001d99b0 sp=0xc0001d9650 pc=0x810474
cmd/go/internal/modload.LoadPackages({0xb2d2d0?, 0xc000188000}, {{0x0, 0x0}, 0xc000215ec0, 0x1, {0x0, 0x0}, 0x1, 0x1, ...}, ...)
	/usr/local/go/src/cmd/go/internal/modload/load.go:344 +0x308 fp=0xc0001d9c60 sp=0xc0001d99b0 pc=0x80b6e8
cmd/go/internal/modcmd.runTidy({0xb2d2d0, 0xc000188000}, 0xc0001ba4e0?, {0xc0001a2030?, 0x8?, 0x4?})
	/usr/local/go/src/cmd/go/internal/modcmd/tidy.go:114 +0x198 fp=0xc0001d9d88 sp=0xc0001d9c60 pc=0x90cf58
main.invoke(0xe269a0, {0xc0001a2020, 0x2, 0x2})
	/usr/local/go/src/cmd/go/main.go:225 +0x34e fp=0xc0001d9e60 sp=0xc0001d9d88 pc=0x9400ae
main.main()
	/usr/local/go/src/cmd/go/main.go:179 +0x7d1 fp=0xc0001d9f80 sp=0xc0001d9e60 pc=0x93fa31
runtime.main()
	/usr/local/go/src/runtime/proc.go:250 +0x212 fp=0xc0001d9fe0 sp=0xc0001d9f80 pc=0x439eb2
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0001d9fe8 sp=0xc0001d9fe0 pc=0x468041

goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000060fb0 sp=0xc000060f90 pc=0x43a276
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:369
runtime.forcegchelper()
	/usr/local/go/src/runtime/proc.go:302 +0xad fp=0xc000060fe0 sp=0xc000060fb0 pc=0x43a10d
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000060fe8 sp=0xc000060fe0 pc=0x468041
created by runtime.init.6
	/usr/local/go/src/runtime/proc.go:290 +0x25

goroutine 3 [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000061790 sp=0xc000061770 pc=0x43a276
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:369
runtime.bgsweep(0x0?)
	/usr/local/go/src/runtime/mgcsweep.go:297 +0xd7 fp=0xc0000617c8 sp=0xc000061790 pc=0x426957
runtime.gcenable.func1()
	/usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000617e0 sp=0xc0000617c8 pc=0x41b5c6
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000617e8 sp=0xc0000617e0 pc=0x468041
created by runtime.gcenable
	/usr/local/go/src/runtime/mgc.go:178 +0x6b

goroutine 4 [GC scavenge wait]:
runtime.gopark(0xe33680?, 0xb8f238?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000061f70 sp=0xc000061f50 pc=0x43a276
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:369
runtime.(*scavengerState).park(0xe33680)
	/usr/local/go/src/runtime/mgcscavenge.go:389 +0x53 fp=0xc000061fa0 sp=0xc000061f70 pc=0x4249b3
runtime.bgscavenge(0x0?)
	/usr/local/go/src/runtime/mgcscavenge.go:622 +0x65 fp=0xc000061fc8 sp=0xc000061fa0 pc=0x424fa5
runtime.gcenable.func2()
	/usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000061fe0 sp=0xc000061fc8 pc=0x41b566
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000061fe8 sp=0xc000061fe0 pc=0x468041
created by runtime.gcenable
	/usr/local/go/src/runtime/mgc.go:179 +0xaa

goroutine 18 [finalizer wait]:
runtime.gopark(0x0?, 0xa89728?, 0x0?, 0x0?, 0x2000000020?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000060628 sp=0xc000060608 pc=0x43a276
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:369
runtime.runfinq()
	/usr/local/go/src/runtime/mfinal.go:180 +0x10f fp=0xc0000607e0 sp=0xc000060628 pc=0x41a6cf
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000607e8 sp=0xc0000607e0 pc=0x468041
created by runtime.createfing
	/usr/local/go/src/runtime/mfinal.go:157 +0x45

goroutine 53771 [runnable]:
runtime.convTstring({0xc0053a15c5?, 0x25?})
	/usr/local/go/src/runtime/iface.go:388 +0x89 fp=0xc008ba3ad8 sp=0xc008ba3ad0 pc=0x40cc29
cmd/go/internal/modindex.dirHash({0xc000030004, 0x32}, {0xc007328630, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:116 +0x39f fp=0xc008ba3c00 sp=0xc008ba3ad8 pc=0x7d525f
cmd/go/internal/modindex.openIndexPackage.func1()
	/usr/local/go/src/cmd/go/internal/modindex/read.go:218 +0x85 fp=0xc008ba3d08 sp=0xc008ba3c00 pc=0x7d5d45
cmd/go/internal/par.(*Cache).Do(0xe32bc0, {0x97f3e0, 0xc004f3d520}, 0xc00809ed90)
	/usr/local/go/src/cmd/go/internal/par/work.go:128 +0xc3 fp=0xc008ba3d50 sp=0xc008ba3d08 pc=0x5bb6e3
cmd/go/internal/modindex.openIndexPackage({0xc000030004, 0x32}, {0xc007328630, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:216 +0xd0 fp=0xc008ba3dc8 sp=0xc008ba3d50 pc=0x7d5c50
cmd/go/internal/modindex.GetPackage({0xc000030004, 0x32}, {0xc007328630, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:145 +0x156 fp=0xc008ba3e10 sp=0xc008ba3dc8 pc=0x7d5516
cmd/go/internal/modload.scanDir({0xc000030004?, 0x0?}, {0xc007328630, 0xa4}, 0x0?)
	/usr/local/go/src/cmd/go/internal/modload/load.go:2105 +0x4b fp=0xc008ba3f10 sp=0xc008ba3e10 pc=0x816d8b
cmd/go/internal/modload.matchPackages.func4.1.1()
	/usr/local/go/src/cmd/go/internal/modload/search.go:133 +0x4c fp=0xc008ba3f60 sp=0xc008ba3f10 pc=0x8290cc
cmd/go/internal/par.(*Queue).Add.func1()
	/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d fp=0xc008ba3fe0 sp=0xc008ba3f60 pc=0x5bad7d
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc008ba3fe8 sp=0xc008ba3fe0 pc=0x468041
created by cmd/go/internal/par.(*Queue).Add
	/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x225

goroutine 116 [GC worker (idle)]:
runtime.gopark(0x8edbeb06ef8?, 0x3?, 0x5b?, 0x41?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0005aef50 sp=0xc0005aef30 pc=0x43a276
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0005aefe0 sp=0xc0005aef50 pc=0x41d711
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0005aefe8 sp=0xc0005aefe0 pc=0x468041
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1159 +0x25

goroutine 115 [GC worker (idle)]:
runtime.gopark(0x8edccd76d83?, 0x3?, 0xe6?, 0x77?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0005b0750 sp=0xc0005b0730 pc=0x43a276
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0005b07e0 sp=0xc0005b0750 pc=0x41d711
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0005b07e8 sp=0xc0005b07e0 pc=0x468041
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1159 +0x25

goroutine 99 [GC worker (idle)]:
runtime.gopark(0x8edccd78360?, 0x3?, 0x74?, 0x24?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0005b0f50 sp=0xc0005b0f30 pc=0x43a276
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0005b0fe0 sp=0xc0005b0f50 pc=0x41d711
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0005b0fe8 sp=0xc0005b0fe0 pc=0x468041
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1159 +0x25

goroutine 98 [GC worker (idle)]:
runtime.gopark(0xe65800?, 0x3?, 0x33?, 0x90?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0005b1f50 sp=0xc0005b1f30 pc=0x43a276
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0005b1fe0 sp=0xc0005b1f50 pc=0x41d711
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0005b1fe8 sp=0xc0005b1fe0 pc=0x468041
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1159 +0x25

goroutine 114 [GC worker (idle)]:
runtime.gopark(0x8edccd781e2?, 0x1?, 0x81?, 0x5b?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0005aa750 sp=0xc0005aa730 pc=0x43a276
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0005aa7e0 sp=0xc0005aa750 pc=0x41d711
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0005aa7e8 sp=0xc0005aa7e0 pc=0x468041
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1159 +0x25

goroutine 130 [GC worker (idle)]:
runtime.gopark(0x8eda8166fa7?, 0x3?, 0x51?, 0x36?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0005aaf50 sp=0xc0005aaf30 pc=0x43a276
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0005aafe0 sp=0xc0005aaf50 pc=0x41d711
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0005aafe8 sp=0xc0005aafe0 pc=0x468041
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1159 +0x25

goroutine 93 [GC worker (idle)]:
runtime.gopark(0x8edccd7780d?, 0x3?, 0xde?, 0x44?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0005ab750 sp=0xc0005ab730 pc=0x43a276
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0005ab7e0 sp=0xc0005ab750 pc=0x41d711
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0005ab7e8 sp=0xc0005ab7e0 pc=0x468041
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1159 +0x25

goroutine 94 [GC worker (idle)]:
runtime.gopark(0x8edcc577f9b?, 0x1?, 0xb6?, 0x49?, 0x0?)
	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0005abf50 sp=0xc0005abf30 pc=0x43a276
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0005abfe0 sp=0xc0005abf50 pc=0x41d711
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0005abfe8 sp=0xc0005abfe0 pc=0x468041
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1159 +0x25

goroutine 53772 [runnable]:
runtime.gcTrigger.test({0x0?, 0x0?, 0x0?})
	/usr/local/go/src/runtime/mgc.go:548 +0xfc fp=0xc008ba7490 sp=0xc008ba7488 pc=0x41b9bc
runtime.mallocgc(0xa5, 0x0, 0x0)
	/usr/local/go/src/runtime/malloc.go:1137 +0x7fd fp=0xc008ba7508 sp=0xc008ba7490 pc=0x40f07d
runtime.makeslicecopy(0xc007328d10?, 0xa4?, 0x994400?, 0xc007328d10)
	/usr/local/go/src/runtime/slice.go:57 +0x89 fp=0xc008ba7550 sp=0xc008ba7508 pc=0x44e669
syscall.ByteSliceFromString({0xc007328d10, 0xa4})
	/usr/local/go/src/syscall/syscall.go:52 +0x8e fp=0xc008ba7588 sp=0xc008ba7550 pc=0x47842e
syscall.BytePtrFromString(...)
	/usr/local/go/src/syscall/syscall.go:68
syscall.openat(0xc0083b6000?, {0xc007328d10?, 0x0?}, 0x0?, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:64 +0x3c fp=0xc008ba7600 sp=0xc008ba7588 pc=0x47ac7c
syscall.Open(...)
	/usr/local/go/src/syscall/syscall_linux.go:232
os.openFileNolog({0xc007328d10, 0xa4}, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:216 +0x9b fp=0xc008ba7648 sp=0xc008ba7600 pc=0x4dd83b
os.OpenFile({0xc007328d10, 0xa4}, 0x0, 0x72be5d74?)
	/usr/local/go/src/os/file.go:337 +0x45 fp=0xc008ba7680 sp=0xc008ba7648 pc=0x4db685
os.Open(...)
	/usr/local/go/src/os/file.go:317
io/ioutil.ReadDir({0xc007328d10?, 0x0?})
	/usr/local/go/src/io/ioutil/ioutil.go:70 +0x29 fp=0xc008ba7700 sp=0xc008ba7680 pc=0x594689
cmd/go/internal/fsys.readDir({0xc007328d10, 0xa4})
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:297 +0x27 fp=0xc008ba7740 sp=0xc008ba7700 pc=0x5982c7
cmd/go/internal/fsys.ReadDir({0xc007328d10, 0xa4})
	/usr/local/go/src/cmd/go/internal/fsys/fsys.go:322 +0x2c5 fp=0xc008ba7ad8 sp=0xc008ba7740 pc=0x5986e5
cmd/go/internal/modindex.dirHash({0xc000030004, 0x32}, {0xc007328d10, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:90 +0x1a8 fp=0xc008ba7c00 sp=0xc008ba7ad8 pc=0x7d5068
cmd/go/internal/modindex.openIndexPackage.func1()
	/usr/local/go/src/cmd/go/internal/modindex/read.go:218 +0x85 fp=0xc008ba7d08 sp=0xc008ba7c00 pc=0x7d5d45
cmd/go/internal/par.(*Cache).Do(0xe32bc0, {0x97f3e0, 0xc0049b5640}, 0xc008ba9d90)
	/usr/local/go/src/cmd/go/internal/par/work.go:128 +0xc3 fp=0xc008ba7d50 sp=0xc008ba7d08 pc=0x5bb6e3
cmd/go/internal/modindex.openIndexPackage({0xc000030004, 0x32}, {0xc007328d10, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:216 +0xd0 fp=0xc008ba7dc8 sp=0xc008ba7d50 pc=0x7d5c50
cmd/go/internal/modindex.GetPackage({0xc000030004, 0x32}, {0xc007328d10, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:145 +0x156 fp=0xc008ba7e10 sp=0xc008ba7dc8 pc=0x7d5516
cmd/go/internal/modload.scanDir({0xc000030004?, 0x0?}, {0xc007328d10, 0xa4}, 0x0?)
	/usr/local/go/src/cmd/go/internal/modload/load.go:2105 +0x4b fp=0xc008ba7f10 sp=0xc008ba7e10 pc=0x816d8b
cmd/go/internal/modload.matchPackages.func4.1.1()
	/usr/local/go/src/cmd/go/internal/modload/search.go:133 +0x4c fp=0xc008ba7f60 sp=0xc008ba7f10 pc=0x8290cc
cmd/go/internal/par.(*Queue).Add.func1()
	/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d fp=0xc008ba7fe0 sp=0xc008ba7f60 pc=0x5bad7d
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc008ba7fe8 sp=0xc008ba7fe0 pc=0x468041
created by cmd/go/internal/par.(*Queue).Add
	/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x225

goroutine 53768 [runnable]:
crypto/sha256.(*digest).Sum(0xc007309a00, {0xc001197b20, 0x0, 0x20})
	/usr/local/go/src/crypto/sha256/sha256.go:215 +0x205 fp=0xc001197ab8 sp=0xc001197ab0 pc=0x5aa105
cmd/go/internal/cache.(*Cache).put(0x100?, {0x17, 0x7b, 0x65, 0xb8, 0xeb, 0x70, 0x1, 0x49, 0x93, ...}, ...)
	/usr/local/go/src/cmd/go/internal/cache/cache.go:454 +0x125 fp=0xc001197b68 sp=0xc001197ab8 pc=0x7c86e5
cmd/go/internal/cache.(*Cache).Put(...)
	/usr/local/go/src/cmd/go/internal/cache/cache.go:432
cmd/go/internal/cache.(*Cache).PutBytes(0x0?, {0x17, 0x7b, 0x65, 0xb8, 0xeb, 0x70, 0x1, 0x49, 0x93, ...}, ...)
	/usr/local/go/src/cmd/go/internal/cache/cache.go:467 +0xd7 fp=0xc001197c00 sp=0xc001197b68 pc=0x7c8937
cmd/go/internal/modindex.openIndexPackage.func1()
	/usr/local/go/src/cmd/go/internal/modindex/read.go:227 +0x1ad fp=0xc001197d08 sp=0xc001197c00 pc=0x7d5e6d
cmd/go/internal/par.(*Cache).Do(0xe32bc0, {0x97f3e0, 0xc00476b6c0}, 0xc00118dd90)
	/usr/local/go/src/cmd/go/internal/par/work.go:128 +0xc3 fp=0xc001197d50 sp=0xc001197d08 pc=0x5bb6e3
cmd/go/internal/modindex.openIndexPackage({0xc000030004, 0x32}, {0xc006624fd0, 0xa3})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:216 +0xd0 fp=0xc001197dc8 sp=0xc001197d50 pc=0x7d5c50
cmd/go/internal/modindex.GetPackage({0xc000030004, 0x32}, {0xc006624fd0, 0xa3})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:145 +0x156 fp=0xc001197e10 sp=0xc001197dc8 pc=0x7d5516
cmd/go/internal/modload.scanDir({0xc000030004?, 0x0?}, {0xc006624fd0, 0xa3}, 0x0?)
	/usr/local/go/src/cmd/go/internal/modload/load.go:2105 +0x4b fp=0xc001197f10 sp=0xc001197e10 pc=0x816d8b
cmd/go/internal/modload.matchPackages.func4.1.1()
	/usr/local/go/src/cmd/go/internal/modload/search.go:133 +0x4c fp=0xc001197f60 sp=0xc001197f10 pc=0x8290cc
cmd/go/internal/par.(*Queue).Add.func1()
	/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d fp=0xc001197fe0 sp=0xc001197f60 pc=0x5bad7d
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc001197fe8 sp=0xc001197fe0 pc=0x468041
created by cmd/go/internal/par.(*Queue).Add
	/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x225

goroutine 53767 [runnable]:
syscall.Syscall6(0xc00083e7e0?, 0x63?, 0xa213c0?, 0x1?, 0xc008c8b850?, 0x7c5596?, 0xc008c8b820?)
	/usr/local/go/src/syscall/syscall_linux.go:90 +0x36 fp=0xc008c8b768 sp=0xc008c8b6e0 pc=0x47e036
syscall.openat(0xc5a49cac09d2a755?, {0xc00083e7e0?, 0x59e063a007cb5063?}, 0xc004179fe0?, 0x1b6)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:68 +0x94 fp=0xc008c8b7e0 sp=0xc008c8b768 pc=0x47acd4
syscall.Open(...)
	/usr/local/go/src/syscall/syscall_linux.go:232
os.openFileNolog({0xc00083e7e0, 0x63}, 0x41, 0x1b6)
	/usr/local/go/src/os/file_unix.go:216 +0x9b fp=0xc008c8b828 sp=0xc008c8b7e0 pc=0x4dd83b
os.OpenFile({0xc00083e7e0, 0x63}, 0x41, 0x7cb5063?)
	/usr/local/go/src/os/file.go:337 +0x45 fp=0xc008c8b860 sp=0xc008c8b828 pc=0x4db685
cmd/go/internal/cache.(*Cache).putIndexEntry(0xc000128150, {0x99, 0xff, 0x4f, 0x35, 0x51, 0x18, 0x27, 0x34, 0x55, ...}, ...)
	/usr/local/go/src/cmd/go/internal/cache/cache.go:400 +0x397 fp=0xc008c8bab8 sp=0xc008c8b860 pc=0x7c8237
cmd/go/internal/cache.(*Cache).put(0x100?, {0x99, 0xff, 0x4f, 0x35, 0x51, 0x18, 0x27, 0x34, 0x55, ...}, ...)
	/usr/local/go/src/cmd/go/internal/cache/cache.go:462 +0x1d8 fp=0xc008c8bb68 sp=0xc008c8bab8 pc=0x7c8798
cmd/go/internal/cache.(*Cache).Put(...)
	/usr/local/go/src/cmd/go/internal/cache/cache.go:432
cmd/go/internal/cache.(*Cache).PutBytes(0x0?, {0x99, 0xff, 0x4f, 0x35, 0x51, 0x18, 0x27, 0x34, 0x55, ...}, ...)
	/usr/local/go/src/cmd/go/internal/cache/cache.go:467 +0xd7 fp=0xc008c8bc00 sp=0xc008c8bb68 pc=0x7c8937
cmd/go/internal/modindex.openIndexPackage.func1()
	/usr/local/go/src/cmd/go/internal/modindex/read.go:227 +0x1ad fp=0xc008c8bd08 sp=0xc008c8bc00 pc=0x7d5e6d
cmd/go/internal/par.(*Cache).Do(0xe32bc0, {0x97f3e0, 0xc005028940}, 0xc00809fd90)
	/usr/local/go/src/cmd/go/internal/par/work.go:128 +0xc3 fp=0xc008c8bd50 sp=0xc008c8bd08 pc=0x5bb6e3
cmd/go/internal/modindex.openIndexPackage({0xc000030004, 0x32}, {0xc0066248f0, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:216 +0xd0 fp=0xc008c8bdc8 sp=0xc008c8bd50 pc=0x7d5c50
cmd/go/internal/modindex.GetPackage({0xc000030004, 0x32}, {0xc0066248f0, 0xa4})
	/usr/local/go/src/cmd/go/internal/modindex/read.go:145 +0x156 fp=0xc008c8be10 sp=0xc008c8bdc8 pc=0x7d5516
cmd/go/internal/modload.scanDir({0xc000030004?, 0x0?}, {0xc0066248f0, 0xa4}, 0x0?)
	/usr/local/go/src/cmd/go/internal/modload/load.go:2105 +0x4b fp=0xc008c8bf10 sp=0xc008c8be10 pc=0x816d8b
cmd/go/internal/modload.matchPackages.func4.1.1()
	/usr/local/go/src/cmd/go/internal/modload/search.go:133 +0x4c fp=0xc008c8bf60 sp=0xc008c8bf10 pc=0x8290cc
cmd/go/internal/par.(*Queue).Add.func1()
	/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d fp=0xc008c8bfe0 sp=0xc008c8bf60 pc=0x5bad7d
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc008c8bfe8 sp=0xc008c8bfe0 pc=0x468041
created by cmd/go/internal/par.(*Queue).Add
	/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x225

Findings

With GODEBUG=gofsystrace=1 I was able to trace down the issue. The repository in question contains a lot of non Go files and go mod tidy parses them all. Moving these files into a directory with the prefix _ resolves this issue - but I think this might be more a workaround for this particular case. Unfortunately this happens on a non-public repository, so I can't share it directly.

@gopherbot gopherbot added the compiler/runtime label Aug 3, 2022
@seankhliao
Copy link
Member

seankhliao commented Aug 3, 2022

What counts as "a lot"?

@florianl
Copy link
Contributor Author

florianl commented Aug 3, 2022

sorry for being unspecifc on this. "a lot" can be translated to ~491635 files.

@seankhliao seankhliao changed the title runtime: cannot allocate memory cmd/go: "runtime: cannot allocate memory" in workspaces with many non go files Aug 3, 2022
@seankhliao seankhliao added NeedsInvestigation GoCommand and removed compiler/runtime labels Aug 3, 2022
@seankhliao
Copy link
Member

seankhliao commented Aug 3, 2022

cc @bcmills @matloob

@bcmills bcmills added this to the Go1.20 milestone Aug 3, 2022
@bcmills
Copy link
Member

bcmills commented Aug 3, 2022

That's kind of amazing. 😅

Are these all regular files, or some directories, or some symlinks? Are they used as go:embed targets, or just ordinary non-.go file?

@bcmills
Copy link
Member

bcmills commented Aug 3, 2022

Does setting GODEBUG=goindex=0 restore the Go 1.18 behavior? (I would assume that it does, but if not that would be an interesting datapoint.)

@florianl
Copy link
Contributor Author

florianl commented Aug 3, 2022

Are these all regular files, or some directories, or some symlinks?

just regular text based files. and not used within go:embed

Does setting GODEBUG=goindex=0 restore the Go 1.18 behavior?

yes - it does. with GODEBUG=goindex=0 set go mod tidy just does its job as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GoCommand NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

4 participants