Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: pthread_create failed: Resource temporarily unavailable #51678

Open
sb10 opened this issue Mar 15, 2022 · 8 comments
Open

cmd/go: pthread_create failed: Resource temporarily unavailable #51678

sb10 opened this issue Mar 15, 2022 · 8 comments
Labels
GoCommand NeedsInvestigation
Milestone

Comments

@sb10
Copy link

@sb10 sb10 commented Mar 15, 2022

What version of Go are you using (go version)?

$ go version
go version go1.17.8 linux/amd64

Does this issue reproduce with the latest release?

This is the latest release. It reproduced on older 1.17.2 as well.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/nfs/users/nfs_s/sb10/.cache/go-build"
GOENV="/nfs/users/nfs_s/sb10/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/nfs/users/nfs_s/sb10/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/nfs/users/nfs_s/sb10/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/software/isg/languages/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/software/isg/languages/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.8"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="0"
GOMOD="/nfs/users/nfs_s/sb10/src/go/github.com/wtsi-ssg/wrstat/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1726564216=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Clone any package with a go.mod file, and try to go mod tidy it. Eg.

git clone https://github.com/pascaldekloe/jwt.git
cd jwt
go mod tidy

What did you expect to see?

No error.

What did you see instead?

runtime/cgo: pthread_create failed: Resource temporarily unavailable failures. With a package with no dependencies, the failure rate is quite low. With a package with many dependencies, the failure rate is 100%.

This suggests I'm close to a thread limit, but it doesn't seem like it:

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3090577
max locked memory       (kbytes, -l) 65536
max memory size         (kbytes, -m) 5000000
open files                      (-n) 131072
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 3090577
virtual memory          (kbytes, -v) 5000000
file locks                      (-x) unlimited

$ cat /proc/sys/kernel/pid_max
65536

$ cat /proc/sys/kernel/threads-max
6181154

My home directory and software installation is on a shared drive, and doing the exact same thing on other virtualised machines on the network is and has been fine for years (the issue isn't limited to go mod tidy, but also go build etc). I'm encountering this issue for the first time on a new machine on the network, which is "bare metal", but has been set up the same way as the virtualised machines with ansible. So same version of GCC AFAICT.

More problematic, other users on the same "bare metal" machine don't have the issue, suggesting there's some issue that is specific to my environment, but that should be the same on all machines.

I'm trying to figure out what the difference and cause is, but nothing obvious presents itself.

go mod tidy Output
runtime/cgo: pthread_create failed: Resource temporarily unavailable
runtime/cgo: pthread_create failed: Resource temporarily unavailable
SIGABRT: abort
PC=0x7ff2f00b5e87 m=61 sigcode=18446744073709551610

goroutine 0 [idle]:
runtime: unknown pc 0x7ff2f00b5e87
stack: frame={sp:0x7ff1c77fd7f0, fp:0x0} stack=[0x7ff1c6ffe270,0x7ff1c77fde70)
0x00007ff1c77fd6f0: 0x00007ff1c77fda90 0x00007ff1c77fd778
0x00007ff1c77fd700: 0x0000000000452a2d <runtime.funcspdelta+0x000000000000004d> 0x0000000000c50828
0x00007ff1c77fd710: 0x0000000000d398c0 0x0000000000021daf
0x00007ff1c77fd720: 0x0000000000000000 0x00007ff1c77fd8d0
0x00007ff1c77fd730: 0x0000000000000000 0x0000000000a61c68
0x00007ff1c77fd740: 0x0000000000000000 0x000000c000055698
0x00007ff1c77fd750: 0x00007ff1c77fd778 0x000000000041f5a5 <runtime.scanstack.func1+0x0000000000000025>
0x00007ff1c77fd760: 0x00007ff1c77fda90 0x00007ff1c77fdb70
0x00007ff1c77fd770: 0x0000000000d398c0 0x00007ff1c77fdae8
0x00007ff1c77fd780: 0x0000000000457a43 <runtime.gentraceback+0x0000000000000ac3> 0x0000000000c50828
0x00007ff1c77fd790: 0x0000000000d398c0 0x0000000000d398c0
0x00007ff1c77fd7a0: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd7b0: 0x0000000000000000 0x0000000000000100
0x00007ff1c77fd7c0: 0x0000000100000100 0x0000000100000000
0x00007ff1c77fd7d0: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd7e0: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd7f0: <0x0000000000000000 0x000000000000000e
0x00007ff1c77fd800: 0x000000c000941fd8 0x000000c00080efd8
0x00007ff1c77fd810: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd820: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd830: 0x000000000000000d 0x000000000000000f
0x00007ff1c77fd840: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd850: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd860: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd870: 0xfffffffe7fffffff 0xffffffffffffffff
0x00007ff1c77fd880: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd890: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd8a0: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd8b0: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd8c0: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd8d0: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd8e0: 0xffffffffffffffff 0xffffffffffffffff
runtime: unknown pc 0x7ff2f00b5e87
stack: frame={sp:0x7ff1c77fd7f0, fp:0x0} stack=[0x7ff1c6ffe270,0x7ff1c77fde70)
0x00007ff1c77fd6f0: 0x00007ff1c77fda90 0x00007ff1c77fd778
0x00007ff1c77fd700: 0x0000000000452a2d <runtime.funcspdelta+0x000000000000004d> 0x0000000000c50828
0x00007ff1c77fd710: 0x0000000000d398c0 0x0000000000021daf
0x00007ff1c77fd720: 0x0000000000000000 0x00007ff1c77fd8d0
0x00007ff1c77fd730: 0x0000000000000000 0x0000000000a61c68
0x00007ff1c77fd740: 0x0000000000000000 0x000000c000055698
0x00007ff1c77fd750: 0x00007ff1c77fd778 0x000000000041f5a5 <runtime.scanstack.func1+0x0000000000000025>
0x00007ff1c77fd760: 0x00007ff1c77fda90 0x00007ff1c77fdb70
0x00007ff1c77fd770: 0x0000000000d398c0 0x00007ff1c77fdae8
0x00007ff1c77fd780: 0x0000000000457a43 <runtime.gentraceback+0x0000000000000ac3> 0x0000000000c50828
0x00007ff1c77fd790: 0x0000000000d398c0 0x0000000000d398c0
0x00007ff1c77fd7a0: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd7b0: 0x0000000000000000 0x0000000000000100
0x00007ff1c77fd7c0: 0x0000000100000100 0x0000000100000000
0x00007ff1c77fd7d0: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd7e0: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd7f0: <0x0000000000000000 0x000000000000000e
0x00007ff1c77fd800: 0x000000c000941fd8 0x000000c00080efd8
0x00007ff1c77fd810: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd820: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd830: 0x000000000000000d 0x000000000000000f
0x00007ff1c77fd840: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd850: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd860: 0x0000000000000000 0x0000000000000000
0x00007ff1c77fd870: 0xfffffffe7fffffff 0xffffffffffffffff
0x00007ff1c77fd880: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd890: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd8a0: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd8b0: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd8c0: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd8d0: 0xffffffffffffffff 0xffffffffffffffff
0x00007ff1c77fd8e0: 0xffffffffffffffff 0xffffffffffffffff

goroutine 1 [chan receive]:
cmd/go/internal/modload.loadFromRoots({0xa72660, 0xc0000280f8}, {{{0x0, 0x0}, 0xc000230090, 0x1, {0x0, 0x0}, 0x1, 0x1, ...}, ...})
/usr/local/go/src/cmd/go/internal/modload/load.go:998 +0x6e5
cmd/go/internal/modload.LoadPackages({0xa72660, 0xc0000280f8}, {{0x0, 0x0}, 0xc000230090, 0x1, {0x0, 0x0}, 0x1, 0x1, ...}, ...)
/usr/local/go/src/cmd/go/internal/modload/load.go:329 +0x328
cmd/go/internal/modcmd.runTidy({0xa72660, 0xc0000280f8}, 0xc000030468, {0xc0000201a0, 0x4, 0x35})
/usr/local/go/src/cmd/go/internal/modcmd/tidy.go:114 +0x1b7
main.invoke(0xd82680, {0xc0000201a0, 0x1, 0x1})
/usr/local/go/src/cmd/go/main.go:216 +0x2f6
main.main()
/usr/local/go/src/cmd/go/main.go:173 +0x78e

goroutine 217 [semacquire]:
strings.(*Builder).grow(...)
/usr/local/go/src/strings/builder.go:68
strings.(*Builder).Grow(...)
/usr/local/go/src/strings/builder.go:82
strings.Join({0xc000a17c50, 0xc00099c176, 0x3}, {0xa5d5f0, 0x2})
/usr/local/go/src/strings/strings.go:434 +0x125
path/filepath.join({0xc000a17c50, 0x5, 0xe})
/usr/local/go/src/path/filepath/path_unix.go:46 +0x85
path/filepath.Join(...)
/usr/local/go/src/path/filepath/path.go:213
internal/goroot.IsStandardPackage({0xc00002a420, 0x0}, {0x995031, 0x0}, {0xc00099c171, 0x0})
/usr/local/go/src/internal/goroot/gc.go:23 +0x8f
cmd/go/internal/modload.importFromModules({0xa72660, 0xc0000280f8}, {0xc00099c171, 0xe}, 0xc00011c280, 0xc000112480)
/usr/local/go/src/cmd/go/internal/modload/import.go:259 +0x1ea
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc000232370)
/usr/local/go/src/cmd/go/internal/modload/load.go:1592 +0x30a
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 15 [semacquire]:
os.newFile(0x1f, {0xc0006199c0, 0x3e}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc0006199c0, 0xc0006199c0}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc0006199c0, 0x3e}, 0x0, 0x3e)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc0006199c0, 0xc000812b88}, 0xc0000282a9, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc000198200, 0x5, 0x2}, 0x2, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc00068e240, 0x2d}, 0x2d)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc00068e240, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc0005444d0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 402 [semacquire]:
os.newFile(0x13, {0xc00002c2d0, 0x4b}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc00002c2d0, 0xc00002c2d0}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc00002c2d0, 0x4b}, 0x0, 0x4b)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc00002c2d0, 0xc000511b88}, 0xc000866091, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc000ba4200, 0xa, 0xc000230090}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc00019a300, 0x3b}, 0x3b)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc00019a300, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc000544630)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 307 [semacquire]:
os.newFile(0x32, {0xc000a32080, 0x3a}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc000a32080, 0xc000a32080}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc000a32080, 0x3a}, 0x0, 0x3a)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc000a32080, 0xc0001c9aa8}, 0x418f6d, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc000681000, 0x25, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc000692100, 0x34}, 0x34)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000692100, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc0005a4210)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 21 [semacquire]:
os.newFile(0x1b, {0xc000926b00, 0x3c}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc000926b00, 0xc000926b00}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc000926b00, 0x3c}, 0x0, 0x3c)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc000926b00, 0xc000139b58}, 0xc0005ac146, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc00074e400, 0x15, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc0002bec00, 0x2c}, 0x2c)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc0002bec00, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc00015f130)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 24 [semacquire]:
os.newFile(0x19, {0xc000026a00, 0x3d}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc000026a00, 0xc000026a00}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc000026a00, 0x3d}, 0x0, 0x3d)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc000026a00, 0xc0006a1b88}, 0xc000a240e1, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc0007a4100, 0xd, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc0006178f0, 0x22}, 0x22)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc0006178f0, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc00015f340)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 27 [semacquire]:
os.newFile(0x34, {0xc0002be2a0, 0x30}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc0002be2a0, 0xc0002be2a0}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc0002be2a0, 0x30}, 0x0, 0x30)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc0002be2a0, 0xc000480000}, 0xc00016c100, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc0002b6800, 0x47, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc000298060, 0x23}, 0x23)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000298060, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc00015f550)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 28 [semacquire]:
os.newFile(0xc, {0xc00089ff40, 0x35}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc00089ff40, 0xc00089ff40}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc00089ff40, 0x35}, 0x0, 0x35)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc00089ff40, 0xc000913b88}, 0xc000b1e091, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc000180800, 0x30, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc0002bec90, 0x27}, 0x27)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc0002bec90, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc00015f600)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 29 [semacquire]:
os.newFile(0x1d, {0xc000816360, 0x2e}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc000816360, 0xc000816360}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc000816360, 0x2e}, 0x0, 0x2e)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc000816360, 0xc000495b58}, 0xc0001a60cb, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc000700c00, 0x31, 0xc000230090}, 0x4, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc00058e4e0, 0x27}, 0x27)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc00058e4e0, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc00015f6b0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 32 [GC assist wait]:
bufio.NewReaderSize(...)
/usr/local/go/src/bufio/bufio.go:57
bufio.NewReader(...)
/usr/local/go/src/bufio/bufio.go:63
cmd/go/internal/imports.ReadImports({0xa65e80, 0xc000c2e0a0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/read.go:230 +0x9e
cmd/go/internal/imports.scanFiles({0xc0002a8400, 0x18, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:58 +0x25f
cmd/go/internal/imports.ScanDir({0xc0003967e0, 0x23}, 0x23)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc0003967e0, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc00015f8c0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 33 [semacquire]:
os.newFile(0x22, {0xc0003a4e40, 0x37}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc0003a4e40, 0xc0003a4e40}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc0003a4e40, 0x37}, 0x0, 0x37)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc0003a4e40, 0xc00051a000}, 0xc00082c100, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc000380800, 0x21, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc000396060, 0x23}, 0x23)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000396060, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc00015f970)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 135 [semacquire]:
os.newFile(0x28, {0xc0003b2b00, 0x37}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc0003b2b00, 0xc0003b2b00}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc0003b2b00, 0x37}, 0x0, 0x37)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc0003b2b00, 0xc00060eb88}, 0xc0006b4101, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc000ba4400, 0xa, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc000a8e330, 0x29}, 0x29)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000a8e330, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc000444bb0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 38 [semacquire]:
os.newFile(0x1a, {0xc0002a6180, 0x32}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc0002a6180, 0xc0002a6180}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc0002a6180, 0x32}, 0x0, 0x32)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc0002a6180, 0xc00069fb58}, 0xc00032e171, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc0002b6000, 0x72, 0xc000230090}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc000616060, 0x21}, 0x21)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000616060, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc00015fd90)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 434 [semacquire]:
os.newFile(0x11, {0xc00069c190, 0x4f}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc00069c190, 0xc00069c190}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc00069c190, 0x4f}, 0x0, 0x4f)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc00069c190, 0x0}, 0xc000910ad0, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc0006b0200, 0x9, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc0006ee190, 0x49}, 0x49)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc0006ee190, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc0004456b0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 87 [GC assist wait]:
os.newFile(0x31, {0xc00041c280, 0x37}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc00041c280, 0xc00041c280}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc00041c280, 0x37}, 0x0, 0x37)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc00041c280, 0xc000a14b58}, 0xc0007661a1, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc00003e400, 0x14, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc000616030, 0x26}, 0x26)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000616030, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc000444420)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 98 [semacquire]:
os.newFile(0x20, {0xc00078e4b0, 0x2c}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc00078e4b0, 0xc00078e4b0}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc00078e4b0, 0x2c}, 0x0, 0x2c)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc00078e4b0, 0xc0001cfb58}, 0xc000bc406b, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc000c08200, 0xf, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc00018e2d0, 0x24}, 0x24)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc00018e2d0, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc000544000)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 99 [semacquire]:
os.newFile(0x1c, {0xc000a8e210, 0x2d}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc000a8e210, 0xc000a8e210}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc000a8e210, 0x2d}, 0x0, 0x2d)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc000a8e210, 0xc000c0a038}, 0xc000340100, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc00074e600, 0x11, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc00002b7d0, 0x23}, 0x23)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc00002b7d0, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc0005440b0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 245 [semacquire]:
bufio.NewReaderSize(...)
/usr/local/go/src/bufio/bufio.go:57
bufio.NewReader(...)
/usr/local/go/src/bufio/bufio.go:63
cmd/go/internal/imports.ReadImports({0xa65e80, 0xc00062c088}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/read.go:230 +0x9e
cmd/go/internal/imports.scanFiles({0xc00003e600, 0x1a, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:58 +0x25f
cmd/go/internal/imports.ScanDir({0xc000816300, 0x2b}, 0x2b)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000816300, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc00015e4d0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 246 [GC assist wait]:
syscall.ByteSliceFromString(...)
/usr/local/go/src/syscall/syscall.go:53
syscall.BytePtrFromString(...)
/usr/local/go/src/syscall/syscall.go:69
syscall.openat(0x27, {0xc00027e6c0, 0x0}, 0x0, 0x0)
/usr/local/go/src/syscall/zsyscall_linux_amd64.go:65 +0x85
syscall.Open(...)
/usr/local/go/src/syscall/syscall_linux.go:155
os.openFileNolog({0xc00027e6c0, 0xc00027e6c0}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:217 +0x9b
os.OpenFile({0xc00027e6c0, 0x3b}, 0x0, 0x3b)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc00027e6c0, 0xc00093db88}, 0xc000baa411, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc0004f0000, 0x109, 0xc000230090}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc0003960c0, 0x26}, 0x26)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc0003960c0, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc00015e630)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 213 [semacquire]:
bufio.NewReaderSize(...)
/usr/local/go/src/bufio/bufio.go:57
bufio.NewReader(...)
/usr/local/go/src/bufio/bufio.go:63
cmd/go/internal/imports.ReadImports({0xa65e80, 0xc0003fa058}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/read.go:230 +0x9e
cmd/go/internal/imports.scanFiles({0xc0007a4200, 0xb, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:58 +0x25f
cmd/go/internal/imports.ScanDir({0xc0006160c0, 0x28}, 0x28)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc0006160c0, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc000444840)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 214 [GC assist wait]:
cmd/go/internal/imports.(*importReader).readByte(0xc0005b2460)
/usr/local/go/src/cmd/go/internal/imports/read.go:60 +0x71
cmd/go/internal/imports.(*importReader).peekByte(0xc0005b2460, 0x1)
/usr/local/go/src/cmd/go/internal/imports/read.go:106 +0x1b4
cmd/go/internal/imports.(*importReader).readKeyword(0xc0005b2460, {0x997337, 0x7})
/usr/local/go/src/cmd/go/internal/imports/read.go:139 +0x31
cmd/go/internal/imports.ReadImports({0xa65e80, 0xc000480090}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/read.go:232 +0x191
cmd/go/internal/imports.scanFiles({0xc0001ab000, 0xc0, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:58 +0x25f
cmd/go/internal/imports.ScanDir({0xc00058e1b0, 0x22}, 0x22)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc00058e1b0, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc0004448f0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 215 [semacquire]:
cmd/go/internal/imports.newImportReader(0xc000795140)
/usr/local/go/src/cmd/go/internal/imports/read.go:36 +0x71
cmd/go/internal/imports.ReadImports({0xa65e80, 0xc0001e2090}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/read.go:230 +0x17b
cmd/go/internal/imports.scanFiles({0xc00016a000, 0x1b7, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:58 +0x25f
cmd/go/internal/imports.ScanDir({0xc000410120, 0x26}, 0x26)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000410120, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc0004449a0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 342 [semacquire]:
os.newFile(0x7, {0xc00089f300, 0x3b}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc00089f300, 0xc00089f300}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc00089f300, 0x3b}, 0x0, 0x3b)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc00089f300, 0xc000515b88}, 0xc00082c13b, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc000960080, 0x8, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc000b96180, 0x2c}, 0x2c)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000b96180, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc0001ba790)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 216 [semacquire]:
strings.(*Builder).grow(...)
/usr/local/go/src/strings/builder.go:68
strings.(*Builder).Grow(...)
/usr/local/go/src/strings/builder.go:82
strings.Join({0xc0000f5c50, 0xc00082c23a, 0x3}, {0xa5d5f0, 0x2})
/usr/local/go/src/strings/strings.go:434 +0x125
path/filepath.join({0xc0000f5c50, 0x9, 0xe})
/usr/local/go/src/path/filepath/path_unix.go:46 +0x85
path/filepath.Join(...)
/usr/local/go/src/path/filepath/path.go:213
internal/goroot.IsStandardPackage({0xc00002a420, 0x0}, {0x995031, 0x0}, {0xc00082c231, 0x0})
/usr/local/go/src/internal/goroot/gc.go:23 +0x8f
cmd/go/internal/modload.importFromModules({0xa72660, 0xc0000280f8}, {0xc00082c231, 0xe}, 0xc00011c280, 0xc000112480)
/usr/local/go/src/cmd/go/internal/modload/import.go:259 +0x1ea
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc0002322c0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1592 +0x30a
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 235 [semacquire]:
os.newFile(0x21, {0xc000927640, 0x36}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc000927640, 0xc000927640}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc000927640, 0x36}, 0x0, 0x36)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc000927640, 0xc000592000}, 0xc0006b4000, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc000ba4300, 0xa, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc0007161b0, 0x26}, 0x26)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc0007161b0, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc000444fd0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 237 [semacquire]:
os.newFile(0x2e, {0xc0009a0500, 0x33}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc0009a0500, 0xc0009a0500}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc0009a0500, 0x33}, 0x0, 0x33)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc0009a0500, 0xc000513b88}, 0xc00082c081, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc0002a8600, 0x12, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc000396120, 0x25}, 0x25)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000396120, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc0004451e0)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 238 [semacquire]:
cmd/go/internal/imports.scanFiles({0xc000960100, 0x8, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:89 +0x4ea
cmd/go/internal/imports.ScanDir({0xc000a18150, 0x27}, 0x27)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc000a18150, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc000445290)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 44 [semacquire]:
os.newFile(0x30, {0xc000564a00, 0x34}, 0x1)
/usr/local/go/src/os/file_unix.go:128 +0x65
os.openFileNolog({0xc000564a00, 0xc000564a00}, 0x0, 0x0)
/usr/local/go/src/os/file_unix.go:241 +0x170
os.OpenFile({0xc000564a00, 0x34}, 0x0, 0x34)
/usr/local/go/src/os/file.go:338 +0x45
cmd/go/internal/fsys.OpenFile({0xc000564a00, 0xc000911b58}, 0xc0003ae08b, 0x0)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:357 +0x39c
cmd/go/internal/fsys.Open(...)
/usr/local/go/src/cmd/go/internal/fsys/fsys.go:330
cmd/go/internal/imports.scanFiles({0xc0004c2000, 0x15, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:53 +0x21e
cmd/go/internal/imports.ScanDir({0xc00030e450, 0x2b}, 0x2b)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc00030e450, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc0001bb130)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

goroutine 482 [GC assist wait]:
cmd/go/internal/imports.(*importReader).readByte(0xc00052a410)
/usr/local/go/src/cmd/go/internal/imports/read.go:60 +0x71
cmd/go/internal/imports.(*importReader).peekByte(0xc00052a410, 0x1)
/usr/local/go/src/cmd/go/internal/imports/read.go:106 +0x1b4
cmd/go/internal/imports.ReadImports({0xa65e80, 0xc00065a080}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/read.go:234 +0x1aa
cmd/go/internal/imports.scanFiles({0xc000bb6400, 0x11, 0x0}, 0x0, 0x0)
/usr/local/go/src/cmd/go/internal/imports/scan.go:58 +0x25f
cmd/go/internal/imports.ScanDir({0xc00018e1e0, 0x29}, 0x29)
/usr/local/go/src/cmd/go/internal/imports/scan.go:40 +0xa5
cmd/go/internal/modload.scanDir({0xc00018e1e0, 0xc0000280f8}, 0xc00002e260)
/usr/local/go/src/cmd/go/internal/modload/load.go:1991 +0x25
cmd/go/internal/modload.(*loader).load(0xc00015e6e0, {0xa72660, 0xc0000280f8}, 0xc000ba0160)
/usr/local/go/src/cmd/go/internal/modload/load.go:1622 +0x565
cmd/go/internal/modload.(*loader).pkg.func1.1()
/usr/local/go/src/cmd/go/internal/modload/load.go:1389 +0x29
cmd/go/internal/par.(*Queue).Add.func1()
/usr/local/go/src/cmd/go/internal/par/queue.go:58 +0x7d
created by cmd/go/internal/par.(*Queue).Add
/usr/local/go/src/cmd/go/internal/par/queue.go:56 +0x238

rax 0x0
rbx 0x7ff2f0463840
rcx 0x7ff2f00b5e87
rdx 0x0
rdi 0x2
rsi 0x7ff1c77fd7f0
rbp 0xa6d7a2
rsp 0x7ff1c77fd7f0
r8 0x0
r9 0x7ff1c77fd7f0
r10 0x8
r11 0x246
r12 0x7ff258000da0
r13 0x203000
r14 0xc000001a00
r15 0x7ff2c96721d0
rip 0x7ff2f00b5e87
rflags 0x246
cs 0x33
fs 0x0
gs 0x0

@sb10 sb10 closed this as completed Mar 15, 2022
@sb10
Copy link
Author

@sb10 sb10 commented Mar 16, 2022

This problem went away by itself after doing a fresh login to the bare metal server, so I closed the issue.
Then it started happening again after another fresh login.

The only thing that changed was that ansible altered some things on the server, but it isn't obvious why those changes could result in this issue.

Any ideas what to look for or how to debug?

@sb10 sb10 reopened this Mar 16, 2022
@heschi
Copy link
Contributor

@heschi heschi commented Mar 16, 2022

This is much more likely to be a problem with your machine's setup than an issue with Go itself. You may want to look at some of the resources listed in https://github.com/golang/go/wiki/Questions.

@heschi heschi added this to the Backlog milestone Mar 16, 2022
@heschi heschi added the NeedsInvestigation label Mar 16, 2022
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Mar 16, 2022

The error means that pthread_create is consistently failing. I say "consistently" because if pthread_create fails with "resource temporarily unavailable" it is retried up to 20 times, with an increasing delay between each retry. So basically this means that your system is maxed out in some way: it's out of memory, or there are too many running threads, or too many running processes, or something.

This is unlikely to be a problem with the Go language or standard library. It's much more likely to be a problem with your Go program--perhaps it has a memory leak--or a problem with other processes running on your system.

@ianlancetaylor ianlancetaylor added the WaitingForInfo label Mar 16, 2022
@sb10
Copy link
Author

@sb10 sb10 commented Mar 16, 2022

This is unlikely to be a problem with the Go language or standard library. It's much more likely to be a problem with your Go program--perhaps it has a memory leak--or a problem with other processes running on your system.

The "Go program" in question is go mod tidy, so it's not "my" program.

It is also not an issue with the physical machine per say. It can't be an issue with too little memory, too many running threads or too many running processes, because I have a tmux session running on the machine from yesterday where go mod tidy works 100% of the time, while a fresh login to the same machine gives a 100% failure rate for go mod tidy.

So it's some kind of environment setting that triggers the issue, but taking a diff of go env and ulimit -a doesn't reveal anything significantly different between the 2 different sessions on the same machine that aren't also shared by another machine where it works.

Lots of other people are using the affected machine, and no one else is reporting issues with it. I don't have issues running any other software in my fresh login session to the machine. The only thing suffering from the issue is go, hence the bug report. Is there any way I can debug further?

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Mar 17, 2022

Thanks, I didn't understand that it was go mod tidy that is crashing.

I would start with simply running top or a similar program while running go mod tidy on a different terminal, and see what it is doing. Another step might be stace -f -e clone go mod tidy and see how many threads it creates. These would just be to understand the scope of the problem.

@sb10
Copy link
Author

@sb10 sb10 commented Mar 17, 2022

The go mod tidy call completes pretty quickly, so most of the time top doesn't show it running, but here's where I managed to catch it. top was running on a terminal from my tmux session, go mod tidy was running from my fresh login session. (I wasn't able to do the reverse and catch a successful run of go mod tidy in the tmux session; I think when it's successful it's faster.)

top -Hu sb10 Output
top - 09:55:59 up 2 days, 18:03, 81 users,  load average: 1.66, 2.05, 1.63
Threads: 1871 total,   4 running, 1461 sleeping,   5 stopped,   1 zombie
%Cpu(s):  1.6 us,  1.4 sy,  0.0 ni, 96.9 id,  0.0 wa,  0.0 hi,  0.1 si,  0.0 st
KiB Mem : 79123840+total, 67656428+free, 56342764 used, 58331340 buff/cache
KiB Swap: 18250417+total, 18250417+free,        0 used. 72989414+avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
60828 sb10 20 0 54168 6428 3744 R 2.3 0.0 0:01.01 top
62305 sb10 20 0 4992040 24456 9756 S 2.3 0.0 0:00.07 go
62307 sb10 20 0 4992040 24456 9756 S 2.3 0.0 0:00.07 go
62308 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62309 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62312 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62316 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62317 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62318 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62320 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62323 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62324 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62325 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62328 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62331 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62338 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62354 sb10 20 0 4992040 24456 9756 R 2.0 0.0 0:00.06 go
62360 sb10 20 0 4992040 24456 9756 S 2.0 0.0 0:00.06 go
62311 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62313 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62314 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62315 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62322 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62326 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62330 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62333 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62334 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62335 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62336 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62340 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62341 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62343 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62344 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62345 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62348 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62349 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62352 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62358 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62366 sb10 20 0 4992040 24456 9756 S 1.6 0.0 0:00.05 go
62319 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62321 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62329 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62332 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62337 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62339 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62342 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62346 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62347 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62353 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62359 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62368 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62369 sb10 20 0 4992040 24456 9756 S 1.3 0.0 0:00.04 go
62327 sb10 20 0 4992040 24456 9756 S 1.0 0.0 0:00.03 go
62355 sb10 20 0 4992040 24456 9756 S 1.0 0.0 0:00.03 go
62362 sb10 20 0 4992040 24456 9756 S 1.0 0.0 0:00.03 go
62363 sb10 20 0 4992040 24456 9756 S 1.0 0.0 0:00.03 go
62364 sb10 20 0 4992040 24456 9756 S 1.0 0.0 0:00.03 go
62365 sb10 20 0 4992040 24456 9756 S 1.0 0.0 0:00.03 go
62370 sb10 20 0 4992040 24456 9756 S 0.7 0.0 0:00.02 go
62371 sb10 20 0 4992040 24456 9756 S 0.7 0.0 0:00.02 go
62373 sb10 20 0 4992040 24456 9756 S 0.7 0.0 0:00.02 go
52031 sb10 20 0 203144 5180 3572 R 0.3 0.0 0:00.71 sshd
3259 sb10 20 0 203144 5304 3692 S 0.0 0.0 0:00.06 sshd
3262 sb10 20 0 33524 5912 3800 S 0.0 0.0 0:00.09 bash
3928 sb10 20 0 21232 3520 3164 S 0.0 0.0 0:00.07 tmux: client
19096 sb10 20 0 33652 6096 3828 S 0.0 0.0 0:00.47 bash
22288 sb10 20 0 60520 6316 5604 S 0.0 0.0 0:00.30 ssh
24739 sb10 20 0 77972 8940 6600 S 0.0 0.0 0:48.64 systemd
24740 sb10 20 0 357112 3364 64 S 0.0 0.0 0:00.00 (sd-pam)
39520 sb10 20 0 32264 6284 3176 S 0.0 0.0 0:03.07 tmux: server
39521 sb10 20 0 33656 6124 3812 S 0.0 0.0 0:00.71 bash
52040 sb10 20 0 33524 5888 3756 S 0.0 0.0 0:00.14 bash
54050 sb10 20 0 33520 5852 3660 S 0.0 0.0 0:00.36 bash
62306 sb10 20 0 4992040 24456 9756 S 0.0 0.0 0:00.00 go
62310 sb10 20 0 4992040 24456 9756 S 0.0 0.0 0:00.00 go
62372 sb10 20 0 4992040 24456 9756 S 0.0 0.0 0:00.00 go
62374 sb10 20 0 4992040 24456 9756 S 0.0 0.0 0:00.00 go
62375 sb10 20 0 4992040 24456 9756 S 0.0 0.0 0:00.00 go

From the fresh session where it fails:

$ strace -f -e clone go mod tidy -compat=1.17 2> out
$ grep -c attached out
65

Complete output is here: https://pastebin.com/9m4Rsf1J

From the tmux session on the same machine where it works:

$ strace -f -e clone go mod tidy -compat=1.17 2> out
$ grep -c attached out
62

Complete output is here: https://pastebin.com/Ry8RW9Wc

The numbers vary on both on repeated runs, but typically in the 50-70 range. Same when grepping for 'clone'.

@seankhliao seankhliao added WaitingForInfo and removed WaitingForInfo labels Jun 12, 2022
@gopherbot
Copy link

@gopherbot gopherbot commented Jul 12, 2022

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@sb10
Copy link
Author

@sb10 sb10 commented Jul 12, 2022

Info provided, closed in error.

@ianlancetaylor ianlancetaylor changed the title runtime/cgo: pthread_create failed: Resource temporarily unavailable cmd/go: pthread_create failed: Resource temporarily unavailable Jul 13, 2022
@ianlancetaylor ianlancetaylor added GoCommand and removed WaitingForInfo labels Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GoCommand NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

5 participants