Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: darwin-amd64 LUCI builders are sometimes too slow #65040

Open
bcmills opened this issue Jan 9, 2024 · 24 comments
Open

x/build: darwin-amd64 LUCI builders are sometimes too slow #65040

bcmills opened this issue Jan 9, 2024 · 24 comments
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin
Milestone

Comments

@bcmills
Copy link
Member

bcmills commented Jan 9, 2024

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")

Go version

aba18d5

Output of go env in your module/workspace:

N/A

What did you do?

Check status of https://ci.chromium.org/p/golang/g/go-gotip/console.

What did you see happen?

Multiple failures involving timeouts on the darwin-amd64_13 builder:

What did you expect to see?

No timeouts: go test -short cmd/go takes <20s to run locally, so either the builders should be faster, or they should set a GO_TEST_TIMEOUT_SCALE that is long enough to reliably run the tests.

The only timeouts on the builders should be for (a) true deadlocks, and (b) tests that take a similarly long time (approaching 3 minutes) when run locally.

@bcmills bcmills added OS-Darwin Builders x/build issues (builders, bots, dashboards) labels Jan 9, 2024
@gopherbot gopherbot added this to the Unreleased milestone Jan 9, 2024
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 9, 2024
@dmitshur
Copy link
Contributor

dmitshur commented Jan 9, 2024

The "darwin-amd64-13" builder on the old dashboard doesn't have GO_TEST_TIMEOUT_SCALE in its configuration, so this isn't a case of that not being ported over. Perhaps our macOS amd64 13 builder configuration is less powerful? If we know by how much, it makes sense to set a corresponding GO_TEST_TIMEOUT_SCALE to compensate and avoid timeouts.

Another possibility is that we're running more tests on Mac builders in LUCI than we are on the old dashboard. For one, the old dashboard uses the macTestPolicy test policy, which attempts to skips some expensive portable tests.

CC @prattmic, @mknyszek.

@dmitshur dmitshur self-assigned this Jan 9, 2024
@dmitshur
Copy link
Contributor

dmitshur commented Jan 9, 2024

We discussed and looked into this a bit.

This turns out not to be a consistent slowness: looking at https://ci.chromium.org/ui/p/golang/builders/ci/gotip-darwin-amd64_13 briefly shows many builds complete successfully in 15-20 min, with some taking 10 or 30 min, but the failures are taking an hour or longer and reaching timeout.

This builder at this time has two different providers. The two recent timeouts happened on the same darwin-amd64-13--ac2a2248-1530-... bot.

@dmitshur dmitshur changed the title x/build: darwin-amd64_13 LUCI builder is too slow x/build: darwin-amd64_13 LUCI builder is sometimes too slow or stalls Jan 9, 2024
@prattmic
Copy link
Member

Update on this. I took a look at median, worst case, etc build times for the main repo builder by bot hostname and there definitely seems to be a lot of variance between different hosts: nearly 4x difference in median runtime between the fastest and slowest hosts.

My first thought was slow network on some hosts, so I eyeballed the difference between a run on the fastest and slowest hosts and found that the slowdown seems to impact pretty much all packages. e.g.,

cmd/go 139s -> 540s
cmd/compile/internal/types2 15s -> 55s (pretty sure there is no network here)

Even small packages, like

cmd/gofmt 0.064s -> 0.79s

Some are similar though

crypto/md5 0.021s -> 0.024s

So it seems unrelated to network.

That slowest machine is https://chromium-swarm.appspot.com/bot?id=darwin-amd64-13--ac2a2248-1530-4adb-bb91-ae854aa7c79c.golang.ca.macservice.goog. Interestingly, it seems to be much faster today. Previously x/tools was timing out, now it is ~15m. x/build is faster, etc.

This definitely still needs more investigation.

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `gotip-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded`)
2023-12-08 00:50 gotip-darwin-amd64-nocgo go@4bf1ca4b cmd/fix.TestRewrite/jni.ArrayElement (log)
=== RUN   TestRewrite/jni.ArrayElement
panic: test timed out after 9m0s
running tests:
	TestRewrite (9m0s)
	TestRewrite/jni.ArrayElement (5s)

goroutine 48 gp=0xc0000076c0 m=4 mp=0xc00004b208 [running]:
panic({0xc2ec460?, 0xc0001913e0?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:779 +0x158 fp=0xc00016ef10 sp=0xc00016ee60 pc=0xc161398
testing.(*M).startAlarm.func1()
...
cmd/fix.TestRewrite.func3(0xc0001a0ea0)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/cmd/fix/main_test.go:116 +0xde fp=0xc000179f70 sp=0xc000179eb0 pc=0xc296b1e
testing.tRunner(0xc0001a0ea0, 0xc000191260)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1689 +0xfb fp=0xc000179fc0 sp=0xc000179f70 pc=0xc1fe63b
testing.(*T).Run.gowrap1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x25 fp=0xc000179fe0 sp=0xc000179fc0 pc=0xc1ff665
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000179fe8 sp=0xc000179fe0 pc=0xc19ad61
created by testing.(*T).Run in goroutine 18
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x390
2024-01-08 16:00 gotip-darwin-amd64_13 go@881869dd cmd/go.TestScript (log)
=== RUN   TestScript/cgo_path_space
=== PAUSE TestScript/cgo_path_space
=== CONT  TestScript/cgo_path_space
    script_test.go:132: 2024-01-08T17:08:45Z
    script_test.go:134: $WORK=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-2324303667/tmpdir1305803261/cgo_path_space2972876599
    script_test.go:156: 
        PATH=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-2324303667/tmpdir1305803261/testbin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/cache/tools/bin:/Users/swarming/.swarming/w/ir/bbagent_utility_packages:/Users/swarming/.swarming/w/ir/bbagent_utility_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages:/Users/swarming/.swarming/w/ir/cipd_bin_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3/bin:/Users/swarming/.swarming/w/ir/cache/cipd_client:/Users/swarming/.swarming/w/ir/cache/cipd_client/bin:/Users/swarming/.swarming/cipd_cache/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
        HOME=/no-home
        CCACHE_DISABLE=1
        GOARCH=amd64
...
        r14    0xd2010000d300
        r15    0x8400
        rip    0x7ff813a840de
        rflags 0x247
        cs     0x7
        fs     0x0
        gs     0x0
        [exit status 2]
    script_test.go:156: FAIL: testdata/script/cgo_path_space.txt:23: context deadline exceeded
--- FAIL: TestScript/cgo_path_space (10.40s)
2024-01-09 15:29 gotip-darwin-amd64_13 go@aba18d5b cmd/fix.TestRewrite (log)
=== RUN   TestRewrite/jni.MapValue
panic: test timed out after 9m0s
running tests:
	TestRewrite (9m0s)
	TestRewrite/jni.MapValue (20s)

goroutine 94 gp=0xc000006c40 m=5 mp=0xc000100908 [running]:
panic({0xc4a8460?, 0xc0004dc200?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:779 +0x158 fp=0xc0004f8f10 sp=0xc0004f8e60 pc=0xc31d338
testing.(*M).startAlarm.func1()
...
cmd/fix.TestRewrite.func3(0xc0001d04e0)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/cmd/fix/main_test.go:145 +0x2e8 fp=0xc000169f70 sp=0xc000169eb0 pc=0xc452dc8
testing.tRunner(0xc0001d04e0, 0xc000026260)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1689 +0xfb fp=0xc000169fc0 sp=0xc000169f70 pc=0xc3ba61b
testing.(*T).Run.gowrap1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x25 fp=0xc000169fe0 sp=0xc000169fc0 pc=0xc3bb645
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000169fe8 sp=0xc000169fe0 pc=0xc356d41
created by testing.(*T).Run in goroutine 34
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x390
2024-01-09 15:29 gotip-darwin-amd64_13 go@aba18d5b cmd/go.TestScript (log)
=== RUN   TestScript/build_pgo_auto_multi
=== PAUSE TestScript/build_pgo_auto_multi
=== CONT  TestScript/build_pgo_auto_multi
    script_test.go:132: 2024-01-09T16:21:55Z
    script_test.go:134: $WORK=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-2333596952/tmpdir3949127613/build_pgo_auto_multi2572497361
    script_test.go:156: 
        PATH=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-2333596952/tmpdir3949127613/testbin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/cache/tools/bin:/Users/swarming/.swarming/w/ir/bbagent_utility_packages:/Users/swarming/.swarming/w/ir/bbagent_utility_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages:/Users/swarming/.swarming/w/ir/cipd_bin_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3/bin:/Users/swarming/.swarming/w/ir/cache/cipd_client:/Users/swarming/.swarming/w/ir/cache/cipd_client/bin:/Users/swarming/.swarming/cipd_cache/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
        HOME=/no-home
        CCACHE_DISABLE=1
        GOARCH=amd64
...
        > ! stderr 'compile.*-pgoprofile=.*nopgo(/|\\\\)nopgo_test\.go'
        # test-only dependencies also have profiles attached (2.137s)
        > stderr 'compile.*-pgoprofile=.*a(/|\\\\)default\.pgo.*testdep(/|\\\\)testdep\.go'
        matched: /Users/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/darwin_amd64/compile -o $WORK/b091/_pkg_.a -trimpath "$WORK/b091=>" -p test/testdep -lang=go1.20 -complete -buildid 5e_-LXiIG4WlgUhcawtb/5e_-LXiIG4WlgUhcawtb -pgoprofile=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-2333596952/tmpdir3949127613/build_pgo_auto_multi2572497361/gopath/src/a/default.pgo -c=4 -shared -nolocalimports -importcfg $WORK/b091/importcfg -pack ./testdep/testdep.go
        > stderr 'compile.*-pgoprofile=.*b(/|\\\\)default\.pgo.*testdep(/|\\\\)testdep\.go'
        matched: /Users/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/darwin_amd64/compile -o $WORK/b244/_pkg_.a -trimpath "$WORK/b244=>" -p test/testdep -lang=go1.20 -complete -buildid 5e_-LXiIG4WlgUhcawtb/5e_-LXiIG4WlgUhcawtb -pgoprofile=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-2333596952/tmpdir3949127613/build_pgo_auto_multi2572497361/gopath/src/b/default.pgo -c=4 -shared -nolocalimports -importcfg $WORK/b244/importcfg -pack ./testdep/testdep.go
        > stderr 'compile.*-pgoprofile=.*a(/|\\\\)default\.pgo.*testdep2(/|\\\\)testdep2\.go'
        matched: /Users/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/darwin_amd64/compile -o $WORK/b092/_pkg_.a -trimpath "$WORK/b092=>" -p test/testdep2 -lang=go1.20 -complete -buildid 2nksUir2tqGHKBXsGUmy/2nksUir2tqGHKBXsGUmy -pgoprofile=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-2333596952/tmpdir3949127613/build_pgo_auto_multi2572497361/gopath/src/a/default.pgo -c=4 -shared -nolocalimports -importcfg $WORK/b092/importcfg -pack ./testdep2/testdep2.go
    script_test.go:156: FAIL: testdata/script/build_pgo_auto_multi.txt:51: context deadline exceeded
--- FAIL: TestScript/build_pgo_auto_multi (21.81s)
2024-01-10 17:35 gotip-darwin-amd64_11 go@e9b3ff15 cmd/cgo/internal/testplugin.TestIssue22175 (log)
=== RUN   TestIssue22175
( GOPATH=$TMPDIR PWD=$TMPDIR/src/testplugin LD_LIBRARY_PATH=$TMPDIR/src/testplugin /opt/s/w/ir/x/w/goroot/bin/go build -gcflags '' -buildmode=plugin -o issue22175_plugin1.so ./issue22175/plugin1.go )
( GOPATH=$TMPDIR PWD=$TMPDIR/src/testplugin LD_LIBRARY_PATH=$TMPDIR/src/testplugin /opt/s/w/ir/x/w/goroot/bin/go build -gcflags '' -buildmode=plugin -o issue22175_plugin2.so ./issue22175/plugin2.go )
( GOPATH=$TMPDIR PWD=$TMPDIR/src/testplugin LD_LIBRARY_PATH=$TMPDIR/src/testplugin /opt/s/w/ir/x/w/goroot/bin/go build -gcflags '' -o issue22175.exe ./issue22175/main.go )
SIGQUIT: quit
PC=0x7fff20618cbe m=0 sigcode=0

goroutine 0 gp=0x4b05720 m=0 mp=0x4b05f20 [idle]:
runtime.pthread_cond_wait(0x4b06568, 0x4b06528)
	/opt/s/w/ir/x/w/goroot/src/runtime/sys_darwin.go:507 +0x2e fp=0x7ffeec371bc0 sp=0x7ffeec371b98 pc=0x48e68ee
...
r12    0x4b06528
r13    0x16
r14    0x3b0100003c00
r15    0x3b00
rip    0x7fff20618cbe
rflags 0x247
cs     0x7
fs     0x0
gs     0x0
*** Test killed with quit: ran too long (10m0s).
2024-01-10 17:35 gotip-darwin-amd64_11 go@e9b3ff15 cmd/compile/internal/loopvar.TestLoopVarHashes (log)
=== RUN   TestLoopVarHashes
    exec.go:193: test timed out while running command: /opt/s/w/ir/x/w/goroot/bin/go run -trimpath cmd/compile/internal/loopvar/testdata/inlines
    loopvar_test.go:203: SIGQUIT: quit
        PC=0x7fff20618cbe m=0 sigcode=0
        
        goroutine 0 gp=0x3680740 m=0 mp=0x3682440 [idle]:
        runtime.pthread_cond_wait(0x3682a88, 0x3682a48)
        	runtime/sys_darwin.go:507 +0x2e fp=0x7ffeee0b7a20 sp=0x7ffeee0b79f8 pc=0x2ba50ae
        runtime.semasleep(0xffffffffffffffff)
        	runtime/os_darwin.go:71 +0xaf fp=0x7ffeee0b7a78 sp=0x7ffeee0b7a20 pc=0x2b806ef
...
panic: test timed out after 9m0s
running tests:
	TestLoopVarHashes (4m56s)

goroutine 89 gp=0xc000007880 m=4 mp=0xc000080008 [running]:
panic({0x8abd8e0?, 0xc000214040?})
	/opt/s/w/ir/x/w/goroot/src/runtime/panic.go:779 +0x158 fp=0xc000105f10 sp=0xc000105e60 pc=0x8865b38
testing.(*M).startAlarm.func1()
	/opt/s/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385 fp=0xc000105fe0 sp=0xc000105f10 pc=0x8945a25
runtime.goexit({})
...

goroutine 116 gp=0xc000185500 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x8b1bcf0?, 0xc0?, 0x9e?, 0x8b1ae18?)
	/opt/s/w/ir/x/w/goroot/src/runtime/proc.go:402 +0xce fp=0xc0001f0f50 sp=0xc0001f0f30 pc=0x88694ae
runtime.gcBgMarkWorker()
	/opt/s/w/ir/x/w/goroot/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001f0fe0 sp=0xc0001f0f50 pc=0x884a3c5
runtime.goexit({})
	/opt/s/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001f0fe8 sp=0xc0001f0fe0 pc=0x88a0e21
created by runtime.gcBgMarkStartWorkers in goroutine 77
	/opt/s/w/ir/x/w/goroot/src/runtime/mgc.go:1234 +0x1c
2024-01-10 17:35 gotip-darwin-amd64_11 go@e9b3ff15 cmd/compile/internal/ssa.TestFmaHash (log)
=== RUN   TestFmaHash
    fmahash_test.go:40: /opt/s/w/ir/x/w/goroot/bin/go build -o /opt/s/w/ir/x/t/TestFmaHash2806456817/001/fma.exe testdata/fma.go
    fmahash_test.go:41: [GOCOMPILEDEBUG=fmahash=1/0 GOOS=linux GOARCH=arm64 HOME=/opt/s/w/ir/x/t/TestFmaHash2806456817/001]
    exec.go:193: test timed out while running command: /opt/s/w/ir/x/w/goroot/bin/go build -o /opt/s/w/ir/x/t/TestFmaHash2806456817/001/fma.exe testdata/fma.go
    fmahash_test.go:44: exit status 2
    fmahash_test.go:50: Expected to match 'fmahash(0?) triggered .*fma.go:29:..;.*fma.go:18:..' with 
        -----
        # math
        ../../../../math/expm1.go:212:43 [bisect-match 0x3453d6efcaf1e845]
        fmahash triggered ../../../../math/expm1.go:212:43 111100011110100001000101
...
        r13    0x16
        r14    0xa1010000a200
        r15    0xa100
        rip    0x7fff20618cbe
        rflags 0x247
        cs     0x7
        fs     0x0
        gs     0x0
        -----
--- FAIL: TestFmaHash (56.50s)
2024-01-17 19:47 gotip-darwin-amd64_13 go@92d7169a cmd/fix.TestRewrite (log)
=== RUN   TestRewrite/jni.StructField
panic: test timed out after 9m0s
running tests:
	TestRewrite (9m0s)
	TestRewrite/jni.StructField (0s)

goroutine 87 gp=0xc000007340 m=2 mp=0xc00004a908 [running]:
panic({0xee3f460?, 0xc0003b8e40?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:779 +0x158 fp=0xc00009bf10 sp=0xc00009be60 pc=0xecb4338
testing.(*M).startAlarm.func1()
...

goroutine 78 gp=0xc0001848c0 m=nil [GC worker (idle)]:
runtime.gopark(0x267e358508cd?, 0x0?, 0x0?, 0x0?, 0x0?)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/proc.go:402 +0xce fp=0xc0001caf50 sp=0xc0001caf30 pc=0xecb7cae
runtime.gcBgMarkWorker()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001cafe0 sp=0xc0001caf50 pc=0xec992e5
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001cafe8 sp=0xc0001cafe0 pc=0xecedd21
created by runtime.gcBgMarkStartWorkers in goroutine 76
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/mgc.go:1234 +0x1c
2024-01-17 19:47 gotip-darwin-amd64_13 go@92d7169a cmd/go.TestScript (log)
=== RUN   TestScript/build_pgo_auto_multi
=== PAUSE TestScript/build_pgo_auto_multi
=== CONT  TestScript/build_pgo_auto_multi
    script_test.go:132: 2024-01-17T20:34:31Z
    script_test.go:134: $WORK=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-1329034139/tmpdir260820126/build_pgo_auto_multi2549261420
    script_test.go:156: 
        PATH=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-1329034139/tmpdir260820126/testbin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/cache/tools/bin:/Users/swarming/.swarming/w/ir/bbagent_utility_packages:/Users/swarming/.swarming/w/ir/bbagent_utility_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages:/Users/swarming/.swarming/w/ir/cipd_bin_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3/bin:/Users/swarming/.swarming/w/ir/cache/cipd_client:/Users/swarming/.swarming/w/ir/cache/cipd_client/bin:/Users/swarming/.swarming/cipd_cache/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
        HOME=/no-home
        CCACHE_DISABLE=1
        GOARCH=amd64
...
        matched: /Users/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/darwin_amd64/compile -o $WORK/b090/_pkg_.a -trimpath "$WORK/b090=>" -p test/a -lang=go1.20 -complete -buildid Rqm7T22QDK7dX9KCHtRw/Rqm7T22QDK7dX9KCHtRw -pgoprofile=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-1329034139/tmpdir260820126/build_pgo_auto_multi2549261420/gopath/src/a/default.pgo -c=4 -shared -nolocalimports -importcfg $WORK/b090/importcfg -pack ./a/a.go ./a/a_test.go
        > stderr 'compile.*-pgoprofile=.*a(/|\\\\)default\.pgo.*dep(/|\\\\)dep\.go'
        matched: /Users/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/darwin_amd64/compile -o $WORK/b093/_pkg_.a -trimpath "$WORK/b093=>" -p test/dep -lang=go1.20 -complete -buildid vGdknpAWUcCWkpjpt7GB/vGdknpAWUcCWkpjpt7GB -pgoprofile=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-1329034139/tmpdir260820126/build_pgo_auto_multi2549261420/gopath/src/a/default.pgo -c=4 -shared -nolocalimports -importcfg $WORK/b093/importcfg -pack ./dep/dep.go
        > stderr 'compile.*-pgoprofile=.*b(/|\\\\)default\.pgo.*b(/|\\\\)b_test\.go'
        matched: /Users/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/darwin_amd64/compile -o $WORK/b243/_pkg_.a -trimpath "$WORK/b243=>" -p test/b -lang=go1.20 -complete -buildid Pb4FQswzaH9tKZXkjg-a/Pb4FQswzaH9tKZXkjg-a -pgoprofile=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-1329034139/tmpdir260820126/build_pgo_auto_multi2549261420/gopath/src/b/default.pgo -c=4 -shared -nolocalimports -importcfg $WORK/b243/importcfg -pack ./b/b.go ./b/b_test.go
        > stderr 'compile.*-pgoprofile=.*b(/|\\\\)default\.pgo.*dep(/|\\\\)dep\.go'
        matched: /Users/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/darwin_amd64/compile -o $WORK/b246/_pkg_.a -trimpath "$WORK/b246=>" -p test/dep -lang=go1.20 -complete -buildid vGdknpAWUcCWkpjpt7GB/vGdknpAWUcCWkpjpt7GB -pgoprofile=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-1329034139/tmpdir260820126/build_pgo_auto_multi2549261420/gopath/src/b/default.pgo -c=4 -shared -nolocalimports -importcfg $WORK/b246/importcfg -pack ./dep/dep.go
        > ! stderr 'compile.*-pgoprofile=.*nopgo(/|\\\\)nopgo_test\.go'
    script_test.go:156: FAIL: testdata/script/build_pgo_auto_multi.txt:46: context deadline exceeded
--- FAIL: TestScript/build_pgo_auto_multi (17.17s)

watchflakes

@cherrymui
Copy link
Member

There seem to be timeouts on other LUCI darwin builders (other than 13). I put them all here for now. Feel free to split if this is really specific to 13. Thanks.

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `gotip-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-01-22 21:47 gotip-darwin-amd64_13 go@b7ae16e0 cmd/fix.TestRewrite (log)
=== RUN   TestRewrite/jni.ArrayElement
panic: test timed out after 9m0s
running tests:
	TestRewrite (9m0s)
	TestRewrite/jni.ArrayElement (18s)

goroutine 97 gp=0xc000084a80 m=3 mp=0xc00004b208 [running]:
panic({0xa010460?, 0xc000026060?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:768 +0x158 fp=0xc000074f10 sp=0xc000074e60 pc=0x9e85318
testing.(*M).startAlarm.func1()
...

goroutine 89 gp=0xc0004041c0 m=nil [GC worker (idle)]:
runtime.gopark(0xe7858117334?, 0x0?, 0x0?, 0x0?, 0x0?)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/proc.go:402 +0xce fp=0xc0001e2f50 sp=0xc0001e2f30 pc=0x9e88c2e
runtime.gcBgMarkWorker()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001e2fe0 sp=0xc0001e2f50 pc=0x9e6a2e5
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001e2fe8 sp=0xc0001e2fe0 pc=0x9ebeda1
created by runtime.gcBgMarkStartWorkers in goroutine 71
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/mgc.go:1234 +0x1c
2024-01-22 21:47 gotip-darwin-amd64_13 go@b7ae16e0 cmd/go.TestScript (log)
=== RUN   TestScript/modfile_flag
=== PAUSE TestScript/modfile_flag
=== CONT  TestScript/modfile_flag
    script_test.go:132: 2024-01-22T22:53:51Z
    script_test.go:134: $WORK=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-1665181012/tmpdir2501751610/modfile_flag3035229305
    script_test.go:156: 
        PATH=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-1665181012/tmpdir2501751610/testbin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/cache/tools/bin:/Users/swarming/.swarming/w/ir/bbagent_utility_packages:/Users/swarming/.swarming/w/ir/bbagent_utility_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages:/Users/swarming/.swarming/w/ir/cipd_bin_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3/bin:/Users/swarming/.swarming/w/ir/cache/cipd_client:/Users/swarming/.swarming/w/ir/cache/cipd_client/bin:/Users/swarming/.swarming/cipd_cache/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
        HOME=/no-home
        CCACHE_DISABLE=1
        GOARCH=amd64
...
        r13    0xf4b08523
        r14    0xb6486dfe
        r15    0xe4a02e24
        rip    0x4e77c7d
        rflags 0xa13
        cs     0x2b
        fs     0x0
        gs     0x0
    script_test.go:156: FAIL: testdata/script/modfile_flag.txt:36: go mod tidy: exit status 2
--- FAIL: TestScript/modfile_flag (6.60s)

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-01-04 17:31 go1.22-darwin-amd64_13 release-branch.go1.22@15dcdeb5 cmd/fix.TestRewrite [ABORT] (log)
=== RUN   TestRewrite/jni.MapValue
panic: test timed out after 9m0s
running tests:
	TestRewrite (9m0s)
	TestRewrite/jni.MapValue (8s)

goroutine 88 gp=0xc000084fc0 m=5 mp=0xc00011c008 [running]:
panic({0x5b2a460?, 0xc000490ca0?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:779 +0x158 fp=0xc00010df10 sp=0xc00010de60 pc=0x599f338
testing.(*M).startAlarm.func1()
...

goroutine 80 gp=0xc000084e00 m=nil [GC worker (idle), 2 minutes]:
runtime.gopark(0x1dd92cb418f8?, 0x0?, 0x0?, 0x0?, 0x0?)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/proc.go:402 +0xce fp=0xc000043750 sp=0xc000043730 pc=0x59a2cae
runtime.gcBgMarkWorker()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000437e0 sp=0xc000043750 pc=0x59842e5
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000437e8 sp=0xc0000437e0 pc=0x59d8d41
created by runtime.gcBgMarkStartWorkers in goroutine 76
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/mgc.go:1234 +0x1c
2024-01-04 17:31 go1.22-darwin-amd64_13 release-branch.go1.22@15dcdeb5 cmd/go.TestScript (log)
=== RUN   TestScript/list_symlink_internal
=== PAUSE TestScript/list_symlink_internal
=== CONT  TestScript/list_symlink_internal
    script_test.go:132: 2024-01-11T00:57:08Z
    script_test.go:134: $WORK=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-838001353/tmpdir1829639548/list_symlink_internal1352693162
    script_test.go:156: 
        PATH=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-838001353/tmpdir1829639548/testbin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/cache/tools/bin:/Users/swarming/.swarming/w/ir/bbagent_utility_packages:/Users/swarming/.swarming/w/ir/bbagent_utility_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages:/Users/swarming/.swarming/w/ir/cipd_bin_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3/bin:/Users/swarming/.swarming/w/ir/cache/cipd_client:/Users/swarming/.swarming/w/ir/cache/cipd_client/bin:/Users/swarming/.swarming/cipd_cache/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
        HOME=/no-home
        CCACHE_DISABLE=1
        GOARCH=amd64
...
        rip    0x7ff808f440de
        rflags 0x247
        cs     0x7
        fs     0x0
        gs     0x0
        [exit status 2]
    script_test.go:156: FAIL: testdata/script/list_symlink_internal.txt:18: wait: testdata/script/list_symlink_internal.txt:14: go run p.go: exit status 2
        testdata/script/list_symlink_internal.txt:15: go build: context deadline exceeded
        testdata/script/list_symlink_internal.txt:16: go install: exit status 2
--- FAIL: TestScript/list_symlink_internal (3.79s)
2024-01-08 18:52 go1.22-darwin-amd64_11 release-branch.go1.22@8eaa7935 cmd/cgo/internal/testplugin.TestIssue22175 [ABORT] (log)
=== RUN   TestIssue22175
( GOPATH=$TMPDIR PWD=$TMPDIR/src/testplugin LD_LIBRARY_PATH=$TMPDIR/src/testplugin /opt/s/w/ir/x/w/goroot/bin/go build -gcflags '' -buildmode=plugin -o issue22175_plugin1.so ./issue22175/plugin1.go )
( GOPATH=$TMPDIR PWD=$TMPDIR/src/testplugin LD_LIBRARY_PATH=$TMPDIR/src/testplugin /opt/s/w/ir/x/w/goroot/bin/go build -gcflags '' -buildmode=plugin -o issue22175_plugin2.so ./issue22175/plugin2.go )
( GOPATH=$TMPDIR PWD=$TMPDIR/src/testplugin LD_LIBRARY_PATH=$TMPDIR/src/testplugin /opt/s/w/ir/x/w/goroot/bin/go build -gcflags '' -o issue22175.exe ./issue22175/main.go )
SIGQUIT: quit
PC=0x7fff20772cbe m=0 sigcode=0

goroutine 0 gp=0xb73c700 m=0 mp=0xb73cf00 [idle]:
runtime.pthread_cond_wait(0xb73d548, 0xb73d508)
	/opt/s/w/ir/x/w/goroot/src/runtime/sys_darwin.go:507 +0x2e fp=0x7ffee573bbc0 sp=0x7ffee573bb98 pc=0xb51c90e
...
r12    0xb73d508
r13    0x16
r14    0x580100005900
r15    0x5400
rip    0x7fff20772cbe
rflags 0x247
cs     0x7
fs     0x0
gs     0x0
*** Test killed with quit: ran too long (10m0s).
2024-01-08 18:52 go1.22-darwin-amd64_11 release-branch.go1.22@8eaa7935 cmd/compile/internal/loopvar.TestLoopVarHashes [ABORT] (log)
=== RUN   TestLoopVarHashes
    exec.go:193: test timed out while running command: /opt/s/w/ir/x/w/goroot/bin/go run -trimpath cmd/compile/internal/loopvar/testdata/inlines
    loopvar_test.go:203: SIGQUIT: quit
        PC=0x7fff20772cbe m=0 sigcode=0
        
        goroutine 0 gp=0xc4b4720 m=0 mp=0xc4b6420 [idle]:
        runtime.pthread_cond_wait(0xc4b6a68, 0xc4b6a28)
        	runtime/sys_darwin.go:507 +0x2e fp=0x7ffee5284a20 sp=0x7ffee52849f8 pc=0xb9d80ce
        runtime.semasleep(0xffffffffffffffff)
        	runtime/os_darwin.go:71 +0xaf fp=0x7ffee5284a78 sp=0x7ffee5284a20 pc=0xb9b36ef
...
panic: test timed out after 9m0s
running tests:
	TestLoopVarHashes (5m0s)

goroutine 135 gp=0xc00024c8c0 m=5 mp=0xc000186008 [running]:
panic({0xcd378e0?, 0xc000218460?})
	/opt/s/w/ir/x/w/goroot/src/runtime/panic.go:779 +0x158 fp=0xc000205f10 sp=0xc000205e60 pc=0xcadfb38
testing.(*M).startAlarm.func1()
	/opt/s/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385 fp=0xc000205fe0 sp=0xc000205f10 pc=0xcbbfa45
runtime.goexit({})
...

goroutine 102 gp=0xc00024c700 m=nil [GC worker (idle)]:
runtime.gopark(0x7c1f42574ce?, 0xcbe8b2c?, 0x0?, 0x0?, 0x0?)
	/opt/s/w/ir/x/w/goroot/src/runtime/proc.go:402 +0xce fp=0xc0001e0f50 sp=0xc0001e0f30 pc=0xcae34ae
runtime.gcBgMarkWorker()
	/opt/s/w/ir/x/w/goroot/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001e0fe0 sp=0xc0001e0f50 pc=0xcac43c5
runtime.goexit({})
	/opt/s/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001e0fe8 sp=0xc0001e0fe0 pc=0xcb1ae41
created by runtime.gcBgMarkStartWorkers in goroutine 63
	/opt/s/w/ir/x/w/goroot/src/runtime/mgc.go:1234 +0x1c
2024-01-08 18:52 go1.22-darwin-amd64_11 release-branch.go1.22@8eaa7935 cmd/compile/internal/ssa.TestFmaHash (log)
=== RUN   TestFmaHash
    fmahash_test.go:40: /opt/s/w/ir/x/w/goroot/bin/go build -o /opt/s/w/ir/x/t/TestFmaHash1510601658/001/fma.exe testdata/fma.go
    fmahash_test.go:41: [GOCOMPILEDEBUG=fmahash=1/0 GOOS=linux GOARCH=arm64 HOME=/opt/s/w/ir/x/t/TestFmaHash1510601658/001]
    exec.go:193: test timed out while running command: /opt/s/w/ir/x/w/goroot/bin/go build -o /opt/s/w/ir/x/t/TestFmaHash1510601658/001/fma.exe testdata/fma.go
    fmahash_test.go:44: exit status 2
    fmahash_test.go:50: Expected to match 'fmahash(0?) triggered .*fma.go:29:..;.*fma.go:18:..' with 
        -----
        # math
        ../../../../math/expm1.go:212:43 [bisect-match 0x3453d6efcaf1e845]
        fmahash triggered ../../../../math/expm1.go:212:43 111100011110100001000101
...
        r13    0x16
        r14    0xed010000ee00
        r15    0xdb00
        rip    0x7fff20772cbe
        rflags 0x247
        cs     0x7
        fs     0x0
        gs     0x0
        -----
--- FAIL: TestFmaHash (37.66s)
2024-01-10 20:16 go1.21-darwin-amd64_11 release-branch.go1.21@2540b143 cmd/cgo/internal/testplugin.TestIssue22295 [ABORT] (log)
=== RUN   TestIssue22295
( GOPATH=$TMPDIR PWD=$TMPDIR/src/testplugin LD_LIBRARY_PATH=$TMPDIR/src/testplugin /opt/s/w/ir/x/w/goroot/bin/go build -gcflags '' -buildmode=plugin -o issue.22295.so ./issue22295.pkg )
SIGQUIT: quit
PC=0x7fff2051f700 m=0 sigcode=0

goroutine 0 [idle]:
syscall.syscall6(0x100cea5?, 0x1089709?, 0xc0000b2060?, 0xc00021e240?, 0x100c000072c38?, 0x1000000000003?, 0x8a96070?)
	/opt/s/w/ir/x/w/goroot/src/runtime/sys_darwin.go:45 +0x98 fp=0xc000072b60 sp=0xc000072aa0 pc=0x1066c38
syscall.wait4(0xc000072be8?, 0x100d225?, 0x90?, 0x114a040?)
	/opt/s/w/ir/x/w/goroot/src/syscall/zsyscall_darwin_amd64.go:43 +0x45 fp=0xc000072bc0 sp=0xc000072b60 pc=0x1074005
...
r12    0x1
r13    0x125e5a0
r14    0xc000138d00
r15    0x1
rip    0x7fff2051f700
rflags 0x246
cs     0x7
fs     0x0
gs     0x0
*** Test killed with quit: ran too long (10m0s).
2024-01-10 20:16 go1.21-darwin-amd64_11 release-branch.go1.21@2540b143 cmd/compile/internal/loopvar.TestLoopVarHashes [ABORT] (log)
=== RUN   TestLoopVarHashes
    loopvar_test.go:202: # cmd/compile/internal/loopvar/testdata/inlines
        cmd/compile/internal/loopvar/testdata/inlines/main.go:27:6: loop variable i now per-iteration [bisect-match 0x7802e115b9336ca4]
        loopvarhash triggered cmd/compile/internal/loopvar/testdata/inlines/main.go:27:6: loop variable i now per-iteration 001100110110110010100100
        cmd/compile/internal/loopvar/testdata/inlines/main.go:27:6: loop variable i now per-iteration (loop inlined into cmd/compile/internal/loopvar/testdata/inlines/main.go:35) [bisect-match 0x7802e115b9336ca4]
        loopvarhash triggered cmd/compile/internal/loopvar/testdata/inlines/main.go:27:6: loop variable i now per-iteration (loop inlined into cmd/compile/internal/loopvar/testdata/inlines/main.go:35) 001100110110110010100100
        f, af, bf, abf, cf sums = 45, 100, 100, 100, 100
        exit status 11
    exec.go:190: test timed out while running command: /opt/s/w/ir/x/w/goroot/bin/go run -trimpath cmd/compile/internal/loopvar/testdata/inlines
    loopvar_test.go:202: SIGQUIT: quit
...
panic: test timed out after 9m0s
running tests:
	TestLoopVarHashes (5m13s)

goroutine 105 [running]:
panic({0x11fa000?, 0xc0001199b0?})
	/opt/s/w/ir/x/w/goroot/src/runtime/panic.go:1017 +0x3ac fp=0xc000218f10 sp=0xc000218e60 pc=0x1039b0c
testing.(*M).startAlarm.func1()
	/opt/s/w/ir/x/w/goroot/src/testing/testing.go:2259 +0x3b9 fp=0xc000218fe0 sp=0xc000218f10 pc=0x110e439
runtime.goexit()
...
testing.tRunner.func2()
	/opt/s/w/ir/x/w/goroot/src/testing/testing.go:1589 +0x25 fp=0xc000106f70 sp=0xc000106f50 pc=0x110a2c5
testing.tRunner(0xc000083a00, 0x1267af8)
	/opt/s/w/ir/x/w/goroot/src/testing/testing.go:1601 +0x129 fp=0xc000106fc0 sp=0xc000106f70 pc=0x110a249
testing.(*T).Run.func1()
	/opt/s/w/ir/x/w/goroot/src/testing/testing.go:1648 +0x25 fp=0xc000106fe0 sp=0xc000106fc0 pc=0x110b1a5
runtime.goexit()
	/opt/s/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000106fe8 sp=0xc000106fe0 pc=0x106fbe1
created by testing.(*T).Run in goroutine 1
	/opt/s/w/ir/x/w/goroot/src/testing/testing.go:1648 +0x3ad
2024-01-10 20:16 go1.21-darwin-amd64_11 release-branch.go1.21@2540b143 cmd/compile/internal/ssa.TestFmaHash (log)
=== RUN   TestFmaHash
    fmahash_test.go:40: /opt/s/w/ir/x/w/goroot/bin/go build -o /opt/s/w/ir/x/t/TestFmaHash2688638556/001/fma.exe testdata/fma.go
    fmahash_test.go:41: [GOCOMPILEDEBUG=fmahash=1/0 GOOS=linux GOARCH=arm64 HOME=/opt/s/w/ir/x/t/TestFmaHash2688638556/001]
    exec.go:190: test timed out while running command: /opt/s/w/ir/x/w/goroot/bin/go build -o /opt/s/w/ir/x/t/TestFmaHash2688638556/001/fma.exe testdata/fma.go
    fmahash_test.go:44: exit status 2
    fmahash_test.go:50: Expected to match 'fmahash(0?) triggered .*fma.go:29:..;.*fma.go:18:..' with 
        -----
        # math
        ../../../../math/expm1.go:212:43 [bisect-match 0x3453d6efcaf1e845]
        fmahash triggered ../../../../math/expm1.go:212:43 111100011110100001000101
...
        r13    0x16
        r14    0x16b0100016c00
        r15    0x16b00
        rip    0x7fff20518cbe
        rflags 0x247
        cs     0x7
        fs     0x0
        gs     0x0
        -----
--- FAIL: TestFmaHash (50.83s)

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-01-18 23:21 x_build-gotip-darwin-amd64_13 build@911ff433 go@66d34c7d x/build/internal/workflow.TestWatchdog/success (log)
=== RUN   TestWatchdog/success
    workflow_test.go:605: task sleepy    : started
    workflow_test.go:624: task sleepy    : LOG: *snore*
    workflow_test.go:607: task sleepy    : error: context canceled
    workflow_test.go:596: workflow "83ca09be-d53a-4481-b26f-b8201c0e7fc6": stalled
    workflow_test.go:417: w.Run() = _, context deadline exceeded, wanted no error
--- FAIL: TestWatchdog/success (10.00s)

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-01-08 18:41 x_tools-gotip-darwin-amd64_13 tools@25a0e9d3 go@b702e043 x/tools/gopls/internal/test/integration/misc.TestStaticcheckRelatedInfo/default [ABORT] (log)
=== RUN   TestStaticcheckRelatedInfo/default
    runner.go:253: waiting on:
        Unmet: completed work "diagnosing initial workspace load" 1 times
        
        err:context deadline exceeded
        
        state:
        #### log messages (see RPC logs for full text):
        	Info: "2024/01/08 13:08:12 go info for /Users/swarming/.s...
        	Info: "2024/01/08 13:08:15 go/packages.Load #302\n\tsnaps...
...
panic: test timed out after 10m0s
running tests:
	TestStaticcheckRelatedInfo (5s)
	TestStaticcheckRelatedInfo/default (5s)

goroutine 54356 [running]:
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385
created by time.goFunc
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/time/sleep.go:177 +0x2d
...
sync.(*WaitGroup).Wait(0x3933d80?)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/sync/waitgroup.go:116 +0x48
golang.org/x/tools/gopls/internal/server.(*server).diagnose(0xc002d2fc00, {0x3a825a8, 0xc002423ce0}, 0xc003f81700)
	/Users/swarming/.swarming/w/ir/x/w/targetrepo1262639461/gopls/internal/server/diagnostics.go:476 +0xe85
golang.org/x/tools/gopls/internal/server.(*server).diagnoseSnapshot(0xc002d2fc00, 0xc003f81700, {0x0, 0x0, 0x0}, 0x0)
	/Users/swarming/.swarming/w/ir/x/w/targetrepo1262639461/gopls/internal/server/diagnostics.go:223 +0x61f
golang.org/x/tools/gopls/internal/server.(*server).addFolders.func3()
	/Users/swarming/.swarming/w/ir/x/w/targetrepo1262639461/gopls/internal/server/general.go:327 +0x45
created by golang.org/x/tools/gopls/internal/server.(*server).addFolders in goroutine 53790
	/Users/swarming/.swarming/w/ir/x/w/targetrepo1262639461/gopls/internal/server/general.go:326 +0x6a9

watchflakes

@bcmills bcmills changed the title x/build: darwin-amd64_13 LUCI builder is sometimes too slow or stalls x/build: darwin-amd64 LUCI builders are sometimes too slow Jan 29, 2024
@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-01-30 18:58 gotip-darwin-amd64_13 go@0b1b6c15 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 97429: exit status 255
        vmmap[97738]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 97429: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 97429: exit status 255
        vmmap[97743]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 97429: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 97429: exit status 255
        vmmap[21549]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 97429: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 97429
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 97429: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (530.25s)
2024-01-30 23:37 gotip-darwin-amd64_12 go@ae457e81 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 21752: exit status 255
        vmmap[22194]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 21752: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 21752: exit status 255
        vmmap[22204]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 21752: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 21752: exit status 255
        vmmap[49133]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 21752: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 21752
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 21752: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (533.04s)
2024-01-30 23:37 gotip-darwin-amd64_14 go@ae457e81 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    exec.go:193: test timed out while running command: /usr/bin/vmmap 21125
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 21125: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (476.47s)
2024-01-31 14:20 gotip-darwin-amd64-nocgo go@13766fe7 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    exec.go:193: test timed out while running command: /usr/bin/vmmap 40812
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 40812: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (480.94s)
2024-01-31 16:50 gotip-darwin-amd64_13 go@1bb947b2 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 59349: exit status 255
        vmmap[59685]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 59349: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 59349: exit status 255
        vmmap[59694]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 59349: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 59349: exit status 255
        vmmap[83041]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 59349: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 59349
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 59349: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (529.46s)
2024-01-31 19:56 gotip-darwin-amd64_12 go@79738217 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 10211: exit status 255
        vmmap[10654]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 10211: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 10211: exit status 255
        vmmap[10667]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 10211: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 10211: exit status 255
        vmmap[37976]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 10211: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 10211
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 10211: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (533.03s)
2024-01-31 20:51 gotip-darwin-amd64_12 go@5c0d0929 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 25045: exit status 255
        vmmap[25493]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 25045: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 25045: exit status 255
        vmmap[25501]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 25045: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 25045: exit status 255
        vmmap[52342]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 25045: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 25045
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 25045: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (532.96s)
2024-01-31 20:51 gotip-darwin-amd64_13 go@5c0d0929 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 71636: exit status 255
        vmmap[71945]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 71636: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 71636: exit status 255
        vmmap[71948]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 71636: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 71636: exit status 255
        vmmap[95099]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 71636: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 71636
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 71636: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (530.22s)
2024-01-31 23:21 gotip-darwin-amd64_12 go@2f6a25f4 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 64985: exit status 255
        vmmap[65419]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 64985: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 64985: exit status 255
        vmmap[65425]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 64985: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 64985: exit status 255
        vmmap[92615]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 64985: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 64985
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 64985: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (533.12s)
2024-01-31 23:21 gotip-darwin-amd64_14 go@2f6a25f4 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    exec.go:193: test timed out while running command: /usr/bin/vmmap 6822
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 6822: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (503.83s)
2024-02-01 15:34 gotip-darwin-amd64_13 go@5b6cd3d0 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 8241: exit status 255
        vmmap[8548]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 8241: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 8241: exit status 255
        vmmap[8551]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 8241: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 8241: exit status 255
        vmmap[21496]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 8241: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 8241
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 8241: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (526.34s)
2024-02-01 18:09 gotip-darwin-amd64_12 go@18840865 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 69209: exit status 255
        vmmap[69644]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 69209: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 69209: exit status 255
        vmmap[69652]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 69209: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 69209: exit status 255
        vmmap[96953]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 69209: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 69209
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 69209: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (533.11s)
2024-02-01 19:29 gotip-darwin-amd64_13 go@29746b48 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 49665: exit status 255
        vmmap[49986]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 49665: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 49665: exit status 255
        vmmap[49993]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 49665: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 49665: exit status 255
        vmmap[73405]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 49665: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 49665
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 49665: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (530.07s)
2024-02-01 20:09 gotip-darwin-amd64_12 go@d864eac2 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 81702: exit status 255
        vmmap[82156]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 81702: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 81702: exit status 255
        vmmap[82165]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 81702: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 81702: exit status 255
        vmmap[9685]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 81702: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 81702
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 81702: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (532.85s)
2024-02-01 21:02 gotip-darwin-amd64_12 go@829f2ce3 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 60799: exit status 255
        vmmap[61241]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 60799: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 60799: exit status 255
        vmmap[61247]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 60799: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 60799: exit status 255
        vmmap[88672]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 60799: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 60799
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 60799: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (533.14s)
2024-02-01 21:42 gotip-darwin-amd64_13 go@c9d88ea2 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 96971: exit status 255
        vmmap[97276]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 96971: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 96971: exit status 255
        vmmap[97282]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 96971: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 96971: exit status 255
        vmmap[10695]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 96971: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 96971
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 96971: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (529.72s)
2024-02-01 21:42 gotip-darwin-amd64_14 go@c9d88ea2 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    exec.go:193: test timed out while running command: /usr/bin/vmmap 20231
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 20231: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (475.93s)
2024-02-01 22:47 gotip-darwin-amd64_13 go@ac08c05d runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 48003: exit status 255
        vmmap[48313]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 48003: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 48003: exit status 255
        vmmap[48317]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 48003: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 48003: exit status 255
        vmmap[71731]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 48003: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 48003
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 48003: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (530.20s)
2024-02-01 22:57 gotip-darwin-amd64-nocgo go@6d3c1ce8 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    exec.go:193: test timed out while running command: /usr/bin/vmmap 69929
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 69929: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (480.73s)
2024-02-01 22:57 gotip-darwin-amd64_13 go@6d3c1ce8 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 5379: exit status 255
        vmmap[5698]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 5379: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 5379: exit status 255
        vmmap[5703]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 5379: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 5379: exit status 255
        vmmap[29290]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 5379: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 5379
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 5379: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (529.98s)
2024-02-01 22:59 gotip-darwin-amd64-nocgo go@117164f9 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    exec.go:193: test timed out while running command: /usr/bin/vmmap 8301
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 8301: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (479.88s)
2024-02-01 22:59 gotip-darwin-amd64_12 go@117164f9 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 34482: exit status 255
        vmmap[34929]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 34482: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 34482: exit status 255
        vmmap[34936]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 34482: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 34482: exit status 255
        vmmap[62094]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 34482: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 34482
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 34482: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (533.08s)
2024-02-02 15:13 gotip-darwin-amd64_12 go@244ccf47 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 24478: exit status 255
        vmmap[24941]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 24478: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 24478: exit status 255
        vmmap[24947]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 24478: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 24478: exit status 255
        vmmap[52410]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 24478: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 24478
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 24478: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (532.81s)
2024-02-02 15:22 gotip-darwin-amd64_13 go@149db960 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 45176: exit status 255
        vmmap[45494]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 45176: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 45176: exit status 255
        vmmap[45498]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 45176: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 45176: exit status 255
        vmmap[68764]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 45176: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 45176
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 45176: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (530.02s)
2024-02-02 17:06 gotip-darwin-amd64_12 go@3b83560b runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 12491: exit status 255
        vmmap[12940]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 12491: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 12491: exit status 255
        vmmap[12948]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 12491: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 12491: exit status 255
        vmmap[40477]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 12491: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 12491
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 12491: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (532.88s)

watchflakes

@mknyszek
Copy link
Contributor

mknyszek commented Feb 2, 2024

Filed #65468 for TestVMInfo.

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-02 18:35 gotip-darwin-amd64_12 go@3d20a327 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 41425: exit status 255
        vmmap[41848]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 41425: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 41425: exit status 255
        vmmap[41856]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 41425: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 41425: exit status 255
        vmmap[69390]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 41425: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 41425
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 41425: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (533.33s)
2024-02-02 22:08 gotip-darwin-amd64_12 go@9819d826 runtime/pprof.TestVMInfo (log)
=== RUN   TestVMInfo
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 57964: exit status 255
        vmmap[58413]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 57964: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 57964: exit status 255
        vmmap[58421]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 57964: exit status 255
...
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:81: /usr/bin/vmmap 57964: exit status 255
        vmmap[85938]: [fatal] Failed to generate corpse from the process: 6 ((os/kern) resource shortage), exiting
    vminfo_darwin_test.go:84: /usr/bin/vmmap 57964: exit status 255
    vminfo_darwin_test.go:69: retrying vmmap after error: exit status 255
    exec.go:193: test timed out while running command: /usr/bin/vmmap 57964
    vminfo_darwin_test.go:79: vmmap output: 
    vminfo_darwin_test.go:84: /usr/bin/vmmap 57964: signal: quit
    vminfo_darwin_test.go:39: failed to parse vmmap output, vmmap reported an error: vmmap no text segment found
--- FAIL: TestVMInfo (533.09s)

watchflakes

@cherrymui
Copy link
Member

TestVMInfo failure is #62352. It seems that it matches this because it keeps retrying until it timed out, so test timed out is printed.

@cherrymui
Copy link
Member

I changed the pattern to "default", so the other issue should take priority.

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-07 19:42 x_tools-gotip-darwin-amd64-longtest tools@76ef6b6a go@1400b268 x/tools/gopls/internal/test/integration/misc.TestWindowsVendoring_Issue56291/default [ABORT] (log)
=== RUN   TestWindowsVendoring_Issue56291/default
    vendor_test.go:99: organizing imports before save: context deadline exceeded
#### Start Gopls Test Logs for "TestWindowsVendoring_Issue56291/default"
[Trace - 01:11:01.860 AM] Sending request 'initialize - (1)'.
Params: {"processId":0,"rootUri":"","capabilities":{"workspace":{"workspaceEdit":{"resourceOperations":["rename"]},"didChangeConfiguration":{},"didChangeWatchedFiles":{"dynamicRegistration":true},"configuration":true},"textDocument":{"completion":{"completionItem":{"snippetSupport":true,"tagSupport":{"valueSet":[1]}}},"documentSymbol":{"hierarchicalDocumentSymbolSupport":true},"codeAction":{"codeActionLiteralSupport":{"codeActionKind":{"valueSet":null}}},"publishDiagnostics":{},"semanticTokens":{"requests":{"full":true},"tokenTypes":["namespace","type","class","enum","interface","struct","typeParameter","parameter","variable","property","enumMember","event","function","method","macro","keyword","modifier","comment","string","number","regexp","operator"],"tokenModifiers":["declaration","definition","readonly","static","deprecated","abstract","async","modification","documentation","defaultLibrary"],"formats":null}},"window":{"workDoneProgress":true}},"initializationOptions":{"completionBudget":"0s","diagnosticsDelay":"10ms","env":{"GO111MODULE":"","GOMODCACHE":"","GOPACKAGESDRIVER":"off","GOPATH":"/Users/swarming/.swarming/w/ir/x/t/gopls-test-3195540426/TestWindowsVendoring_Issue56291/default/gopath","GOPROXY":"file:///Users/swarming/.swarming/w/ir/x/t/gopls-test-3195540426/TestWindowsVendoring_Issue56291/default/proxy","GOSUMDB":"off"},"verboseWorkDoneProgress":true},"trace":"messages","workspaceFolders":[{"uri":"file:///Users/swarming/.swarming/w/ir/x/t/gopls-test-3195540426/TestWindowsVendoring_Issue56291/default/work","name":"work"}]}


[Trace - 01:11:01.861 AM] Received response 'initialize - (1)' in 0ms.
Result: {"capabilities":{"textDocumentSync":{"openClose":true,"change":2,"save":{}},"completionProvider":{"triggerCharacters":["."]},"hoverProvider":true,"signatureHelpProvider":{"triggerCharacters":["(",","]},"definitionProvider":true,"typeDefinitionProvider":true,"implementationProvider":true,"referencesProvider":true,"documentHighlightProvider":true,"documentSymbolProvider":true,"codeActionProvider":true,"codeLensProvider":{},"documentLinkProvider":{},"workspaceSymbolProvider":true,"documentFormattingProvider":true,"renameProvider":true,"foldingRangeProvider":true,"selectionRangeProvider":true,"executeCommandProvider":{"commands":["gopls.add_dependency","gopls.add_import","gopls.add_telemetry_counters","gopls.apply_fix","gopls.change_signature","gopls.check_upgrades","gopls.diagnose_files","gopls.edit_go_directive","gopls.fetch_vulncheck_result","gopls.gc_details","gopls.generate","gopls.go_get_package","gopls.list_imports","gopls.list_known_packages","gopls.maybe_prompt_for_telemetry","gopls.mem_stats","gopls.regenerate_cgo","gopls.remove_dependency","gopls.reset_go_mod_diagnostics","gopls.run_go_work_command","gopls.run_govulncheck","gopls.run_tests","gopls.start_debugging","gopls.start_profile","gopls.stop_profile","gopls.test","gopls.tidy","gopls.toggle_gc_details","gopls.update_go_sum","gopls.upgrade_dependency","gopls.vendor","gopls.views","gopls.workspace_stats"]},"callHierarchyProvider":true,"semanticTokensProvider":{"legend":{"tokenTypes":["namespace","type","class","enum","interface","struct","typeParameter","parameter","variable","property","enumMember","event","function","method","macro","keyword","modifier","comment","string","number","regexp","operator"],"tokenModifiers":["declaration","definition","readonly","static","deprecated","abstract","async","modification","documentation","defaultLibrary"]},"range":true,"full":true},"inlayHintProvider":{},"workspace":{"workspaceFolders":{"supported":true,"changeNotifications":"workspace/didChangeWorkspaceFolders"}}},"serverInfo":{"name":"gopls","version":"{\"GoVersion\":\"devel 1400b268529f1bbacde83b3dc022fec0b196846b\",\"Path\":\"\",\"Main\":{\"Path\":\"\",\"Version\":\"\",\"Sum\":\"\",\"Replace\":null},\"Deps\":null,\"Settings\":null,\"Version\":\"(unknown)\"}"}}

...
panic: test timed out after 10m0s
running tests:
	TestWindowsVendoring_Issue56291 (4s)
	TestWindowsVendoring_Issue56291/default (4s)

goroutine 145258 [running]:
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385
created by time.goFunc
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/time/sleep.go:177 +0x2d
...
golang.org/x/tools/internal/gocommand.runCmdContext.func1()
	/Users/swarming/.swarming/w/ir/x/w/targetrepo479724466/internal/gocommand/invoke.go:307 +0x45
created by golang.org/x/tools/internal/gocommand.runCmdContext in goroutine 145318
	/Users/swarming/.swarming/w/ir/x/w/targetrepo479724466/internal/gocommand/invoke.go:306 +0x17e

goroutine 145322 [chan receive]:
golang.org/x/tools/gopls/internal/protocol.Handlers.AsyncHandler.func2.2()
	/Users/swarming/.swarming/w/ir/x/w/targetrepo479724466/internal/jsonrpc2/handler.go:101 +0x65
created by golang.org/x/tools/gopls/internal/protocol.Handlers.AsyncHandler.func2 in goroutine 144812
	/Users/swarming/.swarming/w/ir/x/w/targetrepo479724466/internal/jsonrpc2/handler.go:100 +0x1c5

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-07 21:58 x_tools-go1.22-darwin-amd64-longtest tools@5fcc6273 release-branch.go1.22@dd31ad7e x/tools/gopls/internal/test/integration/misc.TestRunGovulncheckError/default [ABORT] (log)
=== RUN   TestRunGovulncheckError/default
    runner.go:253: waiting on:
        Unmet: completed work "diagnosing initial workspace load" 1 times
        
        err:context deadline exceeded
        
        state:
        #### log messages (see RPC logs for full text):
        
        #### diagnostics:
...
panic: test timed out after 10m0s
running tests:
	TestRunGovulncheckError (0s)
	TestRunGovulncheckError/default (0s)

goroutine 145844 [running]:
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385
created by time.goFunc
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/time/sleep.go:177 +0x2d
...
net.(*TCPListener).Accept(0xc0025c65a0)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/net/tcpsock.go:327 +0x30
net/http.(*Server).Serve(0xc000c322d0, {0x4e77690, 0xc0025c65a0})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/net/http/server.go:3255 +0x33e
net/http.Serve(...)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/net/http/server.go:2794
golang.org/x/tools/gopls/internal/debug.(*Instance).Serve.func1()
	/Users/swarming/.swarming/w/ir/x/w/targetrepo3981203002/gopls/internal/debug/serve.go:483 +0xa51
created by golang.org/x/tools/gopls/internal/debug.(*Instance).Serve in goroutine 10471
	/Users/swarming/.swarming/w/ir/x/w/targetrepo3981203002/gopls/internal/debug/serve.go:444 +0x450
2024-02-07 21:58 x_tools-go1.22-darwin-amd64-longtest tools@5fcc6273 release-branch.go1.22@dd31ad7e x/tools/gopls/internal/test/integration/misc.TestWindowsVendoring_Issue56291/default (log)
=== RUN   TestWindowsVendoring_Issue56291/default
    vendor_test.go:100: waiting on:
        Unmet: once "All of:\ncompleted work \"diagnosing opened files\" at least 1 time(s)\ncompleted work \"diagnosing changed files\" at least 3 time(s)\ncompleted work \"diagnosing files changed on disk\" at least 2 time(s)\ncompleted work \"diagnosing saved files\" at least 1 time(s)" is met, must have:
        no diagnostics 
        
        err:context deadline exceeded
        
        state:
        #### log messages (see RPC logs for full text):
        	Info: "2024/02/07 16:22:57 go info for /Users/swarming/.s...
...
[Trace - 16:22:58.811 PM] Received notification '$/progress'.
Params: {"token":"7654982806068908304","value":{"kind":"end","message":"Done."}}


[Trace - 16:22:58.813 PM] Received notification '$/progress'.
Params: {"token":"1833575941743003854","value":{"kind":"end","message":"Done."}}


#### End Gopls Test Logs for "TestWindowsVendoring_Issue56291/default"
--- FAIL: TestWindowsVendoring_Issue56291/default (1.74s)
2024-02-08 13:20 x_tools-go1.22-darwin-amd64-longtest tools@f4fa7a75 release-branch.go1.22@20107e05 x/tools/gopls/internal/test/integration/misc.TestRunGovulncheckStd/experimental [ABORT] (log)
=== RUN   TestRunGovulncheckStd/experimental
    vuln_test.go:212: waiting on:
        Unmet: once "completed work for token 6737988853749794769" is met, must have:
        received window/showMessage containing "Found GOSTDLIB"
        no diagnostics for file "go.mod"
        
        err:context deadline exceeded
        
        state:
        #### log messages (see RPC logs for full text):
...
panic: test timed out after 10m0s
running tests:
	TestRunGovulncheckStd (40s)
	TestRunGovulncheckStd/experimental (15s)

goroutine 148624 [running]:
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385
created by time.goFunc
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/time/sleep.go:177 +0x2d
...
encoding/json.(*Decoder).More(...)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/encoding/json/stream.go:483
golang.org/x/tools/gopls/internal/vulncheck/govulncheck.HandleJSON({0xc538078, 0xc000150d20}, {0xc540b40, 0xc00382e000})
	/Users/swarming/.swarming/w/ir/x/w/targetrepo1244243866/gopls/internal/vulncheck/govulncheck/handler.go:36 +0x6d
golang.org/x/tools/gopls/internal/vulncheck/scan.RunGovulncheck.func2()
	/Users/swarming/.swarming/w/ir/x/w/targetrepo1244243866/gopls/internal/vulncheck/scan/command.go:91 +0x29
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/Users/swarming/.swarming/w/ir/x/w/gopath/pkg/mod/golang.org/x/sync@v0.6.0/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 148616
	/Users/swarming/.swarming/w/ir/x/w/gopath/pkg/mod/golang.org/x/sync@v0.6.0/errgroup/errgroup.go:75 +0x96
2024-02-08 13:20 x_tools-gotip-darwin-amd64-longtest tools@f4fa7a75 go@58fea5df x/tools/gopls/internal/test/integration/misc.TestRunGovulncheckError2/experimental [ABORT] (log)
=== RUN   TestRunGovulncheckError2/experimental
    vuln_test.go:84: waiting on:
        Unmet: completed work for token 8125730333237030511
        
        err:context deadline exceeded
        
        state:
        #### log messages (see RPC logs for full text):
        	Info: "2024/02/08 14:40:44 go info for /Users/swarming/.s...
        	Info: "2024/02/08 14:40:44 go/packages.Load #721\n\tsnaps...
...
panic: test timed out after 10m0s
running tests:
	TestRunGovulncheckError2 (13s)
	TestRunGovulncheckError2/experimental (3s)

goroutine 146162 [running]:
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385
created by time.goFunc
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/time/sleep.go:177 +0x2d
...
golang.org/x/tools/gopls/internal/test/integration.(*Env).Await(0xc0070bd8b0, {0xc003c63bf0, 0x1, 0x1})
	/Users/swarming/.swarming/w/ir/x/w/targetrepo1993857887/gopls/internal/test/integration/env.go:337 +0xd3
golang.org/x/tools/gopls/internal/test/integration/misc.TestRunGovulncheckError2.func1(0xc004269520, 0xc0070bd8b0)
	/Users/swarming/.swarming/w/ir/x/w/targetrepo1993857887/gopls/internal/test/integration/misc/vuln_test.go:84 +0x138
golang.org/x/tools/gopls/internal/test/integration.(*Runner).Run.func1(0xc004269520)
	/Users/swarming/.swarming/w/ir/x/w/targetrepo1993857887/gopls/internal/test/integration/runner.go:254 +0xbf5
testing.tRunner(0xc004269520, 0xc001b17b80)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1689 +0xfb
created by testing.(*T).Run in goroutine 145737
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x390

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-09 17:28 x_tools-gotip-darwin-amd64-longtest tools@a5af84e3 go@4a7f3ac8 x/tools/gopls/internal/test/integration/misc.TestRunGovulncheckStd/experimental [ABORT] (log)
=== RUN   TestRunGovulncheckStd/experimental
    vuln_test.go:212: waiting on:
        Unmet: once "completed work for token 2663296913310031948" is met, must have:
        received window/showMessage containing "Found GOSTDLIB"
        no diagnostics for file "go.mod"
        
        err:context deadline exceeded
        
        state:
        #### log messages (see RPC logs for full text):
...
panic: test timed out after 10m0s
running tests:
	TestRunGovulncheckStd (34s)
	TestRunGovulncheckStd/experimental (11s)

goroutine 150512 [running]:
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385
created by time.goFunc
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/time/sleep.go:177 +0x2d
...
os.(*Process).Wait(...)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/os/exec.go:134
os/exec.(*Cmd).Wait(0xc0022ecc60)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:897 +0x3d
golang.org/x/tools/gopls/internal/vulncheck/scan.RunGovulncheck.func1()
	/Users/swarming/.swarming/w/ir/x/w/targetrepo4041595619/gopls/internal/vulncheck/scan/command.go:85 +0x2b7
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/Users/swarming/.swarming/w/ir/x/w/gopath/pkg/mod/golang.org/x/sync@v0.6.0/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 150485
	/Users/swarming/.swarming/w/ir/x/w/gopath/pkg/mod/golang.org/x/sync@v0.6.0/errgroup/errgroup.go:75 +0x96

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-12 19:37 x_tools-go1.22-darwin-amd64-longtest tools@50b4f1b1 release-branch.go1.22@20107e05 x/tools/gopls/internal/test/integration/misc.TestGovulncheckInfo/default [ABORT] (log)
=== RUN   TestGovulncheckInfo/default
    runner.go:253: waiting on:
        Unmet: completed work "diagnosing initial workspace load" 1 times
        
        err:context deadline exceeded
        
        state:
        #### log messages (see RPC logs for full text):
        
        #### diagnostics:
...
panic: test timed out after 10m0s
running tests:
	TestGovulncheckInfo (0s)
	TestGovulncheckInfo/default (0s)

goroutine 158110 [running]:
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385
created by time.goFunc
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/time/sleep.go:177 +0x2d
...
os.(*File).WriteTo(0xc004673ae0?, {0x516ccf8?, 0xc0030c8e70?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/os/file.go:247 +0x49
io.copyBuffer({0x516ccf8, 0xc0030c8e70}, {0x516c8f8, 0xc00259e180}, {0x0, 0x0, 0x0})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/io/io.go:411 +0x9d
io.Copy(...)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/io/io.go:388
golang.org/x/tools/internal/gocommand.runCmdContext.func1()
	/Users/swarming/.swarming/w/ir/x/w/targetrepo49944796/internal/gocommand/invoke.go:307 +0x45
created by golang.org/x/tools/internal/gocommand.runCmdContext in goroutine 158112
	/Users/swarming/.swarming/w/ir/x/w/targetrepo49944796/internal/gocommand/invoke.go:306 +0x17e
2024-02-14 21:26 x_tools-go1.22-darwin-amd64-longtest tools@e3254052 release-branch.go1.22@20107e05 x/tools/gopls/internal/test/integration/misc.TestRunVulncheckPackageDiagnostics/default/forwarded [ABORT] (log)
=== RUN   TestRunVulncheckPackageDiagnostics/default/forwarded
    runner.go:253: waiting on:
        Unmet: completed work "diagnosing initial workspace load" 1 times
        
        err:context deadline exceeded
        
        state:
        #### log messages (see RPC logs for full text):
        	Info: "2024/02/14 14:25:19 Handshake session update\n\tup...
        	Info: "2024/02/14 14:25:20 go info for /Users/swarming/.s...
...
panic: test timed out after 10m0s
running tests:
	TestRunVulncheckPackageDiagnostics (5s)
	TestRunVulncheckPackageDiagnostics/default (1s)
	TestRunVulncheckPackageDiagnostics/default/forwarded (1s)

goroutine 154155 [running]:
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385
created by time.goFunc
...
golang.org/x/tools/gopls/internal/cache.(*Snapshot).WorkspaceMetadata(0xc0017f0b40, {0xc35bcf8?, 0xc0005aaf30?})
	/Users/swarming/.swarming/w/ir/x/w/targetrepo2514254965/gopls/internal/cache/snapshot.go:1081 +0x5e
golang.org/x/tools/gopls/internal/server.(*server).diagnose(0xc0037efa00, {0xc35bcf8, 0xc0005aad80}, 0xc0017f0b40)
	/Users/swarming/.swarming/w/ir/x/w/targetrepo2514254965/gopls/internal/server/diagnostics.go:366 +0x5d8
golang.org/x/tools/gopls/internal/server.(*server).diagnoseSnapshot(0xc0037efa00, 0xc0017f0b40, {0x0, 0x0, 0x0}, 0x0)
	/Users/swarming/.swarming/w/ir/x/w/targetrepo2514254965/gopls/internal/server/diagnostics.go:223 +0x61f
golang.org/x/tools/gopls/internal/server.(*server).addFolders.func3()
	/Users/swarming/.swarming/w/ir/x/w/targetrepo2514254965/gopls/internal/server/general.go:333 +0x45
created by golang.org/x/tools/gopls/internal/server.(*server).addFolders in goroutine 154042
	/Users/swarming/.swarming/w/ir/x/w/targetrepo2514254965/gopls/internal/server/general.go:332 +0x6a9

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-16 15:51 go1.22-darwin-amd64_10.15 release-branch.go1.22@6fbd01a7 cmd/cgo/internal/testplugin.TestMethod [ABORT] (log)
=== RUN   TestMethod
( GOPATH=$TMPDIR PWD=$TMPDIR/src/testplugin LD_LIBRARY_PATH=$TMPDIR/src/testplugin /opt/s/w/ir/x/w/goroot/bin/go build -gcflags '' -buildmode=plugin -o plugin.so ./method/plugin.go )
( GOPATH=$TMPDIR PWD=$TMPDIR/src/testplugin LD_LIBRARY_PATH=$TMPDIR/src/testplugin /opt/s/w/ir/x/w/goroot/bin/go build -gcflags '' -o method.exe ./method/main.go )
SIGQUIT: quit
PC=0x7fff6d49d882 m=0 sigcode=0

goroutine 0 gp=0xc973720 m=0 mp=0xc973f20 [idle]:
runtime.pthread_cond_wait(0xc974568, 0xc974528)
	/opt/s/w/ir/x/w/goroot/src/runtime/sys_darwin.go:507 +0x2e fp=0x7ffee4503bc0 sp=0x7ffee4503b98 pc=0xc754a2e
runtime.semasleep(0xffffffffffffffff)
...
r12    0xc974568
r13    0x16
r14    0x500100005100
r15    0x1b1d1dc0
rip    0x7fff6d49d882
rflags 0x203
cs     0x7
fs     0x0
gs     0x0
*** Test killed with quit: ran too long (10m0s).
2024-02-16 15:51 go1.22-darwin-amd64_10.15 release-branch.go1.22@6fbd01a7 cmd/compile/internal/loopvar.TestLoopVarHashes (log)
=== RUN   TestLoopVarHashes
    loopvar_test.go:203: # cmd/compile/internal/loopvar/testdata/inlines
        cmd/compile/internal/loopvar/testdata/inlines/main.go:27:6: loop variable i now per-iteration [bisect-match 0x7802e115b9336ca4]
        loopvarhash triggered cmd/compile/internal/loopvar/testdata/inlines/main.go:27:6: loop variable i now per-iteration 001100110110110010100100
        cmd/compile/internal/loopvar/testdata/inlines/main.go:27:6: loop variable i now per-iteration (loop inlined into cmd/compile/internal/loopvar/testdata/inlines/main.go:35) [bisect-match 0x7802e115b9336ca4]
        loopvarhash triggered cmd/compile/internal/loopvar/testdata/inlines/main.go:27:6: loop variable i now per-iteration (loop inlined into cmd/compile/internal/loopvar/testdata/inlines/main.go:35) 001100110110110010100100
        f, af, bf, abf, cf sums = 45, 100, 100, 100, 100
        exit status 11
    exec.go:193: test timed out while running command: /opt/s/w/ir/x/w/goroot/bin/go run -trimpath cmd/compile/internal/loopvar/testdata/inlines
    loopvar_test.go:203: SIGQUIT: quit
...
        r15    0x73d7dc0
        rip    0x7fff6d49d882
        rflags 0x203
        cs     0x7
        fs     0x0
        gs     0x0
    loopvar_test.go:208: vx336ca4: did not see triggered main.go:27:6
    loopvar_test.go:216: vx336ca4: did not see bisect-match for main.go:27:6
    loopvar_test.go:224: vx336ca4: did not see expected value of m run
--- FAIL: TestLoopVarHashes (363.04s)
2024-02-16 15:51 go1.22-darwin-amd64_10.15 release-branch.go1.22@6fbd01a7 cmd/compile/internal/ssa.TestFmaHash (log)
=== RUN   TestFmaHash
    fmahash_test.go:40: /opt/s/w/ir/x/w/goroot/bin/go build -o /opt/s/w/ir/x/t/TestFmaHash2333609748/001/fma.exe testdata/fma.go
    fmahash_test.go:41: [GOCOMPILEDEBUG=fmahash=1/0 GOOS=linux GOARCH=arm64 HOME=/opt/s/w/ir/x/t/TestFmaHash2333609748/001]
    exec.go:193: test timed out while running command: /opt/s/w/ir/x/w/goroot/bin/go build -o /opt/s/w/ir/x/t/TestFmaHash2333609748/001/fma.exe testdata/fma.go
    fmahash_test.go:44: exit status 2
    fmahash_test.go:50: Expected to match 'fmahash(0?) triggered .*fma.go:29:..;.*fma.go:18:..' with 
        -----
        # math
        ../../../../math/expm1.go:212:43 [bisect-match 0x3453d6efcaf1e845]
        fmahash triggered ../../../../math/expm1.go:212:43 111100011110100001000101
...
        r13    0x16
        r14    0x1670100016800
        r15    0xe922dc0
        rip    0x7fff6d49d882
        rflags 0x203
        cs     0x7
        fs     0x0
        gs     0x0
        -----
--- FAIL: TestFmaHash (87.92s)

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-22 15:37 gotip-darwin-amd64-longtest go@638b9023 cmd/go.TestScript/repro_build (log)
=== RUN   TestScript/repro_build
=== PAUSE TestScript/repro_build
=== CONT  TestScript/repro_build
    script_test.go:132: 2024-02-22T16:50:39Z
    script_test.go:134: $WORK=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-3474057958/tmpdir1919206175/repro_build165899711
2024/02/22 08:50:39 serving /git/hello/git-upload-pack
2024/02/22 08:50:39 serving /git/hello/info/refs?service=git-upload-pack
2024/02/22 08:50:40 serving /git/hello/git-upload-pack
2024/02/22 08:50:40 serving /git/hello/git-upload-pack
    script_test.go:156: 
...
        r13    0x7c0100007d00
        r14    0x7ff85d999e80
        r15    0x4c00
        rip    0x7ff81a3845d6
        rflags 0x247
        cs     0x7
        fs     0x0
        gs     0x0
    script_test.go:156: FAIL: testdata/script/repro_build.txt:21: go build -a -o gofmt18.exe cmd/gofmt: exit status 2
--- FAIL: TestScript/repro_build (240.24s)
2024-02-22 19:32 gotip-darwin-amd64-longtest go@db57e5a0 cmd/go.TestScript/build_cc_cache_issue64423 (log)
=== RUN   TestScript/build_cc_cache_issue64423
=== PAUSE TestScript/build_cc_cache_issue64423
=== CONT  TestScript/build_cc_cache_issue64423
    script_test.go:132: 2024-02-22T20:51:49Z
    script_test.go:134: $WORK=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-17263927/tmpdir3277359705/build_cc_cache_issue644232722718475
    script_test.go:156: 
        PATH=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-17263927/tmpdir3277359705/testbin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/cache/tools/bin:/Users/swarming/.swarming/w/ir/bbagent_utility_packages:/Users/swarming/.swarming/w/ir/bbagent_utility_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages:/Users/swarming/.swarming/w/ir/cipd_bin_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3/bin:/Users/swarming/.swarming/w/ir/cache/cipd_client:/Users/swarming/.swarming/w/ir/cache/cipd_client/bin:/Users/swarming/.swarming/cipd_cache/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
        HOME=/no-home
        CCACHE_DISABLE=1
        GOARCH=amd64
...
        r13    0x720100007300
        r14    0x7ff853af0e80
        r15    0x6700
        rip    0x7ff8104db5d6
        rflags 0x247
        cs     0x7
        fs     0x0
        gs     0x0
    script_test.go:156: FAIL: testdata/script/build_cc_cache_issue64423.txt:38: go build -x runtime/cgo: exit status 2
--- FAIL: TestScript/build_cc_cache_issue64423 (19.76s)

watchflakes

@dmitshur dmitshur removed their assignment Feb 23, 2024
@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-14 19:47 x_pkgsite-metrics-go1.21-darwin-amd64-longtest pkgsite-metrics@ffcab83d release-branch.go1.21@f38fca30 x/pkgsite-metrics.Test [ABORT] (log)
=== RUN   Test
    all_test.go:18: warning: running go test ./... will skip checking integration tests
go version devel f38fca30a7ea1346c6a7eefeb3253f2eefe8075b darwin/amd64
Running go test ./... will skip integration tests (GO_ECOSYSTEM_INTEGRATION_TESTING != 1)
Checking go and sh files for license header
go vet -all ./...

Installing: honnef.co/go/tools/cmd/staticcheck
staticcheck ./...

...
panic: test timed out after 10m0s
running tests:
	Test (10m0s)

goroutine 7 [running]:
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2259 +0x3b9
created by time.goFunc
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/time/sleep.go:176 +0x2d

...
os/exec.(*Cmd).Wait(0xc00011c000)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:890 +0x45
os/exec.(*Cmd).Run(0xc0000121b0?)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:590 +0x2d
golang.org/x/pkgsite-metrics.Test(0xc000007a00)
	/Users/swarming/.swarming/w/ir/x/w/targetrepo2236787587/all_test.go:32 +0x1b9
testing.tRunner(0xc000007a00, 0x1144dd0)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1595 +0xff
created by testing.(*T).Run in goroutine 1
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1648 +0x3ad
2024-02-23 21:21 gotip-darwin-amd64-longtest go@72030635 cmd/go.TestScript/build_trimpath_cgo (log)
=== RUN   TestScript/build_trimpath_cgo
=== PAUSE TestScript/build_trimpath_cgo
=== CONT  TestScript/build_trimpath_cgo
    script_test.go:132: 2024-02-24T00:02:22Z
    script_test.go:134: $WORK=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-4043279628/tmpdir3803042848/build_trimpath_cgo2650604850
    script_test.go:156: 
        PATH=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-4043279628/tmpdir3803042848/testbin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/cache/tools/bin:/Users/swarming/.swarming/w/ir/bbagent_utility_packages:/Users/swarming/.swarming/w/ir/bbagent_utility_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages:/Users/swarming/.swarming/w/ir/cipd_bin_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3/bin:/Users/swarming/.swarming/w/ir/cache/cipd_client:/Users/swarming/.swarming/w/ir/cache/cipd_client/bin:/Users/swarming/.swarming/cipd_cache/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
        HOME=/no-home
        CCACHE_DISABLE=1
        GOARCH=amd64
...
        r13    0x1a0100001b00
        r14    0x7ff84438ae80
        r15    0x1a00
        rip    0x7ff800d755d6
        rflags 0x247
        cs     0x7
        fs     0x0
        gs     0x0
    script_test.go:156: FAIL: testdata/script/build_trimpath_cgo.txt:38: go build -overlay overlay.json -trimpath -o hello_overlay.exe ./hello_overlay: exit status 2
--- FAIL: TestScript/build_trimpath_cgo (30.66s)
2024-02-23 21:21 gotip-darwin-amd64_11 go@b899e0b8 cmd/cgo/internal/testcarchive.TestExtar [ABORT] (log)
=== RUN   TestExtar
panic: test timed out after 9m0s
running tests:
	TestExtar (36s)

goroutine 103 gp=0xc0002c6380 m=2 mp=0xc00004c908 [running]:
panic({0x2de30a0?, 0xc00028c2a0?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:778 +0x16f fp=0xc000185f10 sp=0xc000185e60 pc=0x2c8b66f
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385 fp=0xc000185fe0 sp=0xc000185f10 pc=0x2d2fc25
...
os/exec.(*Cmd).writerDescriptor.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:577 +0x20 fp=0xc000306f60 sp=0xc000306f00 pc=0x2d3ada0
os/exec.(*Cmd).Start.func2(0x2e13a40?)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:724 +0x2c fp=0xc000306fc8 sp=0xc000306f60 pc=0x2d3ba4c
os/exec.(*Cmd).Start.gowrap1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:736 +0x24 fp=0xc000306fe0 sp=0xc000306fc8 pc=0x2d3b9e4
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000306fe8 sp=0xc000306fe0 pc=0x2cc5501
created by os/exec.(*Cmd).Start in goroutine 101
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:723 +0x9ab
2024-02-23 21:21 gotip-darwin-amd64_11 go@b899e0b8 cmd/cgo/internal/testplugin.TestIssue18676 [ABORT] (log)
=== RUN   TestIssue18676
( GOPATH=$TMPDIR PWD=$TMPDIR/src/testplugin LD_LIBRARY_PATH=$TMPDIR/src/testplugin /Users/swarming/.swarming/w/ir/x/w/goroot/bin/go build -gcflags '' -buildmode=plugin -o plugin.so ./issue18676/plugin.go )
SIGQUIT: quit
PC=0x7fff20490700 m=0 sigcode=0

goroutine 0 gp=0x66bc720 m=0 mp=0x66bcf20 [idle]:
syscall.syscall6(0xc000183f80?, 0x1000000000010?, 0x10000000030?, 0xd217248?, 0x90?, 0x67455b8?, 0x90?)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_darwin.go:45 +0x98 fp=0xc000096b00 sp=0xc000096a40 pc=0x64b00b8
syscall.wait4(0xc000096b88?, 0x6450645?, 0x90?, 0x65cf440?)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/syscall/zsyscall_darwin_amd64.go:44 +0x45 fp=0xc000096b60 sp=0xc000096b00 pc=0x64be285
...
r12    0xc00007a2d0
r13    0x0
r14    0xc000084700
r15    0x3ffffffffffffff
rip    0x7fff20490700
rflags 0x246
cs     0x7
fs     0x0
gs     0x0
*** Test killed with quit: ran too long (10m0s).
2024-02-23 21:21 gotip-darwin-amd64_11 go@b899e0b8 cmd/compile/internal/loopvar.TestLoopVarHashes [ABORT] (log)
=== RUN   TestLoopVarHashes
    exec.go:193: test timed out while running command: /Users/swarming/.swarming/w/ir/x/w/goroot/bin/go run -trimpath cmd/compile/internal/loopvar/testdata/inlines
    loopvar_test.go:203: SIGQUIT: quit
        PC=0x7fff2048bc2a m=0 sigcode=0
        
        goroutine 0 gp=0x62ea900 m=0 mp=0x62ec600 [idle]:
        runtime.kevent(0x4, 0x0, 0x0, 0x7ffeeb4612c8, 0x40, 0x7ffeeb461298)
        	runtime/sys_darwin.go:458 +0x2e fp=0x7ffeeb461240 sp=0x7ffeeb461220 pc=0x57fb1ae
        runtime.netpoll(0xc000024020?)
        	runtime/netpoll_kqueue.go:141 +0xfd fp=0x7ffeeb461ae0 sp=0x7ffeeb461240 pc=0x57d643d
...
panic: test timed out after 9m0s
running tests:
	TestLoopVarHashes (1m57s)

goroutine 109 gp=0xc00016ea80 m=4 mp=0xc000080008 [running]:
panic({0xfe5b8a0?, 0xc0004bc240?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:778 +0x16f fp=0xc000070f10 sp=0xc000070e60 pc=0xfc02d2f
testing.(*M).startAlarm.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x385 fp=0xc000070fe0 sp=0xc000070f10 pc=0xfce3765
runtime.goexit({})
...
testing.tRunner.func2()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1683 +0x25 fp=0xc000123f70 sp=0xc000123f50 pc=0xfcdf545
testing.tRunner(0xc0002fc000, 0xfeb64e8)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1695 +0x125 fp=0xc000123fc0 sp=0xc000123f70 pc=0xfcdf4c5
testing.(*T).Run.gowrap1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x25 fp=0xc000123fe0 sp=0xc000123fc0 pc=0xfce04c5
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000123fe8 sp=0xc000123fe0 pc=0xfc3e4a1
created by testing.(*T).Run in goroutine 1
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x390

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-28 00:05 gotip-darwin-amd64-longtest go@ff8a2c0a cmd/go.TestScript/test_trimpath_test_suffix (log)
=== RUN   TestScript/test_trimpath_test_suffix
=== PAUSE TestScript/test_trimpath_test_suffix
=== CONT  TestScript/test_trimpath_test_suffix
    script_test.go:132: 2024-02-28T07:53:25Z
    script_test.go:134: $WORK=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-1421492415/tmpdir3883958679/test_trimpath_test_suffix4147380678
    script_test.go:156: 
        PATH=/Users/swarming/.swarming/w/ir/x/t/cmd-go-test-1421492415/tmpdir3883958679/testbin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/x/w/goroot/bin:/Users/swarming/.swarming/w/ir/cache/tools/bin:/Users/swarming/.swarming/w/ir/bbagent_utility_packages:/Users/swarming/.swarming/w/ir/bbagent_utility_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages:/Users/swarming/.swarming/w/ir/cipd_bin_packages/bin:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3:/Users/swarming/.swarming/w/ir/cipd_bin_packages/cpython3/bin:/Users/swarming/.swarming/w/ir/cache/cipd_client:/Users/swarming/.swarming/w/ir/cache/cipd_client/bin:/Users/swarming/.swarming/cipd_cache/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
        HOME=/no-home
        CCACHE_DISABLE=1
        GOARCH=amd64
...
        r13    0x0
        r14    0xc000536540
        r15    0x0
        rip    0x7ff815ea8a60
        rflags 0x246
        cs     0x7
        fs     0x0
        gs     0x0
    script_test.go:156: FAIL: testdata/script/test_trimpath_test_suffix.txt:3: go test -trimpath -v .: exit status 2
--- FAIL: TestScript/test_trimpath_test_suffix (1.81s)

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-28 17:41 gotip-darwin-amd64-longtest go@28e0052e reflect:mayMoreStackMove.TestChanOfGC [ABORT] (log)
=== RUN   TestChanOfGC
panic: deadlock in TestChanOfGC

goroutine 282 gp=0xc000007880 m=3 mp=0xc000051208 [running]:
panic({0xd8e37a0?, 0xd948738?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:778 +0x18f fp=0xc00004cf70 sp=0xc00004cec0 pc=0xd56ff8f
reflect_test.TestChanOfGC.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/reflect/all_test.go:6177 +0x87 fp=0xc00004cfe0 sp=0xc00004cf70 pc=0xd753d47
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00004cfe8 sp=0xc00004cfe0 pc=0xd5ae601
...
reflect_test.TestChanOfGC(0xc00016c1a0)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/reflect/all_test.go:6211 +0x3fd fp=0xc0000bbf70 sp=0xc0000bbdd0 pc=0xd7539fd
testing.tRunner(0xc00016c1a0, 0xd946568)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1689 +0xfb fp=0xc0000bbfc0 sp=0xc0000bbf70 pc=0xd65947b
testing.(*T).Run.gowrap1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x25 fp=0xc0000bbfe0 sp=0xc0000bbfc0 pc=0xd65a4a5
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000bbfe8 sp=0xc0000bbfe0 pc=0xd5ae601
created by testing.(*T).Run in goroutine 1
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x390
2024-02-28 20:21 gotip-darwin-amd64-longtest go@88b2f250 reflect:mayMoreStackMove.TestChanOfGC [ABORT] (log)
=== RUN   TestChanOfGC
panic: deadlock in TestChanOfGC

goroutine 225 gp=0xc000007c00 m=5 mp=0xc000051b08 [running]:
panic({0x21ba7a0?, 0x221f738?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:778 +0x18f fp=0xc000102770 sp=0xc0001026c0 pc=0x1e46f8f
reflect_test.TestChanOfGC.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/reflect/all_test.go:6177 +0x87 fp=0xc0001027e0 sp=0xc000102770 pc=0x202ad47
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001027e8 sp=0xc0001027e0 pc=0x1e85601
...
reflect_test.TestChanOfGC(0xc000184340)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/reflect/all_test.go:6199 +0x565 fp=0xc00025ff70 sp=0xc00025fdd0 pc=0x202ab65
testing.tRunner(0xc000184340, 0x221d568)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1689 +0xfb fp=0xc00025ffc0 sp=0xc00025ff70 pc=0x1f3047b
testing.(*T).Run.gowrap1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x25 fp=0xc00025ffe0 sp=0xc00025ffc0 pc=0x1f314a5
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00025ffe8 sp=0xc00025ffe0 pc=0x1e85601
created by testing.(*T).Run in goroutine 1
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x390

watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
default <- builder ~ `(gotip|go1\.\d\d)-darwin-amd64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-02-28 22:57 gotip-darwin-amd64-longtest go@288615dd reflect:mayMoreStackMove.TestChanOfGC [ABORT] (log)
=== RUN   TestChanOfGC
panic: deadlock in TestChanOfGC

goroutine 267 gp=0xc000105340 m=9 mp=0xc000081208 [running]:
panic({0xe7ae7a0?, 0xe813738?})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:778 +0x18f fp=0xc0000ac770 sp=0xc0000ac6c0 pc=0xe43af8f
reflect_test.TestChanOfGC.func1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/reflect/all_test.go:6177 +0x87 fp=0xc0000ac7e0 sp=0xc0000ac770 pc=0xe61ed47
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000ac7e8 sp=0xc0000ac7e0 pc=0xe479601
...
reflect_test.TestChanOfGC(0xc0001221a0)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/reflect/all_test.go:6201 +0x1da fp=0xc000115f70 sp=0xc000115dd0 pc=0xe61e7da
testing.tRunner(0xc0001221a0, 0xe811568)
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1689 +0xfb fp=0xc000115fc0 sp=0xc000115f70 pc=0xe52447b
testing.(*T).Run.gowrap1()
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x25 fp=0xc000115fe0 sp=0xc000115fc0 pc=0xe5254a5
runtime.goexit({})
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000115fe8 sp=0xc000115fe0 pc=0xe479601
created by testing.(*T).Run in goroutine 1
	/Users/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:1742 +0x390

watchflakes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin
Projects
Status: In Progress
Status: No status
Development

No branches or pull requests

6 participants