Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: time out on LUCI Linux/PPC64x builder #65171

Open
cherrymui opened this issue Jan 19, 2024 · 63 comments
Open

x/build: time out on LUCI Linux/PPC64x builder #65171

cherrymui opened this issue Jan 19, 2024 · 63 comments
Labels
arch-ppc64x Builders x/build issues (builders, bots, dashboards) NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@cherrymui
Copy link
Member

cherrymui commented Jan 19, 2024

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
@cherrymui cherrymui changed the title cmd/cgo/internal/test*: time out on LUCI gotip-linux-ppc64le builder all: time out on LUCI gotip-linux-ppc64le builder Jan 19, 2024
@cherrymui cherrymui changed the title all: time out on LUCI gotip-linux-ppc64le builder x/build: time out on LUCI gotip-linux-ppc64le builder Jan 19, 2024
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Jan 19, 2024
@gopherbot gopherbot added this to the Unreleased milestone Jan 19, 2024
@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ==  "gotip-linux-ppc64le" && (`test timed out` || `SIGQUIT` || `context deadline exceeded`)
2023-12-06 21:50 gotip-linux-ppc64le go@e914671f cmd/cgo/internal/testshared.TestIssue62277 (log)
=== RUN   TestIssue62277
panic: test timed out after 9m0s
running tests:
	TestIssue62277 (1s)

goroutine 142 gp=0xc0001d61c0 m=2 mp=0xc000044808 [running]:
panic({0x1c37c0?, 0xc0000b20c0?})
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:779 +0x174 fp=0xc000182ee0 sp=0xc000182e20 pc=0x4d1e4
testing.(*M).startAlarm.func1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x338 fp=0xc000182fc0 sp=0xc000182ee0 pc=0x104058
...
os/exec.(*Cmd).writerDescriptor.func1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:577 +0x4c fp=0xc000185f20 sp=0xc000185eb0 pc=0x13390c
os/exec.(*Cmd).Start.func2(0x0?)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:724 +0x48 fp=0xc000185f98 sp=0xc000185f20 pc=0x1344e8
os/exec.(*Cmd).Start.gowrap1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:736 +0x58 fp=0xc000185fc0 sp=0xc000185f98 pc=0x134488
runtime.goexit({})
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_ppc64x.s:1018 +0x4 fp=0xc000185fc0 sp=0xc000185fc0 pc=0x8bb54
created by os/exec.(*Cmd).Start in goroutine 139
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:723 +0x8ec
2023-12-12 20:05 gotip-linux-ppc64le go@962dade4 cmd/cgo/internal/testshared.TestIssue44031 (log)
=== RUN   TestIssue44031
SIGQUIT: quit
PC=0x8d22c m=0 sigcode=0

goroutine 0 gp=0x360f40 m=0 mp=0x361bc0 [idle]:
runtime.futex(0x361d00, 0x80, 0x0, 0x0, 0x0, 0x0)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_ppc64x.s:644 +0x1c fp=0x7fffe1a6a2c8 sp=0x7fffe1a6a2c8 pc=0x8d22c
runtime.futexsleep(0x1?, 0x55d30?, 0x361bc0?)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/os_linux.go:69 +0x30 fp=0x7fffe1a6a328 sp=0x7fffe1a6a2c8 pc=0x4a5d0
runtime.notesleep(0x361d00)
...
r20  0x4	r21  0x2
r22  0x7fffe1a5aa00	r23  0x1
r24  0x0	r25  0x0
r26  0x0	r27  0xe09d261a
r28  0xd617e65a	r29  0xc00002f7a8
r30  0x360f40	r31  0x1cda0
pc   0x8d22c	ctr  0x0
link 0x4a5d0	xer  0x0
ccr  0x34420884	trap 0xc00
*** Test killed with quit: ran too long (10m0s).
2023-12-15 18:30 gotip-linux-ppc64le go@5e939b3a cmd/cgo/internal/testcarchive.TestPIE (log)
=== RUN   TestPIE
    carchive_test.go:956: [go tool cgo -objdir /home/swarming/.swarming/w/ir/x/t/carchive_test2687818152/_obj664230977 -exportheader p.h p/p.go]
panic: test timed out after 9m0s
running tests:
	TestPIE (25s)

goroutine 94 gp=0xc0002fe1c0 m=3 mp=0xc000045008 [running]:
panic({0x1803a0?, 0xc000182730?})
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:779 +0x174 fp=0xc000089ee0 sp=0xc000089e20 pc=0x4ceb4
testing.(*M).startAlarm.func1()
...
os/exec.(*Cmd).writerDescriptor.func1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:577 +0x4c fp=0xc00028ef20 sp=0xc00028eeb0 pc=0x10814c
os/exec.(*Cmd).Start.func2(0x0?)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:724 +0x48 fp=0xc00028ef98 sp=0xc00028ef20 pc=0x108d28
os/exec.(*Cmd).Start.gowrap1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:736 +0x58 fp=0xc00028efc0 sp=0xc00028ef98 pc=0x108cc8
runtime.goexit({})
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_ppc64x.s:1018 +0x4 fp=0xc00028efc0 sp=0xc00028efc0 pc=0x8b874
created by os/exec.(*Cmd).Start in goroutine 91
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:723 +0x8ec
2023-12-15 18:30 gotip-linux-ppc64le go@5e939b3a cmd/cgo/internal/testshared.TestGlobal (log)
=== RUN   TestGlobal
SIGQUIT: quit
PC=0x8d22c m=0 sigcode=0

goroutine 0 gp=0x360f40 m=0 mp=0x361bc0 [idle]:
runtime.futex(0x361d00, 0x80, 0x0, 0x0, 0x0, 0x0)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_ppc64x.s:644 +0x1c fp=0x7fffe0703828 sp=0x7fffe0703828 pc=0x8d22c
runtime.futexsleep(0x7d870?, 0x55d30?, 0x361bc0?)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/os_linux.go:69 +0x30 fp=0x7fffe0703888 sp=0x7fffe0703828 pc=0x4a5d0
runtime.notesleep(0x361d00)
...
r20  0x4	r21  0x1
r22  0x7fffe06f3f60	r23  0x1
r24  0x0	r25  0x0
r26  0x0	r27  0xdf0a0c9
r28  0x9e3a2147	r29  0xdca58061
r30  0x360f40	r31  0x1cda0
pc   0x8d22c	ctr  0x0
link 0x4a5d0	xer  0x0
ccr  0x34420888	trap 0xc00
*** Test killed with quit: ran too long (10m0s).
2023-12-15 20:51 gotip-linux-ppc64le go@f8170cc0 cmd/go.TestScript (log)
=== RUN   TestScript/cgo_path
=== PAUSE TestScript/cgo_path
=== CONT  TestScript/cgo_path
    script_test.go:132: 2023-12-15T22:29:34Z
    script_test.go:134: $WORK=/home/swarming/.swarming/w/ir/x/t/cmd-go-test-1289633616/tmpdir1500409099/cgo_path280284189
    script_test.go:156: 
        PATH=/home/swarming/.swarming/w/ir/x/t/cmd-go-test-1289633616/tmpdir1500409099/testbin:/home/swarming/.swarming/w/ir/x/w/goroot/bin:/home/swarming/.swarming/w/ir/x/w/goroot/bin:/home/swarming/.swarming/w/ir/x/w/goroot/bin:/home/swarming/.swarming/w/ir/cache/tools/bin:/home/swarming/.swarming/w/ir/bbagent_utility_packages:/home/swarming/.swarming/w/ir/bbagent_utility_packages/bin:/home/swarming/.swarming/w/ir/cipd_bin_packages:/home/swarming/.swarming/w/ir/cipd_bin_packages/bin:/home/swarming/.swarming/w/ir/cache/cipd_client:/home/swarming/.swarming/w/ir/cache/cipd_client/bin:/home/swarming/.swarming/cipd_cache/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
        HOME=/no-home
        CCACHE_DISABLE=1
        GOARCH=ppc64le
...
        r24  0x0	r25  0x0
        r26  0x0	r27  0x110
        r28  0x81e9bc6b	r29  0xc0000581e0
        r30  0x10cf6040	r31  0x100418f4
        pc   0x100082f4	ctr  0x0
        link 0x1000828c	xer  0x0
        ccr  0x34428888	trap 0xc00
        [exit status 2]
    script_test.go:156: FAIL: testdata/script/cgo_path.txt:14: context deadline exceeded
--- FAIL: TestScript/cgo_path (7.88s)

watchflakes

@pmur
Copy link
Contributor

pmur commented Jan 22, 2024

I wonder if LUCI needs to set GO_TEST_TIMEOUT_SCALE=2 for ppc64{,le}? I don't think I see that set in the logs. The POWER8 builders can be particularly slow. I think the VMs are also subject to sharing CPU time with whatever else is running at OSU.

@cherrymui
Copy link
Member Author

Yeah, GO_TEST_TIMEOUT_SCALE=2 doesn't seem to be set. The old ppc64le builders set GO_TEST_TIMEOUT_SCALE=2, so we probably should do that for LCUI builders as well.

cc @mknyszek

@cherrymui cherrymui added arch-ppc64x NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jan 23, 2024
@mknyszek
Copy link
Contributor

I'll send a CL for this today. Thanks.

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/557857 mentions this issue: main.star: add a host-based test timeout scale

@dmitshur dmitshur moved this to In Progress in Go Release Jan 23, 2024
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jan 23, 2024
@dmitshur dmitshur added the FixPending Issues that have a fix which has not yet been reviewed or submitted. label Jan 23, 2024
gopherbot pushed a commit to golang/build that referenced this issue Jan 24, 2024
This change adds a baseline test time timeout scale that's derived from
the host to better align with the old infrastructure. Such hosts are
listed in SLOW_HOSTS along with their timeout scale.

This CL also changes the way timeout scaling factors compose. Instead of
taking the highest one, this change multiplies them. Although the only
case where this matters seems to mostly work fine with taking the
maximum (-longtest-race) it seems easier to reason about the composition
of scaling factors this way.

Fixes golang/go#65171.

Change-Id: If95fc8a036c2a735396c854cbcf09af7cba0c9f3
Reviewed-on: https://go-review.googlesource.com/c/build/+/557857
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ==  "gotip-linux-ppc64le" && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2023-12-15 20:30 gotip-linux-ppc64le go@3313bbb4 runtime.TestCgoPprofPIE (log)
=== RUN   TestCgoPprofPIE
=== PAUSE TestCgoPprofPIE
=== CONT  TestCgoPprofPIE
    crash_test.go:152: running /home/swarming/.swarming/w/ir/x/w/goroot/bin/go build -o /home/swarming/.swarming/w/ir/x/t/go-build49346469/testprogcgo_-buildmode=pie.exe -buildmode=pie

watchflakes

@cherrymui
Copy link
Member Author

cherrymui commented Jan 25, 2024

Closing. Should be fixed by the CL above. (Apparently gopherbot doesn't do it because it is checked in to a branch.)

@github-project-automation github-project-automation bot moved this from In Progress to Done in Go Release Jan 25, 2024
@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~  `(gotip|go1\.\d\d)-linux-ppc64le` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2023-12-19 15:57 go1.22-linux-ppc64le release-branch.go1.22@fb23428a cmd/cgo/internal/testshared.TestIssue44031 [ABORT] (log)
=== RUN   TestIssue44031
    shared_test.go:89: go install -buildmode=shared -linkshared ./issue44031/a
    shared_test.go:89: go install -buildmode=shared -linkshared ./issue44031/b
panic: test timed out after 9m0s
running tests:
	TestIssue44031 (51s)

goroutine 167 gp=0xc000123880 m=4 mp=0xc000080008 [running]:
panic({0x1c37c0?, 0xc000020240?})
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:779 +0x174 fp=0xc0000d2ee0 sp=0xc0000d2e20 pc=0x4d1c4
...
os/exec.(*Cmd).writerDescriptor.func1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:577 +0x4c fp=0xc0000d1f20 sp=0xc0000d1eb0 pc=0x133a6c
os/exec.(*Cmd).Start.func2(0x0?)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:724 +0x48 fp=0xc0000d1f98 sp=0xc0000d1f20 pc=0x134648
os/exec.(*Cmd).Start.gowrap1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:736 +0x58 fp=0xc0000d1fc0 sp=0xc0000d1f98 pc=0x1345e8
runtime.goexit({})
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_ppc64x.s:1018 +0x4 fp=0xc0000d1fc0 sp=0xc0000d1fc0 pc=0x8bbb4
created by os/exec.(*Cmd).Start in goroutine 141
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:723 +0x8ec

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~  `(gotip|go1\.\d\d)-linux-ppc64le` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || status == "ABORT")
2024-01-31 21:44 x_tools-go1.20-linux-ppc64le tools@5f906919 release-branch.go1.20@746a0727 x/tools/gopls/internal/test/integration/workspace.TestReloadOnlyOnce/default [ABORT] (log)
=== RUN   TestReloadOnlyOnce/default
DETECTED A HANGING GO COMMAND

The gopls test runner has detected a hanging go command. In order to debug
this, the output of ps and lsof/fstat is printed below.

See golang/go#54461 for more details.

ps axo ppid,pid,command:
-------------------------
...
panic: running lsof: exec: "lsof": executable file not found in $PATH

goroutine 14940 [running]:
golang.org/x/tools/internal/gocommand.HandleHangingGoCommand(0xc001a27d10)
	/home/swarming/.swarming/w/ir/x/w/targetrepo1846607283/internal/gocommand/invoke.go:439 +0x390
golang.org/x/tools/internal/gocommand.runCmdContext({0xe60a08, 0xc001f13f20}, 0xc00245f760)
	/home/swarming/.swarming/w/ir/x/w/targetrepo1846607283/internal/gocommand/invoke.go:375 +0x3d4
golang.org/x/tools/internal/gocommand.(*Invocation).run(0xc000d6b450, {0xe60a08, 0xc001f13f20}, {0xe57c80?, 0xc000208870}, {0xe57c80?, 0xc0002088d0})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1846607283/internal/gocommand/invoke.go:270 +0xebc
golang.org/x/tools/internal/gocommand.(*Invocation).runWithFriendlyError(0xc000d6b408?, {0xe60a08, 0xc001f13f20}, {0xe57c80?, 0xc000208870?}, {0xe57c80?, 0xc0002088d0?})
...
golang.org/x/tools/gopls/internal/cache.(*Snapshot).WorkspaceMetadata(0xc003026a20, {0xe60a08?, 0xc001f12c30?})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1846607283/gopls/internal/cache/snapshot.go:1083 +0x68
golang.org/x/tools/gopls/internal/server.(*server).diagnose(0xc001c0f200, {0xe60a08, 0xc002a28d20}, 0xc003026a20)
	/home/swarming/.swarming/w/ir/x/w/targetrepo1846607283/gopls/internal/server/diagnostics.go:366 +0x4cc
golang.org/x/tools/gopls/internal/server.(*server).diagnoseSnapshot(0xc001c0f200, 0xc003026a20, {0xc00184bb70, 0x1, 0x1}, 0x989680)
	/home/swarming/.swarming/w/ir/x/w/targetrepo1846607283/gopls/internal/server/diagnostics.go:223 +0x574
golang.org/x/tools/gopls/internal/server.(*server).diagnoseChangedViews.func1(0xc003026a20, {0xc00184bb70?, 0xc00464b860?, 0xc000e21f88?})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1846607283/gopls/internal/server/diagnostics.go:135 +0xe8
created by golang.org/x/tools/gopls/internal/server.(*server).diagnoseChangedViews
	/home/swarming/.swarming/w/ir/x/w/targetrepo1846607283/gopls/internal/server/diagnostics.go:132 +0x518
2024-01-31 21:44 x_tools-go1.21-linux-ppc64le tools@5f906919 release-branch.go1.21@916e6cdd x/tools/gopls/internal/test/integration/misc.TestHoverBrokenImport_Issue60592/default [ABORT] (log)
=== RUN   TestHoverBrokenImport_Issue60592/default
DETECTED A HANGING GO COMMAND

The gopls test runner has detected a hanging go command. In order to debug
this, the output of ps and lsof/fstat is printed below.

See golang/go#54461 for more details.

ps axo ppid,pid,command:
-------------------------
...
panic: running lsof: exec: "lsof": executable file not found in $PATH [recovered]
	panic: /home/swarming/.swarming/w/ir/x/w/targetrepo4225489592/gopls/internal/protocol/tsserver.go:100: server panic in textDocument/hover request

goroutine 26274 [running]:
golang.org/x/tools/gopls/internal/util/bug.report({0xc00ca10c00, 0x2a})
	/home/swarming/.swarming/w/ir/x/w/targetrepo4225489592/gopls/internal/util/bug/bug.go:80 +0x4f0
golang.org/x/tools/gopls/internal/util/bug.Reportf({0xc57ce5?, 0x12?}, {0xc0014b62b0?, 0x51b0c?, 0x0?})
	/home/swarming/.swarming/w/ir/x/w/targetrepo4225489592/gopls/internal/util/bug/bug.go:51 +0x58
golang.org/x/tools/gopls/internal/protocol.serverDispatch.func1()
	/home/swarming/.swarming/w/ir/x/w/targetrepo4225489592/gopls/internal/protocol/tsserver.go:100 +0xa8
...
      1 4174727 [link] <defunct>
      1 4184384 [compile] <defunct>
      1 4184389 [compile] <defunct>
      1 4184390 [compile] <defunct>
      1 4184421 [asm] <defunct>
      1 4185715 [compile] <defunct>
      1 4190463 [compile] <defunct>
      1 4190464 [compile] <defunct>
      1 4190465 [compile] <defunct>
      1 4190466 [compile] <defunct>
2024-02-02 15:22 gotip-linux-ppc64le go@149db960 cmd/cgo/internal/testcarchive.TestPreemption [ABORT] (log)
=== RUN   TestPreemption
=== PAUSE TestPreemption
=== CONT  TestPreemption
panic: test timed out after 9m0s
running tests:
	TestDeepStack (1s)
	TestPreemption (1s)
	TestSIGPROF (1s)
	TestSharedObject (1s)

...
os/exec.(*Cmd).writerDescriptor.func1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:577 +0x4c fp=0xc000107f20 sp=0xc000107eb0 pc=0x10879c
os/exec.(*Cmd).Start.func2(0x0?)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:724 +0x48 fp=0xc000107f98 sp=0xc000107f20 pc=0x109378
os/exec.(*Cmd).Start.gowrap1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:736 +0x58 fp=0xc000107fc0 sp=0xc000107f98 pc=0x109318
runtime.goexit({})
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_ppc64x.s:1018 +0x4 fp=0xc000107fc0 sp=0xc000107fc0 pc=0x8bec4
created by os/exec.(*Cmd).Start in goroutine 99
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:723 +0x8ec
2024-02-02 15:22 gotip-linux-ppc64le go@149db960 cmd/cgo/internal/testshared.TestTestInstalledShared [ABORT] (log)
=== RUN   TestTestInstalledShared
panic: test timed out after 9m0s
running tests:
	TestTestInstalledShared (14s)

goroutine 147 gp=0xc0005028c0 m=2 mp=0xc000044808 [running]:
panic({0x1c3740?, 0xc00011e8d0?})
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:768 +0x174 fp=0xc0002a2ee0 sp=0xc0002a2e20 pc=0x4d374
testing.(*M).startAlarm.func1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/testing/testing.go:2366 +0x338 fp=0xc0002a2fc0 sp=0xc0002a2ee0 pc=0x104758
...
os/exec.(*Cmd).writerDescriptor.func1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:577 +0x4c fp=0xc0002a0f20 sp=0xc0002a0eb0 pc=0x13409c
os/exec.(*Cmd).Start.func2(0x2107a8?)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:724 +0x48 fp=0xc0002a0f98 sp=0xc0002a0f20 pc=0x134c78
os/exec.(*Cmd).Start.gowrap1()
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:736 +0x58 fp=0xc0002a0fc0 sp=0xc0002a0f98 pc=0x134c18
runtime.goexit({})
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_ppc64x.s:1018 +0x4 fp=0xc0002a0fc0 sp=0xc0002a0fc0 pc=0x8c184
created by os/exec.(*Cmd).Start in goroutine 96
	/home/swarming/.swarming/w/ir/x/w/goroot/src/os/exec/exec.go:723 +0x8ec
2024-02-02 15:22 gotip-linux-ppc64le go@149db960 cmd/go.TestScript (log)
=== RUN   TestScript/test_ppc64le_cgo_inline_plt
=== PAUSE TestScript/test_ppc64le_cgo_inline_plt
=== CONT  TestScript/test_ppc64le_cgo_inline_plt
    script_test.go:132: 2024-02-02T16:42:50Z
    script_test.go:134: $WORK=/home/swarming/.swarming/w/ir/x/t/cmd-go-test-815285317/tmpdir2819220712/test_ppc64le_cgo_inline_plt696775735
    script_test.go:156: 
        PATH=/home/swarming/.swarming/w/ir/x/t/cmd-go-test-815285317/tmpdir2819220712/testbin:/home/swarming/.swarming/w/ir/x/w/goroot/bin:/home/swarming/.swarming/w/ir/x/w/goroot/bin:/home/swarming/.swarming/w/ir/x/w/goroot/bin:/home/swarming/.swarming/w/ir/cache/tools/bin:/home/swarming/.swarming/w/ir/bbagent_utility_packages:/home/swarming/.swarming/w/ir/bbagent_utility_packages/bin:/home/swarming/.swarming/w/ir/cipd_bin_packages:/home/swarming/.swarming/w/ir/cipd_bin_packages/bin:/home/swarming/.swarming/w/ir/cache/cipd_client:/home/swarming/.swarming/w/ir/cache/cipd_client/bin:/home/swarming/.swarming/cipd_cache/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
        HOME=/no-home
        CCACHE_DISABLE=1
        GOARCH=ppc64le
...
        r22  0x7fffd77d03a0	r23  0x1
        r24  0x0	r25  0xf0
        r26  0x100	r27  0x110
        r28  0x15de4464	r29  0xc0000557c0
        r30  0x10d062a0	r31  0x10012cf0
        pc   0x10087cfc	ctr  0x0
        link 0x10041d70	xer  0x0
        ccr  0x34428888	trap 0xc00
    script_test.go:156: FAIL: testdata/script/test_ppc64le_cgo_inline_plt.txt:17: go build -ldflags=-linkmode=internal: exit status 2
--- FAIL: TestScript/test_ppc64le_cgo_inline_plt (78.92s)

watchflakes

@gopherbot gopherbot reopened this Feb 2, 2024
@pmur
Copy link
Contributor

pmur commented Feb 2, 2024

Is there more context to be found about what test is suspected of hanging, and what the timeout is?

I am rebuilding the container images to include lsof.

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power10 tools@254baba6 go@09f9b5e3 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power8 tools@254baba6 go@dea12628 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power10 tools@254baba6 go@18c2461a x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power8 tools@254baba6 go@18c2461a x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power10 tools@254baba6 go@137328f9 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power10 tools@254baba6 go@0531768b x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power8 tools@254baba6 go@8ac0a7c5 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power10 tools@254baba6 go@6a2fb154 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power8 tools@254baba6 go@6a2fb154 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power10 tools@254baba6 go@5ebb4e28 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power8 tools@254baba6 go@5ebb4e28 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power10 tools@254baba6 go@47da5a30 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power8 tools@254baba6 go@9060fa5a x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power8 tools@254baba6 go@47da5a30 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power10 tools@254baba6 go@04807d3a x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power8 tools@254baba6 go@63f762bc x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power10 tools@254baba6 go@01e1e5c2 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2024-11-15 16:16 x_tools-gotip-linux-ppc64_power8 tools@254baba6 go@01e1e5c2 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@timothy-king
Copy link
Contributor

Another dup of #70363 (comment)

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-11-26 15:29 x_tools-gotip-linux-ppc64le_power8 tools@30a3bd93 go@592da0ba x/tools/gopls/internal/test/integration/misc.TestReferencesInTestVariants/default [ABORT] (log)
=== RUN   TestReferencesInTestVariants/default
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x475a78]

goroutine 57320 [running]:
golang.org/x/tools/internal/gcimporter.iexportCommon.func1()
	/home/swarming/.swarming/w/ir/x/w/targetrepo3538615584/internal/gcimporter/iexport.go:344 +0xb0
panic({0xc81980?, 0x187ed10?})
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:787 +0x154
go/token.(*File).Offset(...)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/go/token/position.go:316
golang.org/x/tools/internal/gcimporter.(*iexporter).fileIndexAndOffset(0xc007e17a00, 0x0, 0x1000000000000011)
	/home/swarming/.swarming/w/ir/x/w/targetrepo3538615584/internal/gcimporter/iexport.go:647 +0x1f8
golang.org/x/tools/internal/gcimporter.(*exportWriter).posV2(0xc007472b90, 0x1000000000000011)
	/home/swarming/.swarming/w/ir/x/w/targetrepo3538615584/internal/gcimporter/iexport.go:866 +0x94
golang.org/x/tools/internal/gcimporter.(*exportWriter).pos(0xc007472b98?, 0x9c156?)
	/home/swarming/.swarming/w/ir/x/w/targetrepo3538615584/internal/gcimporter/iexport.go:848 +0x60
golang.org/x/tools/internal/gcimporter.(*iexporter).doDecl(0xc007e17a00, {0x1122fe0, 0xc007a646c0})
	/home/swarming/.swarming/w/ir/x/w/targetrepo3538615584/internal/gcimporter/iexport.go:705 +0x6f8
golang.org/x/tools/internal/gcimporter.iexportCommon({0x110eb00, 0xc00a8bc300}, 0xc008463b00, 0x0, 0x1, 0x2, {0xc007186ec8, 0x1, 0xc00b6e6800?})
	/home/swarming/.swarming/w/ir/x/w/targetrepo3538615584/internal/gcimporter/iexport.go:390 +0x35c
golang.org/x/tools/internal/gcimporter.IExportShallow(0xc008463b00, 0xc007a645a0, 0xc008463b00?)
	/home/swarming/.swarming/w/ir/x/w/targetrepo3538615584/internal/gcimporter/iexport.go:286 +0x84
golang.org/x/tools/gopls/internal/cache.(*typeCheckBatch).checkPackageForImport.func3()
	/home/swarming/.swarming/w/ir/x/w/targetrepo3538615584/gopls/internal/cache/check.go:601 +0x4c
created by golang.org/x/tools/gopls/internal/cache.(*typeCheckBatch).checkPackageForImport in goroutine 57318
	/home/swarming/.swarming/w/ir/x/w/targetrepo3538615584/gopls/internal/cache/check.go:600 +0x528

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-12-07 02:32 gotip-linux-ppc64_power10 go@8c3e3915 runtime.TestChanSendBarrier [ABORT] (log)
=== RUN   TestChanSendBarrier
=== PAUSE TestChanSendBarrier

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-12-09 22:57 x_tools-go1.22-linux-ppc64_power8 tools@6ebf95a6 release-branch.go1.22@8f3f22ee x/tools/gopls/internal/test/marker.Test/codeaction/addtest.txt [ABORT] (log)
=== RUN   Test/codeaction/addtest.txt
=== PAUSE Test/codeaction/addtest.txt
=== CONT  Test/codeaction/addtest.txt
2024-12-09 22:57 x_tools-go1.22-linux-ppc64_power8 tools@6ebf95a6 release-branch.go1.22@8f3f22ee x/tools/gopls/internal/util/safetoken.TestGoplsSourceDoesNotCallTokenFileMethods [ABORT] (log)
=== RUN   TestGoplsSourceDoesNotCallTokenFileMethods
signal: killed

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-12-19 18:59 gotip-linux-ppc64_power10 go@e63eb98e runtime.TestChanSendBarrier [ABORT] (log)
=== RUN   TestChanSendBarrier
=== PAUSE TestChanSendBarrier

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-12-23 21:52 x_tools-go1.22-linux-ppc64_power8 tools@02033b26 release-branch.go1.22@8f3f22ee x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-12-27 21:47 x_tools-go1.22-linux-ppc64_power8 tools@a6adab9c release-branch.go1.22@8f3f22ee x/tools/gopls/internal/test/marker.Test/codeaction/addtest.txt [ABORT] (log)
=== RUN   Test/codeaction/addtest.txt
=== PAUSE Test/codeaction/addtest.txt
=== CONT  Test/codeaction/addtest.txt
2024-12-27 21:47 x_tools-go1.22-linux-ppc64_power8 tools@a6adab9c release-branch.go1.22@8f3f22ee x/tools/gopls/internal/util/safetoken.TestGoplsSourceDoesNotCallTokenFileMethods [ABORT] (log)
=== RUN   TestGoplsSourceDoesNotCallTokenFileMethods
signal: killed

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2024-12-30 17:57 x_tools-go1.22-linux-ppc64_power8 tools@d6cc3cdc release-branch.go1.22@8f3f22ee x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2024-12-30 17:57 x_tools-go1.22-linux-ppc64_power8 tools@d6cc3cdc release-branch.go1.22@8f3f22ee x/tools/gopls/internal/util/safetoken.TestGoplsSourceDoesNotCallTokenFileMethods [ABORT] (log)
=== RUN   TestGoplsSourceDoesNotCallTokenFileMethods
signal: killed

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-01-02 17:51 x_tools-go1.22-linux-ppc64_power8 tools@960f0f41 release-branch.go1.22@8f3f22ee x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2025-01-02 17:51 x_tools-go1.22-linux-ppc64_power8 tools@960f0f41 release-branch.go1.22@8f3f22ee x/tools/gopls/internal/util/safetoken.TestGoplsSourceDoesNotCallTokenFileMethods [ABORT] (log)
=== RUN   TestGoplsSourceDoesNotCallTokenFileMethods
signal: killed

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-01-07 23:38 x_tools-go1.22-linux-ppc64_power8 tools@b4e093ec release-branch.go1.22@8f3f22ee x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2025-01-07 23:38 x_tools-go1.22-linux-ppc64_power8 tools@b4e093ec release-branch.go1.22@8f3f22ee x/tools/gopls/internal/util/safetoken.TestGoplsSourceDoesNotCallTokenFileMethods [ABORT] (log)
=== RUN   TestGoplsSourceDoesNotCallTokenFileMethods
signal: killed

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-01-10 22:09 x_tools-go1.22-linux-ppc64_power8 tools@8f9869c7 release-branch.go1.22@ae9996f9 x/tools/gopls/internal/test/marker.Test/codeaction/addtest.txt [ABORT] (log)
=== RUN   Test/codeaction/addtest.txt
=== PAUSE Test/codeaction/addtest.txt
=== CONT  Test/codeaction/addtest.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-01-16 19:36 go1.24-linux-ppc64le_power8 release-branch.go1.24@8336dfde runtime.TestGdbConst [ABORT] (log)
=== RUN   TestGdbConst
=== PAUSE TestGdbConst
=== CONT  TestGdbConst
    runtime-gdb_test.go:79: gdb version 9.2

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-01-22 15:58 x_tools-gotip-linux-ppc64le_power10 tools@9f4a509f go@f6d17c54 x/tools/gopls/internal/test/integration/completion.TestDefinition/default [ABORT] (log)
=== RUN   TestDefinition/default
DETECTED A HANGING GO COMMAND

The gopls test runner has detected a hanging go command. In order to debug
this, the output of ps and lsof/fstat is printed below.

See golang/go#54461 for more details.

ps axo ppid,pid,command:
-------------------------
...
panic: running lsof: exec: "lsof": executable file not found in $PATH

goroutine 25768 [running]:
golang.org/x/tools/internal/gocommand.HandleHangingGoCommand({0xc01197e6f8?, 0xc01197e594?, 0x189c2c0?}, 0xc00d8e8180)
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/gocommand/invoke.go:455 +0x390
golang.org/x/tools/internal/gocommand.runCmdContext({0x1106d30, 0xc011055260}, 0xc00d8e8180)
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/gocommand/invoke.go:391 +0x3b8
golang.org/x/tools/internal/gocommand.(*Invocation).run(0xc01197eae8, {0x1106d30, 0xc011055260}, {0x10fc4c0, 0xc011055380}, {0x10fc4c0, 0xc0110553b0})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/gocommand/invoke.go:285 +0xc3c
golang.org/x/tools/internal/gocommand.(*Invocation).runWithFriendlyError(0xc01197eaa0?, {0x1106d30, 0xc011055260}, {0x10fc4c0?, 0xc011055380?}, {0x10fc4c0, 0xc0110553b0})
...
golang.org/x/tools/go/packages.goListDriver(0xc008333188, 0xc0119842e0, {0x0, 0x0}, {0xc011984220, 0x2, 0x0?})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/go/packages/golist.go:210 +0x770
golang.org/x/tools/go/packages.defaultDriver.func1(0x8?, {0xc011984220?, 0x0?, 0xc00f57b410?})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/go/packages/packages.go:343 +0x6c
golang.org/x/tools/go/packages.callDriverOnChunks.func1()
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/go/packages/packages.go:390 +0x6c
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/home/swarming/.swarming/w/ir/x/w/gopath/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78 +0x60
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 25648
	/home/swarming/.swarming/w/ir/x/w/gopath/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:75 +0xb4
2025-01-22 15:58 x_tools-gotip-linux-ppc64le_power10 tools@9f4a509f go@f6d17c54 x/tools/gopls/internal/test/integration/diagnostics.TestNoMod/without_workspace_module/default [ABORT] (log)
=== RUN   TestNoMod/without_workspace_module/default
DETECTED A HANGING GO COMMAND

The gopls test runner has detected a hanging go command. In order to debug
this, the output of ps and lsof/fstat is printed below.

See golang/go#54461 for more details.

ps axo ppid,pid,command:
-------------------------
...
panic: running lsof: exec: "lsof": executable file not found in $PATH

goroutine 25349 [running]:
golang.org/x/tools/internal/gocommand.HandleHangingGoCommand({0xc0065b26f8?, 0xc0065b2594?, 0x188c600?}, 0xc00e27cd80)
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/gocommand/invoke.go:455 +0x390
golang.org/x/tools/internal/gocommand.runCmdContext({0x1101358, 0xc00f31ab70}, 0xc00e27cd80)
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/gocommand/invoke.go:391 +0x3b8
golang.org/x/tools/internal/gocommand.(*Invocation).run(0xc0065b2ae8, {0x1101358, 0xc00f31ab70}, {0x10f6b80, 0xc00f31ac90}, {0x10f6b80, 0xc00f31acc0})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/gocommand/invoke.go:285 +0xc3c
golang.org/x/tools/internal/gocommand.(*Invocation).runWithFriendlyError(0xc0065b2aa0?, {0x1101358, 0xc00f31ab70}, {0x10f6b80?, 0xc00f31ac90?}, {0x10f6b80, 0xc00f31acc0})
...
golang.org/x/tools/go/packages.goListDriver(0xc0092280e8, 0xc00ef91260, {0x0, 0x0}, {0xc00ef911a0, 0x2, 0xc00f030980?})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/go/packages/golist.go:210 +0x770
golang.org/x/tools/go/packages.defaultDriver.func1(0x0?, {0xc00ef911a0?, 0x1101390?, 0xc00f015a90?})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/go/packages/packages.go:343 +0x6c
golang.org/x/tools/go/packages.callDriverOnChunks.func1()
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/go/packages/packages.go:390 +0x6c
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/home/swarming/.swarming/w/ir/x/w/gopath/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78 +0x60
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 25347
	/home/swarming/.swarming/w/ir/x/w/gopath/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:75 +0xb4
2025-01-22 15:58 x_tools-gotip-linux-ppc64le_power10 tools@9f4a509f go@f6d17c54 x/tools/gopls/internal/test/integration/misc.TestGoToStdlibDefinition_Issue37045/default [ABORT] (log)
=== RUN   TestGoToStdlibDefinition_Issue37045/default
DETECTED A HANGING GO COMMAND

The gopls test runner has detected a hanging go command. In order to debug
this, the output of ps and lsof/fstat is printed below.

See golang/go#54461 for more details.

ps axo ppid,pid,command:
-------------------------
...
panic: running lsof: exec: "lsof": executable file not found in $PATH

goroutine 13472 [running]:
golang.org/x/tools/internal/gocommand.HandleHangingGoCommand({0xc0000471c0?, 0xc00004705c?, 0x194d160?}, 0xc0152e8000)
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/gocommand/invoke.go:455 +0x390
golang.org/x/tools/internal/gocommand.runCmdContext({0x11824e0, 0xc0152e25a0}, 0xc0152e8000)
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/gocommand/invoke.go:391 +0x3b8
golang.org/x/tools/internal/gocommand.(*Invocation).run(0xc0000475b0, {0x11824e0, 0xc0152e25a0}, {0x1177c20, 0xc0152e26c0}, {0x1177c20, 0xc0152e26f0})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/gocommand/invoke.go:285 +0xc3c
golang.org/x/tools/internal/gocommand.(*Invocation).runWithFriendlyError(0xc000047568?, {0x11824e0, 0xc0152e25a0}, {0x1177c20?, 0xc0152e26c0?}, {0x1177c20, 0xc0152e26f0})
...
golang.org/x/tools/gopls/internal/cache.(*Snapshot).ModTidy.func1({0x1182518?, 0xc014c96af0?}, {0xe3c940?, 0xc0148f70e0?})
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/gopls/internal/cache/mod_tidy.go:80 +0x70
golang.org/x/tools/internal/memoize.(*Promise).run.func2.1()
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/memoize/memoize.go:187 +0xa4
runtime/trace.WithRegion({0x1182518?, 0xc014c96af0?}, {0xc00a6a4bb8, 0x13}, 0xc0108e4f70)
	/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/trace/annotation.go:141 +0xf8
golang.org/x/tools/internal/memoize.(*Promise).run.func2()
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/memoize/memoize.go:180 +0x100
created by golang.org/x/tools/internal/memoize.(*Promise).run in goroutine 13471
	/home/swarming/.swarming/w/ir/x/w/targetrepo1931866900/internal/memoize/memoize.go:179 +0x208

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-01-22 16:28 x_tools-go1.22-linux-ppc64_power8 tools@8bf2b65e release-branch.go1.22@c3c6a500 x/tools/gopls/internal/test/marker.Test/codeaction/fill_struct_resolve.txt [ABORT] (log)
=== RUN   Test/codeaction/fill_struct_resolve.txt
=== PAUSE Test/codeaction/fill_struct_resolve.txt
=== CONT  Test/codeaction/fill_struct_resolve.txt
2025-01-22 17:02 x_tools-go1.22-linux-ppc64_power8 tools@e4adc385 release-branch.go1.22@c3c6a500 x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt
2025-01-22 17:02 x_tools-go1.22-linux-ppc64_power8 tools@e4adc385 release-branch.go1.22@c3c6a500 x/tools/gopls/internal/util/safetoken.TestGoplsSourceDoesNotCallTokenFileMethods [ABORT] (log)
=== RUN   TestGoplsSourceDoesNotCallTokenFileMethods
signal: killed

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-01-30 21:40 x_tools-go1.22-linux-ppc64_power8 tools@db7fffcc release-branch.go1.22@c3c6a500 x/tools/gopls/internal/test/marker.Test/signature/generic.txt [ABORT] (log)
=== RUN   Test/signature/generic.txt
=== PAUSE Test/signature/generic.txt
=== CONT  Test/signature/generic.txt
2025-01-30 21:40 x_tools-go1.22-linux-ppc64_power8 tools@db7fffcc release-branch.go1.22@c3c6a500 x/tools/gopls/internal/test/marker.Test/signature/issue63804.txt [ABORT] (log)
=== RUN   Test/signature/issue63804.txt
=== PAUSE Test/signature/issue63804.txt
=== CONT  Test/signature/issue63804.txt
2025-01-30 21:40 x_tools-go1.22-linux-ppc64_power8 tools@db7fffcc release-branch.go1.22@c3c6a500 x/tools/gopls/internal/test/marker.Test/signature/issue69552.txt [ABORT] (log)
=== RUN   Test/signature/issue69552.txt
=== PAUSE Test/signature/issue69552.txt
=== CONT  Test/signature/issue69552.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-01-31 16:04 x_tools-go1.22-linux-ppc64_power8 tools@a721d4ca release-branch.go1.22@c3c6a500 x/tools/gopls/internal/test/marker.Test/foldingrange/bad.txt [ABORT] (log)
=== RUN   Test/foldingrange/bad.txt
=== PAUSE Test/foldingrange/bad.txt
=== CONT  Test/foldingrange/bad.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-01-31 19:16 x_tools-go1.22-linux-ppc64_power8 tools@5ffcf75f release-branch.go1.22@0cc45e7c x/tools/gopls/internal/test/marker.Test/codeaction/addtest.txt [ABORT] (log)
=== RUN   Test/codeaction/addtest.txt
=== PAUSE Test/codeaction/addtest.txt
=== CONT  Test/codeaction/addtest.txt
2025-01-31 19:16 x_tools-go1.22-linux-ppc64_power8 tools@5ffcf75f release-branch.go1.22@0cc45e7c x/tools/gopls/internal/test/marker.Test/completion/testy.txt [ABORT] (log)
=== RUN   Test/completion/testy.txt
=== PAUSE Test/completion/testy.txt
=== CONT  Test/completion/testy.txt
2025-01-31 19:16 x_tools-go1.22-linux-ppc64_power8 tools@5ffcf75f release-branch.go1.22@0cc45e7c x/tools/gopls/internal/util/safetoken.TestGoplsSourceDoesNotCallTokenFileMethods [ABORT] (log)
=== RUN   TestGoplsSourceDoesNotCallTokenFileMethods
signal: killed
2025-01-31 19:16 x_tools-go1.22-linux-ppc64_power8 tools@51f179ca release-branch.go1.22@0cc45e7c x/tools/gopls/internal/test/marker.Test/callhierarchy/issue64451.txt [ABORT] (log)
=== RUN   Test/callhierarchy/issue64451.txt
=== PAUSE Test/callhierarchy/issue64451.txt

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-02-05 21:57 x_tools-go1.22-linux-ppc64_power8 tools@74b5526d release-branch.go1.22@5817e650 x/tools/gopls/internal/test/marker.Test/codeaction/addtest.txt [ABORT] (log)
=== RUN   Test/codeaction/addtest.txt
=== PAUSE Test/codeaction/addtest.txt
=== CONT  Test/codeaction/addtest.txt
2025-02-05 21:57 x_tools-go1.22-linux-ppc64_power8 tools@74b5526d release-branch.go1.22@5817e650 x/tools/gopls/internal/util/safetoken.TestGoplsSourceDoesNotCallTokenFileMethods [ABORT] (log)
=== RUN   TestGoplsSourceDoesNotCallTokenFileMethods
signal: killed

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-02-06 20:19 x_tools-go1.22-linux-ppc64_power8 tools@0fd02ca5 release-branch.go1.22@5817e650 x/tools/gopls/internal/test/marker.Test/codeaction/addtest.txt [ABORT] (log)
=== RUN   Test/codeaction/addtest.txt
=== PAUSE Test/codeaction/addtest.txt
=== CONT  Test/codeaction/addtest.txt
2025-02-06 20:19 x_tools-go1.22-linux-ppc64_power8 tools@0fd02ca5 release-branch.go1.22@5817e650 x/tools/gopls/internal/util/safetoken.TestGoplsSourceDoesNotCallTokenFileMethods [ABORT] (log)
=== RUN   TestGoplsSourceDoesNotCallTokenFileMethods
signal: killed

watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `(gotip|go1\.\d\d)-linux-ppc64` && (`test timed out` || `SIGQUIT` || `context deadline exceeded` || `running too slowly` || status == "ABORT") && test != "TestServerReadAfterWriteHeader100Continue/h1"
2025-02-07 21:17 x_tools-go1.22-linux-ppc64_power8 tools@a886a1c2 release-branch.go1.22@5817e650 x/tools/gopls/internal/test/marker.Test/codeaction/addtest.txt [ABORT] (log)
=== RUN   Test/codeaction/addtest.txt
=== PAUSE Test/codeaction/addtest.txt
=== CONT  Test/codeaction/addtest.txt
2025-02-07 21:17 x_tools-go1.22-linux-ppc64_power8 tools@a886a1c2 release-branch.go1.22@5817e650 x/tools/gopls/internal/util/safetoken.TestGoplsSourceDoesNotCallTokenFileMethods [ABORT] (log)
=== RUN   TestGoplsSourceDoesNotCallTokenFileMethods
signal: killed

watchflakes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-ppc64x Builders x/build issues (builders, bots, dashboards) NeedsFix The path to resolution is known, but the work has not been done.
Projects
Archived in project
Status: Active
Development

No branches or pull requests

6 participants