Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/coverage: TestCoverageApis failures #56197

Open
gopherbot opened this issue Oct 13, 2022 · 16 comments
Open

runtime/coverage: TestCoverageApis failures #56197

gopherbot opened this issue Oct 13, 2022 · 16 comments
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@gopherbot
Copy link

#!watchflakes
post <- pkg == "runtime/coverage" && test == "TestCoverageApis"

Issue created automatically to collect these failures.

Example (log):

--- FAIL: TestCoverageApis (225.26s)
    emitdata_test.go:44: harness path is /home/gopher/build/tmp/TestCoverageApis3430482397/001/build/harness.exe
    --- FAIL: TestCoverageApis/emitWithCounterClear (161.51s)
        emitdata_test.go:146: running: /home/gopher/build/tmp/TestCoverageApis3430482397/001/build/harness.exe -tp emitWithCounterClear -o /home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear1-edir-y with rdir=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear1-rdir-y and GOCOVERDIR=false
        emitdata_test.go:146: running: /home/gopher/build/tmp/TestCoverageApis3430482397/001/build2/harness.exe -tp emitWithCounterClear -o /home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-edir-y with rdir=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-rdir-y and GOCOVERDIR=false
        emitdata_test.go:158: running: go [tool covdata debugdump -live -pkg=main -i=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-edir-y]
        emitdata_test.go:376: warning: GOCOVERDIR not set, no coverage data emitted
            I run last.
        emitdata_test.go:377: coverage data from "emitWithCounterClear" output match failed: error: output should contain "Func: emitWithCounterClear" but does not
        emitdata_test.go:146: running: /home/gopher/build/tmp/TestCoverageApis3430482397/001/build/harness.exe -tp emitWithCounterClear -o /home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear1-edir-x with rdir=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear1-rdir-x and GOCOVERDIR=true
        emitdata_test.go:146: running: /home/gopher/build/tmp/TestCoverageApis3430482397/001/build2/harness.exe -tp emitWithCounterClear -o /home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-edir-x with rdir=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-rdir-x and GOCOVERDIR=true
        emitdata_test.go:158: running: go [tool covdata debugdump -live -pkg=main -i=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-edir-x]

watchflakes

@gopherbot gopherbot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 13, 2022
@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime/coverage" && test == "TestCoverageApis"
2022-10-06 05:21 openbsd-arm64-jsing go@af668c68 runtime/coverage.TestCoverageApis (log)
--- FAIL: TestCoverageApis (225.26s)
    emitdata_test.go:44: harness path is /home/gopher/build/tmp/TestCoverageApis3430482397/001/build/harness.exe
    --- FAIL: TestCoverageApis/emitWithCounterClear (161.51s)
        emitdata_test.go:146: running: /home/gopher/build/tmp/TestCoverageApis3430482397/001/build/harness.exe -tp emitWithCounterClear -o /home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear1-edir-y with rdir=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear1-rdir-y and GOCOVERDIR=false
        emitdata_test.go:146: running: /home/gopher/build/tmp/TestCoverageApis3430482397/001/build2/harness.exe -tp emitWithCounterClear -o /home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-edir-y with rdir=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-rdir-y and GOCOVERDIR=false
        emitdata_test.go:158: running: go [tool covdata debugdump -live -pkg=main -i=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-edir-y]
        emitdata_test.go:376: warning: GOCOVERDIR not set, no coverage data emitted
            I run last.
        emitdata_test.go:377: coverage data from "emitWithCounterClear" output match failed: error: output should contain "Func: emitWithCounterClear" but does not
        emitdata_test.go:146: running: /home/gopher/build/tmp/TestCoverageApis3430482397/001/build/harness.exe -tp emitWithCounterClear -o /home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear1-edir-x with rdir=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear1-rdir-x and GOCOVERDIR=true
        emitdata_test.go:146: running: /home/gopher/build/tmp/TestCoverageApis3430482397/001/build2/harness.exe -tp emitWithCounterClear -o /home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-edir-x with rdir=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-rdir-x and GOCOVERDIR=true
        emitdata_test.go:158: running: go [tool covdata debugdump -live -pkg=main -i=/home/gopher/build/tmp/TestCoverageApis3430482397/001/emitWithCounterClear2-edir-x]

watchflakes

@bcmills bcmills added this to the Go1.20 milestone Oct 13, 2022
@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime/coverage" && test == "TestCoverageApis"
2022-10-14 12:48 netbsd-amd64-9_0 go@a4b4717f runtime/coverage.TestCoverageApis (log)
panic: test timed out after 3m0s

syscall.Syscall6(0x1e1, 0x1, 0x3e96, 0xc00007bc14, 0x10020, 0x0, 0x0)
	/tmp/workdir/go/src/syscall/asm_unix_amd64.s:39 +0x5 fp=0xc00007bbc0 sp=0xc00007bbb8 pc=0x471f65
os.wait6(0x46e194?, 0xc00006a420?, 0xc00006a3c0?)
	/tmp/workdir/go/src/os/wait6_netbsd.go:16 +0x4d fp=0xc00007bc30 sp=0xc00007bbc0 pc=0x48d48d
os.(*Process).blockUntilWaitable(0xc00001c240)
	/tmp/workdir/go/src/os/wait_wait6.go:20 +0x36 fp=0xc00007bc60 sp=0xc00007bc30 pc=0x48d516
os.(*Process).wait(0xc00001c240)
	/tmp/workdir/go/src/os/exec_unix.go:22 +0x28 fp=0xc00007bcc0 sp=0xc00007bc60 pc=0x488588
os.(*Process).Wait(...)
	/tmp/workdir/go/src/os/exec.go:132
os/exec.(*Cmd).Wait(0xc00012c000)
	/tmp/workdir/go/src/os/exec/exec.go:721 +0x4b fp=0xc00007bd30 sp=0xc00007bcc0 pc=0x4d874b
os/exec.(*Cmd).Run(0xc00001a108?)
	/tmp/workdir/go/src/os/exec/exec.go:500 +0x39 fp=0xc00007bd50 sp=0xc00007bd30 pc=0x4d7659
os/exec.(*Cmd).CombinedOutput(0xc00012c000)
	/tmp/workdir/go/src/os/exec/exec.go:788 +0xa8 fp=0xc00007bd78 sp=0xc00007bd50 pc=0x4d8a68
runtime/coverage.buildHarness(0xc000007860, {0xc0000200c0?, 0x34?}, {0xc00007bf20, 0x2, 0x552?})
	/tmp/workdir/go/src/runtime/coverage/emitdata_test.go:109 +0x29e fp=0xc00007be90 sp=0xc00007bd78 pc=0x51d9be
runtime/coverage.TestCoverageApis(0xc000007860)
	/tmp/workdir/go/src/runtime/coverage/emitdata_test.go:42 +0x1d8 fp=0xc00007bf70 sp=0xc00007be90 pc=0x51cc98
testing.tRunner(0xc000007860, 0x566298)

watchflakes

@thanm
Copy link
Contributor

thanm commented Oct 17, 2022

For this last Oct 14 flake, it looks as though it timed out just building the harness, which is not especially far into the test (hmm). I'll send another CL to skip all the API tests in short mode.

@gopherbot
Copy link
Author

Change https://go.dev/cl/443376 mentions this issue: runtime/coverage: skip more tests in short mode

gopherbot pushed a commit that referenced this issue Oct 17, 2022
Add more skips if short mode testing, since some of these tests
still seem to be timing out on smaller and more underpowered
builders.

Updates #56197.

Change-Id: I469d9fd3a6be5602243234562fa3fe6263968b56
Reviewed-on: https://go-review.googlesource.com/c/go/+/443376
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime/coverage" && test == "TestCoverageApis"
2022-10-17 15:26 linux-ppc64-sid-buildlet go@c83d4fba runtime/coverage.TestCoverageApis (log)
--- FAIL: TestCoverageApis (26.08s)
    emitdata_test.go:44: harness path is /workdir/tmp/TestCoverageApis914856079/001/build/harness.exe
    --- FAIL: TestCoverageApis/emitToFailingWriter (0.05s)
        emitdata_test.go:146: running: /workdir/tmp/TestCoverageApis914856079/001/build/harness.exe -tp emitToFailingWriter -o /workdir/tmp/TestCoverageApis914856079/001/emitToFailingWriter-edir-y with rdir=/workdir/tmp/TestCoverageApis914856079/001/emitToFailingWriter-rdir-y and GOCOVERDIR=false
        emitdata_test.go:146: running: /workdir/tmp/TestCoverageApis914856079/001/build/harness.exe -tp emitToFailingWriter -o /workdir/tmp/TestCoverageApis914856079/001/emitToFailingWriter-edir-x with rdir=/workdir/tmp/TestCoverageApis914856079/001/emitToFailingWriter-rdir-x and GOCOVERDIR=true
        emitdata_test.go:320: panic: runtime error: slice bounds out of range [:4294968768] with capacity 24182

            goroutine 1 [running]:
            panic({0x127860, 0xc000014168})
            	/workdir/go/src/runtime/panic.go:987 +0x564 fp=0xc00008d988 sp=0xc00008d8b8 pc=0x5f644
...
            main.writeStressTest({0x13133c, 0xc}, 0x139f60)
            	/workdir/go/src/runtime/coverage/testdata/harness.go:170 +0xa0 fp=0xc00008deb8 sp=0xc00008de30 pc=0x107e50
            main.emitToFailingWriter()
            	/workdir/go/src/runtime/coverage/testdata/harness.go:206 +0x9c fp=0xc00008def0 sp=0xc00008deb8 pc=0x10802c
            main.main()
            	/workdir/go/src/runtime/coverage/testdata/harness.go:252 +0x230 fp=0xc00008df50 sp=0xc00008def0 pc=0x108470
            runtime.main()
            	/workdir/go/src/runtime/proc.go:250 +0x314 fp=0xc00008dfc0 sp=0xc00008df50 pc=0x63374
            runtime.goexit()
            	/workdir/go/src/runtime/asm_ppc64x.s:899 +0x4 fp=0xc00008dfc0 sp=0xc00008dfc0 pc=0xa5444

watchflakes

romaindoumenc pushed a commit to TroutSoftware/go that referenced this issue Nov 3, 2022
Add more skips if short mode testing, since some of these tests
still seem to be timing out on smaller and more underpowered
builders.

Updates golang#56197.

Change-Id: I469d9fd3a6be5602243234562fa3fe6263968b56
Reviewed-on: https://go-review.googlesource.com/c/go/+/443376
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
@bcmills
Copy link
Member

bcmills commented Jan 25, 2023

@thanm, is this fixed, or is there more to do here? That last failure on 2022-10-17 doesn't particularly look like a timeout to me...

@thanm
Copy link
Contributor

thanm commented Jan 25, 2023

I agree that last failure does not look like a timeout. I will see if I can reproduce it.

@thanm
Copy link
Contributor

thanm commented Jan 25, 2023

Well, so far no luck reproducing it:

gomote run thanm-linux-ppc64-sid-buildlet-0 go/bin/go test -test.count=1000 -test.run=TestCoverageApis/emitToFailingWriter runtime/coverage
...

Looking at the code that triggers the panic, it looks like a corrupt coverage counter section would do it, assuming that is what happened. I will wait and see if any other similar flakes turn up.

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 25, 2023
@thanm
Copy link
Contributor

thanm commented Jan 25, 2023

OK looks like I just needed to be patient and wait a bit longer (I upped the -test.count to 10000 and eventually hit it), albeit with slightly different slice index violation. This should be interesting to work on :-/.

@thanm thanm removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 25, 2023
@thanm
Copy link
Contributor

thanm commented Jan 26, 2023

As near as I can tell this looks to be a relaxed memory model problem, in which two stores to the coverage counter array are being performed out of program order, and this winds up being exposed to the code that is trying to read the counter array while the program is running.

In the 3 or 4 instances of the bug that I've seen, the "number of counters" slot is always very large, e.g. 4294969076, etc. This value looks suspiciously like one of the hard-coded package IDs used by runtime packages. What this says to me is that in the bad scenario we have two threads, A and B. Thread A is walking through the counter array reading values, looking for a non-zero "number of counters" slot. Thread B happens to be executing a function in the runtime that up until this point hasn't been executed, and it's in the prolog section of the function where it is writing the summary values for the function, e.g. numcounters + pkgid + funcid. The program issues three stores for these values, but the processor for whatever reason decides to reorder the stores and it finished the pkgid store first, before the numcounters store. Since it's a runtime package, this explains the bogus value (since runtime packages use pkg IDs that are encoded as negative numbers). Also, we're seeing runtime packages since harness is built with -coverpkg=all.

For the coverage APIs we already restrict the use of coverage.ClearCounters() to atomic mode, but what this bug says to me is that we need to extend this restriction to the other APIs that deal with counter data.

@gopherbot
Copy link
Author

Change https://go.dev/cl/463695 mentions this issue: runtime/coverage: restrict use of all counter-related APIs to atomic mode

@gopherbot gopherbot modified the milestones: Go1.20, Go1.21 Feb 1, 2023
@gopherbot gopherbot reopened this Mar 9, 2023
@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime/coverage" && test == "TestCoverageApis"
2023-03-08 21:46 linux-arm64-longtest go@4df95d51 runtime/coverage.TestCoverageApis (log)
--- FAIL: TestCoverageApis (5.94s)
    emitdata_test.go:47: harness path is /tmp/workdir/tmp/TestCoverageApis2858779810/001/build/harness.exe
    --- FAIL: TestCoverageApis/emitToNilWriter (0.14s)
        emitdata_test.go:149: running: /tmp/workdir/tmp/TestCoverageApis2858779810/001/build/harness.exe -tp emitToNilWriter -o /tmp/workdir/tmp/TestCoverageApis2858779810/001/emitToNilWriter-edir-y with rdir=/tmp/workdir/tmp/TestCoverageApis2858779810/001/emitToNilWriter-rdir-y and GOCOVERDIR=false
        emitdata_test.go:149: running: /tmp/workdir/tmp/TestCoverageApis2858779810/001/build/harness.exe -tp emitToNilWriter -o /tmp/workdir/tmp/TestCoverageApis2858779810/001/emitToNilWriter-edir-x with rdir=/tmp/workdir/tmp/TestCoverageApis2858779810/001/emitToNilWriter-rdir-x and GOCOVERDIR=true
        emitdata_test.go:312: I run last.
            fatal error: internal error: exit hook invoked panic

            goroutine 1 [running]:
            runtime.throw({0x156de1?, 0x6e980?})
            	/tmp/workdir/go/src/runtime/panic.go:1047 +0x7c fp=0x4000061f40 sp=0x4000061f10 pc=0x6a90c
            runtime.runExitHooks(0x0)
            	/tmp/workdir/go/src/runtime/exithook.go:64 +0x1ac fp=0x4000061f70 sp=0x4000061f40 pc=0x1c4ec
            runtime.main()
            	/tmp/workdir/go/src/runtime/proc.go:272 +0x480 fp=0x4000061fd0 sp=0x4000061f70 pc=0x6eaa0
            runtime.goexit()
            	/tmp/workdir/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x4000061fd0 sp=0x4000061fd0 pc=0xbb764

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime/coverage" && test == "TestCoverageApis"
2023-03-25 10:16 linux-amd64-longtest go@9768f736 runtime/coverage.TestCoverageApis (log)
--- FAIL: TestCoverageApis (12.86s)
    emitdata_test.go:52: atomic harness path is /workdir/tmp/TestCoverageApis2774500986/001/build1/harness.exe
    emitdata_test.go:53: non-atomic harness path is /workdir/tmp/TestCoverageApis2774500986/001/build2/harness.exe
    --- FAIL: TestCoverageApis/emitToFailingWriter (0.08s)
        emitdata_test.go:166: running: /workdir/tmp/TestCoverageApis2774500986/001/build1/harness.exe -tp emitToFailingWriter -o /workdir/tmp/TestCoverageApis2774500986/001/emitToFailingWriter-edir-y with rdir=/workdir/tmp/TestCoverageApis2774500986/001/emitToFailingWriter-rdir-y and GOCOVERDIR=false
        emitdata_test.go:166: running: /workdir/tmp/TestCoverageApis2774500986/001/build1/harness.exe -tp emitToFailingWriter -o /workdir/tmp/TestCoverageApis2774500986/001/emitToFailingWriter-edir-x with rdir=/workdir/tmp/TestCoverageApis2774500986/001/emitToFailingWriter-rdir-x and GOCOVERDIR=true
        emitdata_test.go:343: panic: runtime error: slice bounds out of range [:4294972440] with capacity 24269

            goroutine 1 [running]:
            panic({0x55bb80?, 0xc000512000?})
...
            main.writeStressTest({0x564dcf, 0xc}, 0x56d8d0)
            	/workdir/go/src/runtime/coverage/testdata/harness.go:170 +0x85 fp=0xc00010df08 sp=0xc00010de98 pc=0x542805
            main.emitToFailingWriter()
            	/workdir/go/src/runtime/coverage/testdata/harness.go:206 +0x13a fp=0xc00010df30 sp=0xc00010df08 pc=0x542aba
            main.main()
            	/workdir/go/src/runtime/coverage/testdata/harness.go:252 +0x38f fp=0xc00010df80 sp=0xc00010df30 pc=0x54314f
            runtime.main()
            	/workdir/go/src/runtime/proc.go:261 +0x4e2 fp=0xc00010dfe0 sp=0xc00010df80 pc=0x4705a2
            runtime.goexit()
            	/workdir/go/src/runtime/asm_amd64.s:1647 +0x1 fp=0xc00010dfe8 sp=0xc00010dfe0 pc=0x4cab41

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime/coverage" && test == "TestCoverageApis"
2023-04-12 20:52 windows-amd64-longtest go@12e65dbd runtime/coverage.TestCoverageApis (log)
--- FAIL: TestCoverageApis (20.94s)
    emitdata_test.go:52: atomic harness path is C:\Users\gopher\AppData\Local\Temp\1\TestCoverageApis920098751\001\build1\harness.exe
    emitdata_test.go:53: non-atomic harness path is C:\Users\gopher\AppData\Local\Temp\1\TestCoverageApis920098751\001\build2\harness.exe
    --- FAIL: TestCoverageApis/emitWithCounterClear (0.14s)
        emitdata_test.go:166: running: C:\Users\gopher\AppData\Local\Temp\1\TestCoverageApis920098751\001\build1\harness.exe -tp emitWithCounterClear -o C:\Users\gopher\AppData\Local\Temp\1\TestCoverageApis920098751\001\emitWithCounterClear-edir-y with rdir=C:\Users\gopher\AppData\Local\Temp\1\TestCoverageApis920098751\001\emitWithCounterClear-rdir-y and GOCOVERDIR=false
        emitdata_test.go:178: running: go [tool covdata debugdump -live -pkg=command-line-arguments -i=C:\Users\gopher\AppData\Local\Temp\1\TestCoverageApis920098751\001\emitWithCounterClear-edir-y]
        emitdata_test.go:166: running: C:\Users\gopher\AppData\Local\Temp\1\TestCoverageApis920098751\001\build1\harness.exe -tp emitWithCounterClear -o C:\Users\gopher\AppData\Local\Temp\1\TestCoverageApis920098751\001\emitWithCounterClear-edir-x with rdir=C:\Users\gopher\AppData\Local\Temp\1\TestCoverageApis920098751\001\emitWithCounterClear-rdir-x and GOCOVERDIR=true
        emitdata_test.go:358: I run last.
            fatal error: internal error: exit hook invoked panic

            goroutine 1 [running]:
            runtime.throw({0x11b2d39?, 0xc0000c2018?})
            	C:/workdir/go/src/runtime/panic.go:1076 +0xa6 fp=0xc0000c1f50 sp=0xc0000c1f20 pc=0x109f226
            runtime.runExitHooks(0x0)
            	C:/workdir/go/src/runtime/exithook.go:64 +0x1e5 fp=0xc0000c1f80 sp=0xc0000c1f50 pc=0x103e0e5
            runtime.main()
            	C:/workdir/go/src/runtime/proc.go:283 +0x569 fp=0xc0000c1fe0 sp=0xc0000c1f80 pc=0x10a3fc9
            runtime.goexit()
            	C:/workdir/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000c1fe8 sp=0xc0000c1fe0 pc=0x10fa9e1

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime/coverage" && test == "TestCoverageApis"
2023-04-21 21:05 darwin-amd64-longtest go@cedf5008 runtime/coverage.TestCoverageApis (log)
--- FAIL: TestCoverageApis (64.56s)
    emitdata_test.go:52: atomic harness path is /tmp/buildlet/tmp/TestCoverageApis1586029160/001/build1/harness.exe
    emitdata_test.go:53: non-atomic harness path is /tmp/buildlet/tmp/TestCoverageApis1586029160/001/build2/harness.exe
    --- FAIL: TestCoverageApis/emitToWriter (0.88s)
        emitdata_test.go:166: running: /tmp/buildlet/tmp/TestCoverageApis1586029160/001/build1/harness.exe -tp emitToWriter -o /tmp/buildlet/tmp/TestCoverageApis1586029160/001/emitToWriter-edir-y with rdir=/tmp/buildlet/tmp/TestCoverageApis1586029160/001/emitToWriter-rdir-y and GOCOVERDIR=false
        emitdata_test.go:178: running: go [tool covdata debugdump -live -pkg=command-line-arguments -i=/tmp/buildlet/tmp/TestCoverageApis1586029160/001/emitToWriter-edir-y]
        emitdata_test.go:166: running: /tmp/buildlet/tmp/TestCoverageApis1586029160/001/build1/harness.exe -tp emitToWriter -o /tmp/buildlet/tmp/TestCoverageApis1586029160/001/emitToWriter-edir-x with rdir=/tmp/buildlet/tmp/TestCoverageApis1586029160/001/emitToWriter-rdir-x and GOCOVERDIR=true
        emitdata_test.go:178: running: go [tool covdata debugdump -live -pkg=command-line-arguments -i=/tmp/buildlet/tmp/TestCoverageApis1586029160/001/emitToWriter-edir-x]
        emitdata_test.go:182: 'go tool covdata failed (exit status 1): /* WARNING: the format of this dump is not stable and is
             * expected to change from one Go release to the next.
             *
             * produced by:
             *	/tmp/buildlet/go/pkg/tool/darwin_amd64/covdata debugdump -live -pkg=command-line-arguments -i=/tmp/buildlet/tmp/TestCoverageApis1586029160/001/emitToWriter-edir-x
             */
            Cover mode: atomic
            Cover granularity: perblock
            data file /tmp/buildlet/tmp/TestCoverageApis1586029160/001/emitToWriter-edir-x/covcounters.0abcdef.99.77 GOOS=darwin GOARCH=amd64  program args: [/tmp/buildlet/tmp/TestCoverageApis1586029160/001/build1/harness.exe -tp emitToWriter -o /tmp/buildlet/tmp/TestCoverageApis1586029160/001/emitToWriter-edir-x]

            warning: func payload inconsistency: id [p=0,f=1] nf=0 len(ctrs)=1751 in VisitFuncCounterData, ignored
            error: merging counters: len(dst)=13 len(src)=7

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime/coverage" && test == "TestCoverageApis"
2023-07-10 19:19 darwin-amd64-longtest go@07ede7a5 runtime/coverage.TestCoverageApis (log)
--- FAIL: TestCoverageApis (37.85s)
    emitdata_test.go:52: atomic harness path is /tmp/buildlet/tmp/TestCoverageApis2135884621/001/build1/harness.exe
    emitdata_test.go:53: non-atomic harness path is /tmp/buildlet/tmp/TestCoverageApis2135884621/001/build2/harness.exe
    --- FAIL: TestCoverageApis/emitToNilWriter (0.38s)
        emitdata_test.go:166: running: /tmp/buildlet/tmp/TestCoverageApis2135884621/001/build1/harness.exe -tp emitToNilWriter -o /tmp/buildlet/tmp/TestCoverageApis2135884621/001/emitToNilWriter-edir-y with rdir=/tmp/buildlet/tmp/TestCoverageApis2135884621/001/emitToNilWriter-rdir-y and GOCOVERDIR=false
        emitdata_test.go:166: running: /tmp/buildlet/tmp/TestCoverageApis2135884621/001/build1/harness.exe -tp emitToNilWriter -o /tmp/buildlet/tmp/TestCoverageApis2135884621/001/emitToNilWriter-edir-x with rdir=/tmp/buildlet/tmp/TestCoverageApis2135884621/001/emitToNilWriter-rdir-x and GOCOVERDIR=true
        emitdata_test.go:329: I run last.
            fatal error: internal error: exit hook invoked panic

            goroutine 1 [running]:
            runtime.throw({0x117a958?, 0x127e438?})
            	/tmp/buildlet/go/src/runtime/panic.go:1077 +0xa5 fp=0xc00011df10 sp=0xc00011dee0 pc=0x106f285
            runtime.runExitHooks(0x0)
            	/tmp/buildlet/go/src/runtime/exithook.go:64 +0x1e5 fp=0xc00011df40 sp=0xc00011df10 pc=0x100f005
            runtime.main()
            	/tmp/buildlet/go/src/runtime/proc.go:289 +0x745 fp=0xc00011dfe0 sp=0xc00011df40 pc=0x10748a5
            runtime.goexit()
            	/tmp/buildlet/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00011dfe8 sp=0xc00011dfe0 pc=0x10d20c1

watchflakes

@gopherbot gopherbot removed this from the Go1.21 milestone Aug 8, 2023
@gopherbot gopherbot added this to the Go1.22 milestone Aug 8, 2023
@gopherbot gopherbot modified the milestones: Go1.22, Go1.23 Feb 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Status: In Progress
Status: Done
Development

No branches or pull requests

3 participants