Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: TestGdbBacktrace failures #58698

Closed
gopherbot opened this issue Feb 24, 2023 · 14 comments
Closed

runtime: TestGdbBacktrace failures #58698

gopherbot opened this issue Feb 24, 2023 · 14 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@gopherbot
Copy link
Contributor

gopherbot commented Feb 24, 2023

#!watchflakes
post <- pkg == "runtime" && test == "TestGdbBacktrace" && (`No child processes` || `No such process`)

Issue created automatically to collect these failures.

Example (log):

--- FAIL: TestGdbBacktrace (2323.65s)
    runtime-gdb_test.go:83: gdb version 10.1
    runtime-gdb_test.go:468: GDB command timed out after 38m43.017693938s: /usr/bin/gdb -nx -batch -iex add-auto-load-safe-path /workdir/go/src/runtime -ex set startup-with-shell off -ex break main.eee -ex run -ex backtrace -ex continue /workdir/tmp/TestGdbBacktrace2238907183/001/a.exe
    runtime-gdb_test.go:473: gdb output:
        Loading Go Runtime support.
        Breakpoint 1 at 0x80a2830: file /workdir/tmp/TestGdbBacktrace2238907183/001/main.go, line 17.
        [New LWP 532860]
        [New LWP 532862]
        [New LWP 532864]
        [New LWP 532865]

        Thread 1 "a.exe" hit Breakpoint 1, main.eee (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:17
        17	func eee() bool { return true }
        #0  main.eee (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:17
        #1  0x080a280f in main.ddd (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:14
        #2  0x080a27da in main.ccc (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:11
        #3  0x080a27aa in main.bbb (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:8
        #4  0x080a277a in main.aaa (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:5
        #5  0x080a286a in main.main () at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:22
        waiting for new child: No child processes.
    runtime-gdb_test.go:488: gdb exited with error: signal: killed

watchflakes

@gopherbot gopherbot added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. compiler/runtime Issues related to the Go compiler and/or runtime. labels Feb 24, 2023
@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime" && test == "TestGdbBacktrace"
2023-02-13 19:42 linux-386-longtest go@0b922bfa runtime.TestGdbBacktrace (log)
--- FAIL: TestGdbBacktrace (2323.65s)
    runtime-gdb_test.go:83: gdb version 10.1
    runtime-gdb_test.go:468: GDB command timed out after 38m43.017693938s: /usr/bin/gdb -nx -batch -iex add-auto-load-safe-path /workdir/go/src/runtime -ex set startup-with-shell off -ex break main.eee -ex run -ex backtrace -ex continue /workdir/tmp/TestGdbBacktrace2238907183/001/a.exe
    runtime-gdb_test.go:473: gdb output:
        Loading Go Runtime support.
        Breakpoint 1 at 0x80a2830: file /workdir/tmp/TestGdbBacktrace2238907183/001/main.go, line 17.
        [New LWP 532860]
        [New LWP 532862]
        [New LWP 532864]
        [New LWP 532865]

        Thread 1 "a.exe" hit Breakpoint 1, main.eee (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:17
        17	func eee() bool { return true }
        #0  main.eee (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:17
        #1  0x080a280f in main.ddd (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:14
        #2  0x080a27da in main.ccc (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:11
        #3  0x080a27aa in main.bbb (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:8
        #4  0x080a277a in main.aaa (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:5
        #5  0x080a286a in main.main () at /workdir/tmp/TestGdbBacktrace2238907183/001/main.go:22
        waiting for new child: No child processes.
    runtime-gdb_test.go:488: gdb exited with error: signal: killed

watchflakes

@mknyszek mknyszek added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Mar 1, 2023
@mknyszek mknyszek added this to the Backlog milestone Mar 1, 2023
@mknyszek
Copy link
Contributor

mknyszek commented Mar 1, 2023

In triage, best guess is that this is just a "GDB tests are sometimes flaky" but we'll wait on more failures.

Than and Cherry note that the GDB version isn't very old.

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime" && test == "TestGdbBacktrace"
2023-03-16 19:44 linux-arm64 go@19281498 runtime.TestGdbBacktrace (log)
--- FAIL: TestGdbBacktrace (0.33s)
    runtime-gdb_test.go:81: gdb version 10.1
    runtime-gdb_test.go:471: gdb output:
        Loading Go Runtime support.
        Breakpoint 1 at 0x64e84: file /tmp/workdir/tmp/TestGdbBacktrace2506571992/001/main.go, line 17.
        [New LWP 180945]
        [New LWP 180946]
        [New LWP 180947]

        Thread 1 "a.exe" hit Breakpoint 1, 0x0000000000064e84 in main.eee (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace2506571992/001/main.go:17
        17	func eee() bool { return true }
        #0  0x0000000000064e84 in main.eee (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace2506571992/001/main.go:17
        #1  0x0000000000064e68 in main.ddd (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace2506571992/001/main.go:14
        #2  0x0000000000064e1c in main.ccc (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace2506571992/001/main.go:11
        #3  0x0000000000064ddc in main.bbb (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace2506571992/001/main.go:8
        #4  0x0000000000064d9c in main.aaa (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace2506571992/001/main.go:5
        #5  0x0000000000064eac in main.main () at /tmp/workdir/tmp/TestGdbBacktrace2506571992/001/main.go:22
        Unable to fetch general registers.: No such process.
        [LWP 180947 exited]
        [LWP 180945 exited]
        [LWP 180940 exited]
    runtime-gdb_test.go:486: gdb exited with error: exit status 1

watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime" && test == "TestGdbBacktrace"
2023-03-16 19:51 linux-arm64-boringcrypto go@e91876f4 runtime.TestGdbBacktrace (log)
--- FAIL: TestGdbBacktrace (0.28s)
    runtime-gdb_test.go:81: gdb version 10.1
    runtime-gdb_test.go:471: gdb output:
        Loading Go Runtime support.
        Breakpoint 1 at 0x64e84: file /tmp/workdir/tmp/TestGdbBacktrace4270477819/001/main.go, line 17.
        [New LWP 188825]
        [New LWP 188826]
        [New LWP 188827]

        Thread 1 "a.exe" hit Breakpoint 1, 0x0000000000064e84 in main.eee (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace4270477819/001/main.go:17
        17	func eee() bool { return true }
        #0  0x0000000000064e84 in main.eee (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace4270477819/001/main.go:17
        #1  0x0000000000064e68 in main.ddd (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace4270477819/001/main.go:14
        #2  0x0000000000064e1c in main.ccc (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace4270477819/001/main.go:11
        #3  0x0000000000064ddc in main.bbb (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace4270477819/001/main.go:8
        #4  0x0000000000064d9c in main.aaa (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace4270477819/001/main.go:5
        #5  0x0000000000064eac in main.main () at /tmp/workdir/tmp/TestGdbBacktrace4270477819/001/main.go:22
        Unable to fetch general registers.: No such process.
        [LWP 188827 exited]
        [LWP 188825 exited]
        [LWP 188822 exited]
    runtime-gdb_test.go:486: gdb exited with error: exit status 1

watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime" && test == "TestGdbBacktrace"
2023-03-30 18:50 linux-amd64 go@8890c3a0 runtime.TestGdbBacktrace (log)
--- FAIL: TestGdbBacktrace (148.14s)
    runtime-gdb_test.go:81: gdb version 10.1
    runtime-gdb_test.go:466: GDB command timed out after 2m27.709563668s: /usr/bin/gdb -nx -batch -iex add-auto-load-safe-path /workdir/go/src/runtime -ex set startup-with-shell off -ex break main.eee -ex run -ex backtrace -ex continue /workdir/tmp/TestGdbBacktrace732529650/001/a.exe
    runtime-gdb_test.go:471: gdb output:
        Loading Go Runtime support.
        Breakpoint 1 at 0x455180: file /workdir/tmp/TestGdbBacktrace732529650/001/main.go, line 17.
        [New LWP 69987]
        [New LWP 69988]
        [New LWP 69989]

        Thread 1 "a.exe" hit Breakpoint 1, main.eee (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace732529650/001/main.go:17
        17	func eee() bool { return true }
        #0  main.eee (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace732529650/001/main.go:17
        #1  0x0000000000455176 in main.ddd (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace732529650/001/main.go:14
        #2  0x000000000045514f in main.ccc (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace732529650/001/main.go:11
        #3  0x000000000045512f in main.bbb (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace732529650/001/main.go:8
        #4  0x000000000045510f in main.aaa (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace732529650/001/main.go:5
        #5  0x00000000004551af in main.main () at /workdir/tmp/TestGdbBacktrace732529650/001/main.go:22
        waiting for new child: No child processes.
    runtime-gdb_test.go:486: gdb exited with error: signal: killed

watchflakes

@bcmills
Copy link
Contributor

bcmills commented Apr 6, 2023

@bcmills
Copy link
Contributor

bcmills commented Apr 6, 2023

In triage, best guess is that this is just a "GDB tests are sometimes flaky" but we'll wait on more failures.

Even if the problem is “GDB tests are sometimes flaky”, arguably they ought to skip in case of known failure modes.

(At the very least, there ought to be specific upstream issues we can refer to for the failures — if the problem is upstream and we don't report it, then it presumably won't ever get fixed.)

@bcmills
Copy link
Contributor

bcmills commented Apr 6, 2023

@bcmills bcmills removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 11, 2023
@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime" && test == "TestGdbBacktrace" && (`No child processes` || `No such process`)
2023-05-04 17:01 linux-arm64 go@8337ca09 runtime.TestGdbBacktrace (log)
--- FAIL: TestGdbBacktrace (0.27s)
    runtime-gdb_test.go:81: gdb version 10.1
    runtime-gdb_test.go:471: gdb output:
        Loading Go Runtime support.
        Breakpoint 1 at 0x66464: file /tmp/workdir/tmp/TestGdbBacktrace3646086843/001/main.go, line 17.
        [New LWP 193720]
        [New LWP 193721]
        [New LWP 193722]

        Thread 1 "a.exe" hit Breakpoint 1, 0x0000000000066464 in main.eee (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace3646086843/001/main.go:17
        17	func eee() bool { return true }
        #0  0x0000000000066464 in main.eee (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace3646086843/001/main.go:17
        #1  0x0000000000066448 in main.ddd (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace3646086843/001/main.go:14
        #2  0x00000000000663fc in main.ccc (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace3646086843/001/main.go:11
        #3  0x00000000000663bc in main.bbb (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace3646086843/001/main.go:8
        #4  0x000000000006637c in main.aaa (~r0=<optimized out>) at /tmp/workdir/tmp/TestGdbBacktrace3646086843/001/main.go:5
        #5  0x000000000006648c in main.main () at /tmp/workdir/tmp/TestGdbBacktrace3646086843/001/main.go:22
        Unable to fetch general registers.: No such process.
        [LWP 193722 exited]
        [LWP 193720 exited]
        [LWP 193717 exited]
    runtime-gdb_test.go:486: gdb exited with error: exit status 1

watchflakes

@bcmills
Copy link
Contributor

bcmills commented Jun 1, 2023

@bcmills
Copy link
Contributor

bcmills commented Jun 1, 2023

@golang/runtime, could someone please at least update the skips for this test?

If the problem is “GDB is flaky” we shouldn't be annoying Go developers (and Go users who run go test all as they should!) with ongoing test flakes that aren't actionable.

@gopherbot
Copy link
Contributor Author

Change https://go.dev/cl/499975 mentions this issue: runtime: update skips for TestGdbBacktrace

@gopherbot gopherbot reopened this Jun 5, 2023
@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime" && test == "TestGdbBacktrace" && (`No child processes` || `No such process`)
2023-06-05 16:41 linux-ppc64le-power10osu go@587c1c19 runtime.TestGdbBacktrace (log)
--- FAIL: TestGdbBacktrace (0.66s)
    runtime-gdb_test.go:81: gdb version 12.1
    runtime-gdb_test.go:471: gdb output:
        Loading Go Runtime support.
        Breakpoint 1 at 0x6c964: file /workdir/tmp/TestGdbBacktrace3086457048/001/main.go, line 17.
        [New LWP 48184]
        [New LWP 48185]
        [New LWP 48188]
        [New LWP 48191]

        Thread 1 "a.exe" hit Breakpoint 1, 0x000000000006c964 in main.eee (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace3086457048/001/main.go:17
        17	func eee() bool { return true }
        #0  0x000000000006c964 in main.eee (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace3086457048/001/main.go:17
        #1  0x000000000006c944 in main.ddd (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace3086457048/001/main.go:14
        #2  0x000000000006c8f4 in main.ccc (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace3086457048/001/main.go:11
        #3  0x000000000006c8b4 in main.bbb (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace3086457048/001/main.go:8
        #4  0x000000000006c874 in main.aaa (~r0=<optimized out>) at /workdir/tmp/TestGdbBacktrace3086457048/001/main.go:5
        #5  0x000000000006c994 in main.main () at /workdir/tmp/TestGdbBacktrace3086457048/001/main.go:22
        reading register pc (#64): No such process.
        [LWP 48191 exited]
        [LWP 48185 exited]
        [LWP 48184 exited]
        [LWP 48181 exited]
    runtime-gdb_test.go:488: gdb exited with error: exit status 1

watchflakes

@gopherbot
Copy link
Contributor Author

Change https://go.dev/cl/500959 mentions this issue: runtime: skip another GDB "no such process" case

@golang golang locked and limited conversation to collaborators Jun 4, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Archived in project
Development

No branches or pull requests

3 participants