Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: TestCgoPprofThread failures with invalid traceback origin #52726

Closed
bcmills opened this issue May 5, 2022 · 3 comments
Closed

runtime: TestCgoPprofThread failures with invalid traceback origin #52726

bcmills opened this issue May 5, 2022 · 3 comments
Labels
NeedsInvestigation OS-Linux release-blocker
Milestone

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented May 5, 2022

--- FAIL: TestCgoPprofThread (0.86s)
    crash_cgo_test.go:345: [/tmp/workdir-host-linux-amd64-wsl/go/bin/go tool pprof -traces /tmp/workdir-host-linux-amd64-wsl/tmp/go-build1132426278/testprogcgo_.exe /tmp/workdir-host-linux-amd64-wsl/tmp/prof944139058]:
        File: testprogcgo_.exe
        Build ID: 261d318cd4ab866e2474b8a08356fc426efe8d5a
        Type: cpu
        Time: May 5, 2022 at 8:35am (CST)
        Duration: 301.74ms, Total samples = 110ms (36.45%)
        -----------+-------------------------------------------------------
              10ms   cpuHogThread
                     cpuHogThread2
                     runtime.cgocall
                     main._Cfunc_runCPUHogThread
                     main.pprofThread
                     main.CgoPprofThread
                     main.main
                     runtime.main
        -----------+-------------------------------------------------------
             100ms   cpuHogThread
                     cpuHogThread2
        -----------+-------------------------------------------------------
    crash_cgo_test.go:357: invalid traceback origin: got=[cpuHogThread cpuHogThread2 runtime.cgocall main._Cfunc_runCPUHogThread main.pprofThread main.CgoPprofThread main.main runtime.main]; want=[cpuHogThread ... cpuHogThread2]
    crash_cgo_test.go:345: [/tmp/workdir-host-linux-amd64-wsl/go/bin/go tool pprof -traces /tmp/workdir-host-linux-amd64-wsl/tmp/prof944139058]:
        File: testprogcgo_.exe
        Build ID: 261d318cd4ab866e2474b8a08356fc426efe8d5a
        Type: cpu
        Time: May 5, 2022 at 8:35am (CST)
        Duration: 301.74ms, Total samples = 110ms (36.45%)
        -----------+-------------------------------------------------------
              10ms   cpuHogThread
                     cpuHogThread2
                     runtime.cgocall
                     main._Cfunc_runCPUHogThread
                     main.pprofThread
                     main.CgoPprofThread
                     main.main
                     runtime.main
        -----------+-------------------------------------------------------
             100ms   cpuHogThread
                     cpuHogThread2
        -----------+-------------------------------------------------------
    crash_cgo_test.go:357: invalid traceback origin: got=[cpuHogThread cpuHogThread2 runtime.cgocall main._Cfunc_runCPUHogThread main.pprofThread main.CgoPprofThread main.main runtime.main]; want=[cpuHogThread ... cpuHogThread2]
FAIL
FAIL	runtime	26.366s

greplogs -l -e 'FAIL: TestCgoPprofThread .*(?:\n[ ]{4}.*)* invalid traceback origin'
2022-05-04T15:38:55-9511f6d/linux-amd64-wsl
2021-12-09T12:28:48-65633fe/linux-amd64-wsl
2021-11-29T00:57:09-78af02e/linux-amd64-longtest
2021-11-16T17:13:33-29ec902/linux-amd64-jessie
2021-11-11T11:00:33-a01a6d6/linux-amd64
2021-10-28T22:38:00-af05d8b/linux-amd64-fedora
2021-10-11T15:28:50-702e337/linux-amd64-wsl
2021-10-09T01:04:29-e1c294a/linux-amd64-fedora
2021-10-06T19:45:35-e38ec96/linux-amd64-longtest
2021-10-05T19:15:03-ae83301/linux-amd64-staticlockranking
2021-09-15T03:29:39-9fc2889/linux-amd64-wsl
2021-09-13T18:02:42-e74e363/linux-amd64-wsl
2021-09-04T19:33:54-28dae3d/linux-amd64-wsl
2021-09-02T14:02:42-b8420ba/linux-amd64-wsl
2021-09-02T01:06:17-df4c625/linux-amd64-wsl
2021-09-01T23:41:53-2872496/linux-amd64-wsl
2021-08-16T14:27:24-b9f135d/linux-amd64-wsl
2021-02-24T16:38:32-ae1fa08/linux-amd64-wsl
2020-12-11T17:26:14-1341a3d/linux-amd64-wsl

(CC @golang/runtime)

@bcmills
Copy link
Member Author

@bcmills bcmills commented May 5, 2022

Marking as release-blocker for Go 1.19 because linux/amd64 is a first class port (and several of these failures are on regular builders rather than the -wsl builder).

This issue at least needs to be triaged for Go 1.19; if the runtime team decides that it isn't a priority, this test (or this failure mode) should call testenv.SkipFlaky to reduce noise from failures.

@bcmills bcmills added NeedsInvestigation release-blocker OS-Linux labels May 5, 2022
@bcmills bcmills added this to the Go1.19 milestone May 5, 2022
@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented May 17, 2022

The cgo traceback function ( https://cs.opensource.google/go/go/+/master:src/runtime/testdata/testprogcgo/threadpprof.go;l=45 ) pretends all C samples are in cpuHogThread. But if a profiling signal lands outside of the C thread, e.g. before/when the thread is created, we will get a sample like above, which looks like Go calls into that C thread. Perhaps the cgo traceback function should only return those PCs when a signal lands on that thread.

@gopherbot
Copy link

@gopherbot gopherbot commented May 17, 2022

Change https://go.dev/cl/406934 mentions this issue: runtime: deflake TestCgoPprofThread

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation OS-Linux release-blocker
Projects
None yet
Development

No branches or pull requests

3 participants