Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: TestUsingVDSO failures #63734

Closed
gopherbot opened this issue Oct 25, 2023 · 6 comments
Closed

runtime: TestUsingVDSO failures #63734

gopherbot opened this issue Oct 25, 2023 · 6 comments
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@gopherbot
Copy link
Contributor

gopherbot commented Oct 25, 2023

#!watchflakes
post <- pkg == "runtime" && test == "TestUsingVDSO" && `signal: segmentation fault`

Issue created automatically to collect these failures.

Example (log):

--- FAIL: TestUsingVDSO (0.08s)
    vdso_test.go:50: GO_WANT_HELPER_PROCESS=1 /usr/bin/strace -f -e clock_gettime /workdir/tmp/go-build1040845572/b001/runtime.test -test.run=^TestUsingVDSO$
    vdso_test.go:56: /usr/bin/strace: [ Process PID=74855 runs in 32 bit mode. ]
        /usr/bin/strace: Process 74908 attached
        /usr/bin/strace: Process 74906 attached
        /usr/bin/strace: Process 74917 attached
        [pid 74917] ????( <unfinished ...>
        [pid 74908] +++ exited with 0 +++
        [pid 74906] +++ exited with 0 +++
    vdso_test.go:59: signal: segmentation fault (core dumped)
runtime.test: some test left a core file behind

watchflakes

@gopherbot gopherbot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 25, 2023
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Oct 25, 2023
@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "runtime" && test == "TestUsingVDSO"
2023-10-14 22:30 linux-386-clang go@bc9dc8d4 runtime.TestUsingVDSO (log)
--- FAIL: TestUsingVDSO (0.08s)
    vdso_test.go:50: GO_WANT_HELPER_PROCESS=1 /usr/bin/strace -f -e clock_gettime /workdir/tmp/go-build1040845572/b001/runtime.test -test.run=^TestUsingVDSO$
    vdso_test.go:56: /usr/bin/strace: [ Process PID=74855 runs in 32 bit mode. ]
        /usr/bin/strace: Process 74908 attached
        /usr/bin/strace: Process 74906 attached
        /usr/bin/strace: Process 74917 attached
        [pid 74917] ????( <unfinished ...>
        [pid 74908] +++ exited with 0 +++
        [pid 74906] +++ exited with 0 +++
    vdso_test.go:59: signal: segmentation fault (core dumped)
runtime.test: some test left a core file behind

watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime" && test == "TestUsingVDSO" && `signal: segmentation fault`
2024-01-12 00:55 linux-386-buster go@eb30ac34 runtime.TestUsingVDSO (log)
--- FAIL: TestUsingVDSO (0.06s)
    vdso_test.go:50: GO_WANT_HELPER_PROCESS=1 /usr/bin/strace -f -e clock_gettime /workdir/tmp/go-build587421342/b091/runtime.test -test.run=^TestUsingVDSO$
    vdso_test.go:56: /usr/bin/strace: [ Process PID=79301 runs in 32 bit mode. ]
        /usr/bin/strace: Process 79352 attached
        /usr/bin/strace: Process 79390 attached
        /usr/bin/strace: Process 79350 attached
        /usr/bin/strace: Process 79356 attached
        [pid 79350] ????( <unfinished ...>
        [pid 79390] +++ exited with 0 +++
        [pid 79352] +++ exited with 0 +++
    vdso_test.go:59: signal: segmentation fault (core dumped)
runtime.test: some test left a core file behind

watchflakes

@bcmills
Copy link
Contributor

bcmills commented Jan 16, 2024

That's two on linux/386 — which is (for better or for worse) still a first class port — and it appears that this test is new for Go 1.22 (added in https://go.dev/cl/523955 for #62309).

@golang/runtime: given the above, should this be a release-blocker (to either fix or add a skip for the test)?

@aclements
Copy link
Member

This smells like an strace bug. If the subprocess itself were SIGSEGV'ing, I would expect strace to report that signal. If so, then this a test-only failure, and I'd be fine with doing a SkipFlaky if strace exits with a SIGSEGV, or perhaps just retrying the test so we catch persistent failures.

@aclements
Copy link
Member

For example, if the test binary itself were failing, I would expect output like this before strace exited with SIGSEGV:

[pid 644584] --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
+++ killed by SIGSEGV +++

The "killed by SIGSEGV" would only appear if we fail before installing signal handlers, but then I would expect other tests to be failing in the same way. If we're failing after installing signal handlers, we'd still see the first line, and then the process would exit with status 2, not SIGSEGV.

So, it's definitely strace itself that's crashing.

@gopherbot
Copy link
Contributor Author

Change https://go.dev/cl/556357 mentions this issue: runtime: skip test if strace crashes

@github-project-automation github-project-automation bot moved this from Todo to Done in Go Compiler / Runtime Jan 17, 2024
ezz-no pushed a commit to ezz-no/go-ezzno that referenced this issue Feb 18, 2024
Very occasionally, at least on linux/386, strace itself will crash in
TestUsingVDSO. Detect these crashes and just skip the test.

Fixes golang#63734.

Change-Id: I050494459d47dd96c0b8dc0b16353cb532fba93e
Reviewed-on: https://go-review.googlesource.com/c/go/+/556357
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
@dmitshur dmitshur modified the milestones: Backlog, Go1.22 Apr 3, 2024
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done.
Projects
Archived in project
Development

No branches or pull requests

6 participants