Skip to content

runtime: deadlock while stopping the world (most recently openbsd/amd64) #62541

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
gopherbot opened this issue Sep 8, 2023 · 7 comments
Open

runtime: deadlock while stopping the world (most recently openbsd/amd64) #62541

gopherbot opened this issue Sep 8, 2023 · 7 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-OpenBSD
Milestone
Backlog

Comments

@gopherbot
Copy link
Contributor

gopherbot commented Sep 8, 2023 •
edited by mknyszek
Loading 

#!watchflakes
default <- log ~ `SIGQUIT` && log ~ `stopping the world`

Issue created automatically to collect these failures.

Example (log):

SIGQUIT: quit
PC=0x80b76f7 m=14 sigcode=0

eax    0x4
ebx    0x0
ecx    0x8407b00
edx    0x854aa80
edi    0xb09dfe7c
esi    0xf
ebp    0x3
...
	/tmp/workdir/go/src/runtime/proc.go:1262 +0xc7 fp=0x8438e1c sp=0x8438df4 pc=0x8087577
runtime.stopTheWorldGC(0xa)
	/tmp/workdir/go/src/runtime/proc.go:1310 +0x45 fp=0x8438e34 sp=0x8438e1c pc=0x8087705
runtime.StartTrace()
	/tmp/workdir/go/src/runtime/trace.go:284 +0x1c fp=0x8438e7c sp=0x8438e34 pc=0x80a432c
runtime/trace.Start({0x8237a2c, 0x847e270})
	/tmp/workdir/go/src/runtime/trace/trace.go:125 +0x75 fp=0x8438e98 sp=0x8438e7c pc=0x81105e5
runtime/trace_test.TestTraceFutileWakeup(0x85340f0)
	/tmp/workdir/go/src/runtime/trace/trace_test.go:510 +0xc5 fp=0x8438f98 sp=0x8438e98 pc=0x81b2315
testing.tRunner(0x85340f0, 0x8208078)

— watchflakes
@gopherbot gopherbot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 8, 2023
@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "runtime/trace" && test == "TestTraceFutileWakeup"
2023-09-08 18:57 netbsd-386-9_3 go@729e21b2 runtime/trace.TestTraceFutileWakeup (log) 
SIGQUIT: quit
PC=0x80b76f7 m=14 sigcode=0

eax    0x4
ebx    0x0
ecx    0x8407b00
edx    0x854aa80
edi    0xb09dfe7c
esi    0xf
ebp    0x3
...
	/tmp/workdir/go/src/runtime/proc.go:1262 +0xc7 fp=0x8438e1c sp=0x8438df4 pc=0x8087577
runtime.stopTheWorldGC(0xa)
	/tmp/workdir/go/src/runtime/proc.go:1310 +0x45 fp=0x8438e34 sp=0x8438e1c pc=0x8087705
runtime.StartTrace()
	/tmp/workdir/go/src/runtime/trace.go:284 +0x1c fp=0x8438e7c sp=0x8438e34 pc=0x80a432c
runtime/trace.Start({0x8237a2c, 0x847e270})
	/tmp/workdir/go/src/runtime/trace/trace.go:125 +0x75 fp=0x8438e98 sp=0x8438e7c pc=0x81105e5
runtime/trace_test.TestTraceFutileWakeup(0x85340f0)
	/tmp/workdir/go/src/runtime/trace/trace_test.go:510 +0xc5 fp=0x8438f98 sp=0x8438e98 pc=0x81b2315
testing.tRunner(0x85340f0, 0x8208078)

— watchflakes

@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Sep 8, 2023
@mknyszek
Copy link
Contributor

This doesn't look like a failure in TestTraceFutileWakeup, but rather a deadlock (or something) when trying to stop-the-world. It's also on NetBSD.

@mknyszek mknyszek changed the title runtime/trace: TestTraceFutileWakeup failures runtime: deadlock while stopping the world (currently netbsd/386) Sep 13, 2023
@mknyszek mknyszek added this to the Backlog milestone Sep 13, 2023
@mknyszek
Copy link
Contributor

I think let's wait for more failures here. If it's a real issue on more platforms I want to track that. If it only happens on NetBSD, then I'll mark it as such.

@mknyszek
Copy link
Contributor

Oh, this may be related to #62524 given that it's a NetBSD failure. Once we have a resolution there, let's close this optimistically.

@mknyszek mknyszek added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed help wanted labels Sep 13, 2023
@mknyszek
Copy link
Contributor

Oh, that does have a resolution. Closing optimistically.

@github-project-automation github-project-automation bot moved this from Todo to Done in Go Compiler / Runtime Sep 13, 2023
@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
default <- log ~ `SIGQUIT` && log ~ `stopping the world`
2024-03-30 00:20 gotip-openbsd-amd64 go@ba9c445f cmd/go/internal/vcweb/vcstest.TestScripts (log) 
=== RUN   TestScripts
    vcstest_test.go:120: GET http://127.0.0.1:39717:
        <html>
        <title>vcweb</title>
        <pre>
        <b>vcweb</b>
        
        This server serves various version control repos for testing the go command.
        
        For an overview of the script language, see <a href="/help">/help</a>.
...
        hg/custom-hg-hello.txt		2024-03-29T17:38:11-07:00	custom-hg-hello.txt:1: handle hg: server for `hg` VCS is not installed
        hg/hello.txt			2024-03-29T17:38:11-07:00	hello.txt:1: handle hg: server for `hg` VCS is not installed
        hg/hgrepo1.txt			2024-03-29T17:38:11-07:00	hgrepo1.txt:1: handle hg: server for `hg` VCS is not installed
        hg/vgotest1.txt			2024-03-29T17:38:11-07:00	vgotest1.txt:1: handle hg: server for `hg` VCS is not installed
        insecure.txt			2024-03-29T17:38:11-07:00	ok
        svn/hello.txt			2024-03-29T17:38:15-07:00	hello.txt:1: handle svn: server for `svn` VCS is not installed
        svn/nonexistent.txt		2024-03-29T17:38:11-07:00	nonexistent.txt:1: handle svn: server for `svn` VCS is not installed
        svn/test1-svn-git.txt		2024-03-29T17:38:11-07:00	test1-svn-git.txt:1: handle svn: server for `svn` VCS is not installed
        svn/test2-svn-git.txt		2024-03-29T17:38:11-07:00	test2-svn-git.txt:1: handle svn: server for `svn` VCS is not installed
--- FAIL: TestScripts (6.91s)
2024-03-30 00:20 gotip-openbsd-amd64 go@ba9c445f cmd/internal/moddeps.TestAllDependencies/cmd(quick) (log) 
=== RUN   TestAllDependencies/cmd(quick)
    moddeps_test.go:53: module cmd in directory /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd
    moddeps_test.go:65: /home/swarming/.swarming/w/ir/x/w/goroot/bin/go list -mod=vendor -deps ./...: exit status 1
        runtime: failed to create new OS thread
    moddeps_test.go:66: (Run 'go mod vendor' in /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd to ensure that dependencies have been vendored.)
--- FAIL: TestAllDependencies/cmd(quick) (6.18s)
2024-03-30 00:20 gotip-openbsd-amd64 go@ba9c445f cmd/link.TestMinusRSymsWithSameName (log) 
=== RUN   TestMinusRSymsWithSameName
=== PAUSE TestMinusRSymsWithSameName
=== CONT  TestMinusRSymsWithSameName
    elf_test.go:168: clang [-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/home/swarming/.swarming/w/ir/x/t/go-build1878212184=/tmp/go-build -gno-record-gcc-switches] -c -o /home/swarming/.swarming/w/ir/x/t/TestMinusRSymsWithSameName1043036679/001/x0.o /home/swarming/.swarming/w/ir/x/t/TestMinusRSymsWithSameName1043036679/001/x0.c
    elf_test.go:168: clang [-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/home/swarming/.swarming/w/ir/x/t/go-build1878212184=/tmp/go-build -gno-record-gcc-switches] -c -o /home/swarming/.swarming/w/ir/x/t/TestMinusRSymsWithSameName1043036679/001/x1.o /home/swarming/.swarming/w/ir/x/t/TestMinusRSymsWithSameName1043036679/001/x1.c
    elf_test.go:176: clang [-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/home/swarming/.swarming/w/ir/x/t/go-build1878212184=/tmp/go-build -gno-record-gcc-switches] -nostdlib -r -o /home/swarming/.swarming/w/ir/x/t/TestMinusRSymsWithSameName1043036679/001/ldr.syso [/home/swarming/.swarming/w/ir/x/t/TestMinusRSymsWithSameName1043036679/001/x0.o /home/swarming/.swarming/w/ir/x/t/TestMinusRSymsWithSameName1043036679/001/x1.o]
    elf_test.go:196: /home/swarming/.swarming/w/ir/x/w/goroot/bin/go build
    elf_test.go:201: runtime: failed to create new OS thread
    elf_test.go:202: exit status 1
--- FAIL: TestMinusRSymsWithSameName (21.54s)

— watchflakes

@gopherbot gopherbot reopened this Apr 18, 2024
@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
default <- log ~ `SIGQUIT` && log ~ `stopping the world`
2024-04-26 18:12 gotip-openbsd-amd64 go@b384ee7c cmd/go/internal/vcweb/vcstest.TestScripts (log) 
=== RUN   TestScripts
    vcstest_test.go:120: GET http://127.0.0.1:1822:
        <html>
        <title>vcweb</title>
        <pre>
        <b>vcweb</b>
        
        This server serves various version control repos for testing the go command.
        
        For an overview of the script language, see <a href="/help">/help</a>.
...
        hg/custom-hg-hello.txt		2024-04-26T11:30:16-07:00	custom-hg-hello.txt:1: handle hg: server for `hg` VCS is not installed
        hg/hello.txt			2024-04-26T11:30:16-07:00	hello.txt:1: handle hg: server for `hg` VCS is not installed
        hg/hgrepo1.txt			2024-04-26T11:30:58-07:00	hgrepo1.txt:1: handle hg: server for `hg` VCS is not installed
        hg/vgotest1.txt			2024-04-26T11:30:24-07:00	vgotest1.txt:1: handle hg: server for `hg` VCS is not installed
        insecure.txt			2024-04-26T11:30:16-07:00	ok
        svn/hello.txt			2024-04-26T11:30:16-07:00	hello.txt:1: handle svn: server for `svn` VCS is not installed
        svn/nonexistent.txt		2024-04-26T11:30:16-07:00	nonexistent.txt:1: handle svn: server for `svn` VCS is not installed
        svn/test1-svn-git.txt		2024-04-26T11:30:16-07:00	test1-svn-git.txt:1: handle svn: server for `svn` VCS is not installed
        svn/test2-svn-git.txt		2024-04-26T11:30:16-07:00	test2-svn-git.txt:1: handle svn: server for `svn` VCS is not installed
--- FAIL: TestScripts (10.05s)
2024-04-26 18:12 gotip-openbsd-amd64 go@b384ee7c cmd/link.TestExtLinkCmdlineDeterminism (log) 
=== RUN   TestExtLinkCmdlineDeterminism
=== PAUSE TestExtLinkCmdlineDeterminism
=== CONT  TestExtLinkCmdlineDeterminism
    link_test.go:1250: output differ:
        host link: "clang" "-m64" "-pthread" "-Wl,-nopie" "-o" a.out "-Wl,--export-dynamic-symbol=F1" "-Wl,--export-dynamic-symbol=F2" "-Wl,--export-dynamic-symbol=F3" "-Wl,--export-dynamic-symbol=__guard_local" "-Wl,--export-dynamic-symbol=_cgo_panic" "-Wl,--export-dynamic-symbol=_cgo_topofstack" "-Wl,--export-dynamic-symbol=crosscall2" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/go.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000000.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000001.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000002.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000003.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000004.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000005.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000006.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000007.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000008.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000009.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000010.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000011.o" "-O2" "-g" "-O2" "-g" "-lpthread" "-no-pie"
        ==========
        host link: "clang" "-m64" "-pthread" "-Wl,-nopie" "-o" a.out "-Wl,--export-dynamic-symbol=F1" "-Wl,--export-dynamic-symbol=F2" "-Wl,--export-dynamic-symbol=F3" "-Wl,--export-dynamic-symbol=__guard_local" "-Wl,--export-dynamic-symbol=_cgo_panic" "-Wl,--export-dynamic-symbol=_cgo_topofstack" "-Wl,--export-dynamic-symbol=crosscall2" "-Qunused-arguments" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/go.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000000.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000001.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000002.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000003.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000004.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000005.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000006.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000007.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000008.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000009.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000010.o" "/home/swarming/.swarming/w/ir/x/t/TestExtLinkCmdlineDeterminism1762047829/001/linktmp/000011.o" "-O2" "-g" "-O2" "-g" "-lpthread" "-no-pie"
--- FAIL: TestExtLinkCmdlineDeterminism (82.32s)

— watchflakes

@seankhliao seankhliao removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jul 14, 2024
@mknyszek mknyszek changed the title runtime: deadlock while stopping the world (currently netbsd/386) runtime: deadlock while stopping the world (most recently openbsd/amd64) Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-OpenBSD
Projects
Go Compiler / Runtime
Status: Todo
Test Flakes
Status: Done
Milestone
Backlog
Development

No branches or pull requests
3 participants
@mknyszek @gopherbot @seankhliao