Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: TestGdbCoreSignalBacktrace fails when run outside of all.bash #61487

Closed
siebenmann opened this issue Jul 20, 2023 · 10 comments
Closed
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@siebenmann
Copy link

What version of Go are you using (go version)?

This is the current git tip:

$ go version
go version devel go1.22-57e2eb64eb Thu Jul 20 20:54:38 2023 +0000 linux/amd64

Does this issue reproduce with the latest release?

No. This doesn't happen with a 'git checkout go1.20.6', building that Go, and then running go test -short std cmd.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/cks/.cache/go-build'
GOENV='/home/cks/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/cks/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/cks/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/w/620/cks/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/w/620/cks/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='devel go1.22-57e2eb64eb Thu Jul 20 20:54:38 2023 +0000'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/w/620/cks/go/src/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1776540564=/tmp/go-build -gno-record-gcc-switches'

This is on an up to date Ubuntu 22.04 LTS Linux server, on amd64.

What did you do?

In a git tip clone, run ./all.bash, which will succeed. Then run go test -short std cmd, which will report various things and eventually a FAIL for TestGdbCoreSignalBacktrace, although it will print a bunch of ?s and other things that may be the sign of additional problems. I'm attaching the full 'go test' output rather than trying to cut it down but omit something that turns out to matter.

go-test-out.txt

This may be somehow related to #61484.

@cherrymui
Copy link
Member

The ?s are fine. They are just packages that don't have tests.

The TestGdbCoreSignalBacktrace failure is probably sporadic. Perhaps when you ran all.bash it succeeded. You may try running go test -count=1 -run=TestGdbCoreSignalBacktrace runtime and see if it always fails or fails sometimes. Thanks.

@siebenmann
Copy link
Author

Running go test -count=1 -run=TestGdbCoreSignalBacktrace runtime appears to always fail for me. I ran it in a 'while ! go test ...' loop to be sure, and it's still failing away now as I write this. I've also never seen all.bash fail, although I haven't monitored that as extensively.

@bcmills bcmills changed the title 'go test -short std cmd' fails in TestGdbCoreSignalBacktrace although all.bash succeeds runtime: 'go test -short std cmd' fails in TestGdbCoreSignalBacktrace Jul 21, 2023
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 21, 2023
@bcmills bcmills changed the title runtime: 'go test -short std cmd' fails in TestGdbCoreSignalBacktrace runtime: TestGdbCoreSignalBacktrace fails when run outside of all.bash Jul 21, 2023
@bcmills bcmills added this to the Go1.22 milestone Jul 21, 2023
@heschi heschi added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 21, 2023
@heschi
Copy link
Contributor

heschi commented Jul 21, 2023

cc @golang/runtime

@cherrymui
Copy link
Member

Perhaps all.bash (cmd/dist) changes some environment and that particular test is skipped (e.g. due to failing to create a core file or something)?

Are you running all.bash and the go test command under the same user and same environment?

@siebenmann
Copy link
Author

I'm running both all.bash and go test in the same user and environment. I first noticed this in the context of #61484, when I was asked to run go test directly to see if the Go build cache growth occurred there or only when I ran all.bash. I can't tell if all.bash is running that test; in an all.bash run the report I get is:

[...]
ok      reflect 0.609s
ok      regexp  0.535s
ok      regexp/syntax   0.736s
ok      runtime 25.048s
ok      runtime/cgo     0.014s
ok      runtime/coverage        0.003s
ok      runtime/debug   0.122s
ok      runtime/internal/atomic 0.062s
ok      runtime/internal/math   0.019s
ok      runtime/internal/sys    0.014s
ok      runtime/internal/syscall        0.005s
ok      runtime/internal/wasitest       0.007s
ok      runtime/metrics 0.012s
ok      runtime/pprof   7.964s
ok      runtime/trace   1.999s
[...]

@ianlancetaylor
Copy link
Contributor

The test is failing because it expects a core file to exits, but it doesn't. The test does pass for me on my linux-amd64 system.

There is a check in the test for whether a core file will be generated:

	// Ensure there is enough RLIMIT_CORE available to generate a full core.
	var lim syscall.Rlimit
	err := syscall.Getrlimit(syscall.RLIMIT_CORE, &lim)
	if err != nil {
		t.Fatalf("error getting rlimit: %v", err)
	}
	// Minimum RLIMIT_CORE max to allow. This is a conservative estimate.
	// Most systems allow infinity.
	const minRlimitCore = 100 << 20 // 100 MB
	if lim.Max < minRlimitCore {
		t.Skipf("RLIMIT_CORE max too low: %#+v", lim)
	}

In run.bash we disable core files:

# no core files, please
ulimit -c 0

Therefore, the reason this test passes when run via all.bash is that core files are disabled and the test is skipped.

When run directly, the test assumes that the core file will be named core. On modern Linux systems the name of a core file is controlled by the contents of /proc/sys/kernel/core_pattern. On my system that file contains just core. @siebenmann what does it look like on your system? Thanks.

@siebenmann
Copy link
Author

siebenmann commented Jul 21, 2023

On this system, /proc/sys/kernel/core_pattern contains core, but /proc/sys/kernel/core_uses_pid is set to 1, which creates core files named 'core.PID-VALUE'. The latter setting appears to be the Ubuntu 22.04 default, and perhaps the upstream kernel default.

@ianlancetaylor
Copy link
Contributor

Thanks, that is what is causing the problem on your system. On my system /proc/sys/kernel/core_uses_pid is 0.

Can you see if https://go.dev/cl/511659 fixes the problem for you? Thanks.

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/511659 mentions this issue: runtime: consider core PID in gdb test

@siebenmann
Copy link
Author

This CL fixes the problem for me; go test -count=1 -run=TestGdbCoreSignalBacktrace runtime now succeeds, as does go test -short std cmd.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

6 participants