testing: Tests timeout with Go 1.20.x, but pass with Go 1.19 and 1.18 in ~20 seconds.

[MarvinJWendt]

What version of Go are you using (go version)?

go version go1.20.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
  GOARCH="amd64"
  GOBIN=""
  GOCACHE="/home/runner/.cache/go-build"
  GOENV="/home/runner/.config/go/env"
  GOEXE=""
  GOEXPERIMENT=""
  GOFLAGS=""
  GOHOSTARCH="amd64"
  GOHOSTOS="linux"
  GOINSECURE=""
  GOMODCACHE="/home/runner/go/pkg/mod"
  GONOPROXY=""
  GONOSUMDB=""
  GOOS="linux"
  GOPATH="/home/runner/go"
  GOPRIVATE=""
  GOPROXY="https://proxy.golang.org,direct"
  GOROOT="/opt/hostedtoolcache/go/1.[20](https://github.com/pterm/pterm/actions/runs/4332579616/jobs/7565164242#step:2:21).1/x64"
  GOSUMDB="sum.golang.org"
  GOTMPDIR=""
  GOTOOLDIR="/opt/hostedtoolcache/go/1.20.1/x64/pkg/tool/linux_amd64"
  GOVCS=""
  GOVERSION="go1.20.1"
  GCCGO="gccgo"
  GOAMD64="v1"
  AR="ar"
  CC="gcc"
  CXX="g++"
  CGO_ENABLED="1"
  GOMOD="/dev/null"
  GOWORK=""
  CGO_CFLAGS="-O2 -g"
  CGO_CPPFLAGS=""
  CGO_CXXFLAGS="-O2 -g"
  CGO_FFLAGS="-O2 -g"
  CGO_LDFLAGS="-O2 -g"
  PKG_CONFIG="pkg-config"
  GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build33[22](https://github.com/pterm/pterm/actions/runs/4332579616/jobs/7565164242#step:2:23)00[46](https://github.com/pterm/pterm/actions/runs/4332579616/jobs/7565164242#step:2:47)00=/tmp/go-build -gno-record-gcc-switches"

What did you do?

My package tests pass on go 1.19 and below, but not on Go 1.20.x.
When I run the tests with go 1.19 they pass in 19 seconds. When I do the same with Go > 1.20 they fail on a timeout after 11 minutes. I have verified this behavior on multiple systems and operating systems (pipeline matrix test and local).
I used -v -p 1 as tests flags, and it seems like random tests are hanging.

I have also verified that the tests pass on Go 1.18.

Pipeline runs

Both pipelines runs are on an ubuntu machine. No code has been modified between the runs, except for the workflows go version.

Passed tests (go 1.19): https://github.com/pterm/pterm/actions/runs/4332770035/jobs/7565480959
Failed tests (go 1.20.1): https://github.com/pterm/pterm/actions/runs/4332579616/jobs/7565164242

What did you expect to see?

Tests passing

What did you see instead?

Test failing

[fzipp]

The code seems to be riddled with data races, see go test -race .

[MarvinJWendt]

Hi @fzipp yes it looks like that, but that never failed the tests since go 1.15.
I am also running the tests with -p 1 to prevent multiple tests at the same time (that should help with the races, right?).
I am curious why go 1.20 constantly lets the tests fail.
Seems like this repo had a similar issue with go 1.20: concourse/concourse#8680
They also use Stdin inside their tests, maybe that's the reason?
Unfortunately, I use Stdin and Stdout in most tests, so it's not as easy to fix as they did.

I tried to find the commit that broke the tests by git bisecting the Go buildchain, but I am not experienced in building Go and sometimes the go build crashes (I guess my system is just not set up properly for building Go).

Fixing the race conditions is an open PR in my library, but also the biggest refactor we ever had (and most likely a breaking change). So we have to be very careful with it.

My current workaround is to use Go 1.19 in the pipeline, but that shouldn't be a requirement to run our tests.

Do you have any idea how I could instruct Go 1.20 to work with the tests? Or maybe a hint what change in Go broke this?

[qiulaidongfeng]

I use go1.20.1 to run
Go test - face - test. cpu=1 -trimpath There is still data competition
race.txt

Go1.20 New fields for exec.Cmd
Cancel, if Cancel is not nil, then Cmd must be created with CommandContext
WaitDelay, the default value is 0, then I/O pipes will be read before EOF. I remember that the default behavior is the same before.
If you don't use the new fields in go1.20 exec.Cmd, you can fix the data competition first.

[ianlancetaylor]

If the code has data races, then we aren't going to investigate why the behavior has changed. Sorry. If someone wants to take a look please go ahead, but we're unlikely to make any changes to the Go tools because of racy code.