Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go test -v does not stream logs when testing multiple packages #46959

Open
leighmcculloch opened this issue Jun 28, 2021 · 3 comments
Open

Comments

@leighmcculloch
Copy link
Contributor

@leighmcculloch leighmcculloch commented Jun 28, 2021

What version of Go are you using (go version)?

$ go version
go version go1.16.5 linux/amd64

Does this issue reproduce with the latest release?

Yes, this reproduces on Go 1.16.5.

This also reproduces on Go 1.14, the version that introduced test log streaming.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN="/home/vscode/.local/bin"
GOCACHE="/home/vscode/.cache/go-build"
GOENV="/home/vscode/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/vscode/.local/bin/go/1.16.5"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/vscode/.local/bin/go/1.16.5/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.5"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="0"
GOMOD="/workspaces/stellar--experimental-payment-channels/sdk/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build112618337=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I ran go test -v ./... in a Go module that has multiple packages, each with tests.

What did you expect to see?

I expected to see the logs stream for the test(s) currently running.

According to the Go 1.14 release notes:

go test -v now streams t.Log output as it happens, rather than at the end of all tests.

Ref: https://golang.org/doc/go1.14

What did you see instead?

The logs did not stream to stdout, but were batched until each test completed.

When I run the tests for a single package (go test -v ./pkg1), or if I run the tests without package parallelism (go test -p 1 -v ./...), the logs do stream.

It looks to me like this is probably behaving as intended, especially since Go 1.14 had this same behavior, but the release notes are misleading since they simply state that logs stream but not that they only stream under specific conditions.

I haven't found documentation about the interactions between log streaming and test package parallelism. I haven't found any documentation about test package parallelism either, and the help pages inside the go tool itself about the -p flag are difficult to find. To find a definition of it, one has to follow two references, so the path to discovering it is go help test, then go help testflags, then go help build.

Possibly this is an opportunity for improved docs, but it's unclear to me where that would need to go.

@mvdan
Copy link
Member

@mvdan mvdan commented Jun 29, 2021

go help test does have a section related to this:

Go test runs in two different modes:

It talks about the difference between running go test without and with package arguments. Initially, I was going to reply to this issue saying that only in the mode without arguments you were meant to get streaming output, but that's clearly wrong - like you say, go test -v ./pkg streams output too.

Implementing output streaming for multiple packages is harder, for what it's worth. Like others mentioned in the original thread, output from testing multiple packages is buffered, and legibility could suffer from streaming. So my guess is that not including support for multiple packages was on purpose.

All that said, I agree this should be better documented. I'd perhaps put a short paragraph in go help test.

@bboreham
Copy link
Contributor

@bboreham bboreham commented Aug 2, 2021

Similar to #27826 ?

@MadLittleMods
Copy link

@MadLittleMods MadLittleMods commented Oct 20, 2021

It would be nice to override this behavior. In my use cases, I'm only running some specific tests via go test -v -run TestImportHistoricalMessages ./tests/... which are all in a single file (let alone the same package) but
the directory,./tests/..., happens to include multiple packages.

For the curious, here are the actual tests: https://github.com/matrix-org/complement/tree/050afc551c2695e8e055903f8e8ad6f3ecce17a0/tests (tests and csapi package)

As a workaround, I can get the log streaming behavior I want by being very selective about the files to run: go test -v -run TestImportHistoricalMessages ./tests/main_test.go ./tests/msc2716_test.go but this is manual process adjusting our generic test runner every time.


Tracking this internally at matrix-org/complement#215

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants