Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testing: Coverage() function doesnt work for go 1.20+ #59590

Closed
jehowell opened this issue Apr 12, 2023 · 9 comments
Closed

testing: Coverage() function doesnt work for go 1.20+ #59590

jehowell opened this issue Apr 12, 2023 · 9 comments
Assignees
@thanm
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@jehowell
Copy link

jehowell commented Apr 12, 2023 •
edited
Loading

What version of Go are you using (go version)?

$ go version
go version go1.20.3 linux/arm64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN="/go/bin"
GOCACHE="/home/vscode/.cache/go-build"
GOENV="/home/vscode/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_arm64"
GOVCS=""
GOVERSION="go1.20.3"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/workspaces/work/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2929308236=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Calling testing.Coverage() always returns 0.0. Looking at the source, this function has not been updated to the new coverage2 system.

What did you expect to see?

testing.Coverage() should return the percent of code coverage for the current unit test.

What did you see instead?

Always returns 0
@jehowell jehowell changed the title affected/package: testing testing: Coverage() function doesnt work for go 1.20+ Apr 12, 2023
@ianlancetaylor
Copy link
Member

CC @thanm

@dr2chase dr2chase added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 13, 2023
@thanm thanm self-assigned this Apr 13, 2023
@thanm
Copy link
Contributor

thanm commented Apr 13, 2023

Thanks for the report. testing.Coverage() is definitely something that slipped through the cracks (it currently has no unit tests); I'll add this to my other cleanup items for code coverage this release.

@jehowell
Copy link
Author

jehowell commented Apr 13, 2023 •
edited
Loading

Awesome thanks! In addition, it would be great if that value Coverage() returns is the same as the one that gets sent to the coverage report after the test. The previous implementation did not calculate this value the same way so the value returned differed from the coverage report (% of blocks vs % of statements).

We use this value as a pseudo way to enforce test coverage. Would love to see this more integrated into the tools in the future to do this in a more official way.

@thanm
Copy link
Contributor

thanm commented Apr 27, 2023 via email

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/495450 mentions this issue: cmd/go,testing: re-implement testing.Coverage

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/495449 mentions this issue: runtime/coverage: add coverage snapshot helper routine

gopherbot pushed a commit that referenced this issue May 23, 2023
@thanm
runtime/coverage: add coverage snapshot helper routine
ef67022 
Add a new function runtime/coverage.snapshot(), which samples the
current values of coverage counters in a running "go test -cover"
binary and returns percentage of statements executed so far. This
function is intended to be used by the function testing.Coverage().

Updates #59590.

Change-Id: I861393701c0cef47b4980aec14331168a9e64e8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/495449
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
@lbergesio
Copy link

Hi! I was expecting this to be backported to 1.20.5 but it was not. I saw the commit is in release-branch.go1.21 but not in 1.21.rc2- Is it planned to be backported to 1.20.6? In 1.20.x this is totally broken. Thaks!

@thanm
Copy link
Contributor

thanm commented Jun 27, 2023

From https://github.com/golang/go/wiki/MinorReleases:

Our default decision should always be to not backport, but fixes for security issues, serious problems with no workaround, and documentation fixes are backported to the most recent two release branches, if applicable to that branch

Generally speaking "serious problem" means your program is crashing or computing incorrect results, that sort of thing. I would be surprised in this case if the release team were to approve a backport given the nature of the bug. Sorry about that.

@lbergesio
Copy link

lbergesio commented Jun 27, 2023 •
edited
Loading

So, testing.Coverage() is returning 0 always, is not that an incorrect computing result?. My program is not crashing but my test suite is.

So I need the build -cover feature introduced in 1.20 but can't break my test suite. I was just asking if I can expect this to be fixed in 1.20.x, otherwise I will wait to 1.21

@lbergesio lbergesio mentioned this issue Aug 22, 2023
Open
@golang golang locked and limited conversation to collaborators Jun 26, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@thanm thanm

Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@lbergesio @dr2chase @ianlancetaylor @jehowell @gopherbot @thanm