New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go test -cover & go test -coverprofile should always output a coverage #24570

Open
gustafj opened this Issue Mar 27, 2018 · 8 comments

Comments

Projects
None yet
6 participants
@gustafj

gustafj commented Mar 27, 2018

Description

Now in 1.10 when go test -cover supports multiple packages, I would expect it to print out a percentage for all packages (including those missing tests).
And for go test -coverprofile, I would expect all packages to be included in the calculated total.

Currently only packages that have at least one test (can be a *_test.go with only the package declaration) is included, see pkg2 below.

What version of Go are you using (go version)?

go version go1.10 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

Linux, amd64

What did you do?

go test ./... -cover
go test ./... -coverprofile cover.out; go tool cover -func cover.out

What did you expect to see?

?      path/to/pkg1   0.001s  coverage:   0.0% of statements [no test files]
ok     path/to/pkg2   0.019s  coverage:   0.0% of statements [no tests to run]
ok     path/to/pkg3   0.371s  coverage: 100.0% of statements
path/to/pkg1/pkg1.go:5: String 0.0%
path/to/pkg2/pkg2.go:5: String 0.0%
path/to/pkg3/pkg3.go:5: String 100.0%
total: (statements) 33.3%

What did you see instead?

?      path/to/pkg1   [no test files]
ok     path/to/pkg2   0.019s  coverage:   0.0% of statements [no tests to run]
ok     path/to/pkg3   0.371s  coverage: 100.0% of statements
path/to/pkg2/pkg2.go:5: String 0.0%
path/to/pkg3/pkg3.go:5: String 100.0%
total: (statements) 50.0%

@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Mar 27, 2018

@gopherbot

This comment has been minimized.

gopherbot commented May 29, 2018

Change https://golang.org/cl/115095 mentions this issue: cmd/go/internal/test: always output a coverage

@mvdan

This comment has been minimized.

Member

mvdan commented May 30, 2018

@kyroy noticed that this issue is very similar to #25492. It seems to me like both should be fixed at once, as they suggest changes in opposite directions. The other issue wants to change a 0.0% with a [no statements], and this one wants to change [no test files] with 0.0%.

/cc @bcmills @egonk

@bcmills

This comment has been minimized.

Member

bcmills commented May 30, 2018

they suggest changes in opposite directions.

#25492 is about the case where there are no statements to cover (and thus the percentage is undefined). That's materially different from the case here, where pkg1 does have a statement and it is not covered: 0.0% coverage is well-defined and correct for a package with one statement and no tests.

@mvdan

This comment has been minimized.

Member

mvdan commented May 30, 2018

My reading of this issue is that go test -cover should always output a percentage if it succeeded. The other issue is precisely about removing a 0.0% output in favor of something else.

@bcmills

This comment has been minimized.

Member

bcmills commented May 30, 2018

My reading of this issue is that go test -cover should always output a percentage if it succeeded.

That's the current issue title, but that behavior seems clearly wrong if there are no statements to cover: 0/0 is not a well-defined percentage.

(I guess we could output NaN%, but that seems strictly less helpful than [no statements].)

@kyroy

This comment has been minimized.

Contributor

kyroy commented May 30, 2018

I like the idea of a consistent output that would be created by just implementing this issue.

Another approach is, as in #25492, for all statements it is true that they are covered by the tests. So one could argue for 100% which is obviously mathematically incorrect.

I am also the creator of the CL. Happy to implement a solution that results from the discussion :)

@gopherbot gopherbot closed this in ebb8a1f Jun 6, 2018

sdboyer added a commit to sdboyer/dep that referenced this issue Jul 2, 2018

ci: Ignore main file in licenseok
This should fix test failures that were being caused by golang/go#24570,
which caused test binaries to be built even when there are no test
files. The poor interaction there arose from the setting of flags in our
init() function, it seems.

sdboyer added a commit to sdboyer/dep that referenced this issue Jul 3, 2018

ci: Ignore main file in licenseok
This should fix test failures that were being caused by golang/go#24570,
which caused test binaries to be built even when there are no test
files. The poor interaction there arose from the setting of flags in our
init() function, it seems.
@gopherbot

This comment has been minimized.

gopherbot commented Jul 6, 2018

Change https://golang.org/cl/122518 mentions this issue: cmd/go: revert "output coverage report even if there are no test files"

gopherbot pushed a commit that referenced this issue Jul 9, 2018

cmd/go: revert "output coverage report even if there are no test files"
Original CL description:

    When using test -cover or -coverprofile the output for "no test files"
    is the same format as for "no tests to run".

Reverting because this CL changed cmd/go to build test binaries for
packages that have no tests, leading to extra work and confusion.

Updates #24570
Fixes #25789
Fixes #26157
Fixes #26242

Change-Id: Ibab1307d39dfaec0de9359d6d96706e3910c8efd
Reviewed-on: https://go-review.googlesource.com/122518
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Jul 9, 2018

CL 115095 was reverted, so reopening this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment