cmd/go: go test should not clutter output with "no test files"

[joelpresence]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.8 darwin/amd64

What operating system and processor architecture are you using (go env)?

amd64, darwin

What did you do?

bash> cd ~/go/src/myproject
bash> go test github.com/presencelabs/...
...
go test outputs some useful test results like
ok github.com/presencelabs/ourapp/apiserver/apiservercore/rendermodels/tests 0.024s
...
go test outputs a lot of unuseful test results like
? github.com/presencelabs/ourapp/apiserver/apiservercore/service [no test files]

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

What did you expect to see?

I only want to see output about which tests ran and which passed and which failed. I do NOT care about packages or dirs that have no tests in them.

What did you see instead?

I don't care about the directories/packages with no test files. I know that they don't have any test files. They are not meant to have any test files since we put our test files in subdirs called test. Telling me repeatedly that these dirs have no test files clutters the test output and obscures the important information like which tests actually ran and which passed and which failed.

Telling me that a dir has no test files should be left to the coverage tool/option. Either go test should NOT log dirs/packages that have no tests by default, or there should be an option to skip that logging as in --no-warn-no-tests or similar.

I'm happy to help with a PR if there's interest. But right now, all this clutter about no tests is really reducing our productivity and we need to run tests like clear; go test github.com/presencelabs/... | grep -v "no test files" which is cumbersome.

By the way, we love go! Thanks for your hard work on it. :-) I'm not meaning to whine, I just want to make go even better.

[kevinburke]

I believe the previous behavior was also to print the package name and a question mark. The [no test files] is to distinguish I think between cases where there were tests, but they were all skipped, and when there are no tests.

[dmitshur]

As I understand, the purpose of go test is to do two things. First, test that the package builds without errors, and second, that tests/examples (maybe even benchmarks, with -bench flag) are executed without failing. Examples without "// Output:" comments are only compiled, not even executed.

Go packages that have no test files can still be tested for build errors. So when you see ? and [no test files], it just means the package compiles successfully but there weren't any test files to use (perhaps because of mismatching build tags). I find this very useful, because not all my Go packages have tests, but I still want to know they were tested for build errors, at the very least.

I don't care about the directories/packages with no test files. I know that they don't have any test files. They are not meant to have any test files since we put our test files in subdirs called test.

You said you did go test github.com/presencelabs/.... The command did what you asked it to, and tested all packages, since you specified /... suffix in the import path pattern.

If you want to test only a single package named test, rather than all, then you should specify that as an argument to go test.

For example:

go test github.com/presencelabs/something/test

You can even get creative with patterns and do something like go test github.com/presencelabs/.../test to test all packages named test inside github.com/presencelabs.

But the point is you shouldn't run go test on packages that you don't want to test. Seeing 1 line of output for every 1 package being tested is very useful.

[joelpresence]

@shurcooL thanks for the response!

I don't look at go test as a way to verify that my code builds without compilation errors. I already have go build or my IDE to do that. I view go test as a way to verify that my tests pass.

Also, I tried github.com/presencelabs/.../test but it failed with:

warning: "github.com/presencelabs/.../test" matched no packages
no packages to test

Any ideas?

I would respectfully disagree with your statement "But the point is you shouldn't run go test on packages that you don't want to test" and revise it to be "But the point is go test shouldn't try to test any packages that don't contain tests".

Thoughts?

Thanks!

[dmitshur]

Also, I tried github.com/presencelabs/.../test but it failed with:

warning: "github.com/presencelabs/.../test" matched no packages

That means you don't have any packages that match that import path pattern. It looks like they're private repos, so I can't help you without knowing what they are.

You can read more about the import path patterns at https://golang.org/cmd/go/#hdr-Description_of_package_lists.

As an example, you can also try go test net/...test to test all packages that end with test inside net:

$ go test net/...test
ok  	net/http/httptest	0.012s
ok  	net/internal/socktest	0.007s

I would respectfully disagree with your statement "But the point is you shouldn't run go test on packages that you don't want to test" and revise it to be "But the point is go test shouldn't try to test any packages that don't contain tests".

Fair enough. I guess it's a matter of opinion/preference/what one is used to. I can see both ways can make sense, in their own way.

However, go test has always included all Go packages since 1.0 (as far as I know), regardless if they have test files or not, so that's why I'm used to and happy with its behavior.

I already have go build or my IDE to do that. I view go test as a way to verify that my tests pass.

Unlike go test, go build has side-effects (it can potentially write files to your current directory), so it's not a great command to check if everything works after you've done a refactor. I don't think having an IDE should be a requirement to test that packages build successfully either.

[joelpresence]

Thanks again @shurcooL !

Actually, my mistake, my tests are in a subdir/subpackage called tests not test so the command go test github.com/presencelabs/.../tests works just fine and only logs actual tests! Yay! :-)

I still would like to see this behavior as at least an option for go test but I have a working solution for my needs right now ...

Thoughts?

[dmitshur]

Actually, my mistake, my tests are in a subdir/subpackage called tests not test so the command go test github.com/presencelabs/.../tests works just fine and only logs actual tests! ... I have a working solution for my needs right now

Glad to hear that! 👍

I still would like to see this behavior as at least an option for go test ... Thoughts?

My personal preference is to set a really high bar for inclusion of flags/options, and aim to have as few as possible. Existing ones are rarely removed. But each one adds some overhead, increasing the number of options people need to be aware of, diluting the value offered by other flags. In an ideal world, only the absolutely critical and useful flags would exist.

This is not a flag I'd want added, since I suspect its value offered would not warrant the cost of its inclusion. It's easy to filter out unwanted output, either by not including it in the go test arguments, or by using something like | grep -v "unwanted", or a custom tool for formatting test output in a way you want.

[joelpresence]

Thanks @shurcooL .

But along the lines of your dilution argument, I'd argue that go test logging a bunch of spammy lines like "no test files" dilutes the value of the log lines, since "no test files" is not interesting or helpful - don't we really just want to see if tests pass or fail? Checking if code compiles is too low a bar, but tests passing or failing is help IMHO ...

[cznic]

I consider the "spam" useful. Particularly when build tags are used it's easy to miss that for some combination of build tags there are effectively no tests. When that is not intended, the "warning" by go test reveals the bug.

[joelpresence]

@cznic thanks!

But how is it a "bug" to not have tests in a given package? I personally consider it kind of sloppy and hard to organize packages with tests directly in the package and prefer to put tests in a sub package, as in the tests for package foo would go in package foo/tests which would import package foo. This is much easier to organize IMHO and makes it much easier to see the important files in foo in an IDE project organizer view (file tree) without seeing all the clutter of the tests directory.

In that case, I know that there are no tests in package foo and I don't want go test to remind me of what I already know ... It feels heavy-handed.

But I respect that others feel differently, and that's OK! :-)

[rsc]

go test github.com/presencelabs/... means apply go test to each of the packages listed by go list github.com/presencelabs/.... Some line must be printed for the package without tests. "ok" is misleading, failing is clearly not what you want, so we print "?" to indicate that there are no results. The [] on the end of the line explain the "?" (sometimes the -run flag means there are simply no matching tests).

This is working as intended, as others have noted.