cmd/dist: modify all.bash to no longer skip testing packages without tests

[dmitshur]

CL 10492 in 2015 applied a change to have all.bash skip testing Go packages in the standard library (those matched by go list std cmd) that have no tests. Sadly, the commit message doesn't specify what exactly motivated the change. A guess based on review comments is that it may have been to avoid doing unnecessary work and speed things up, as well as possibly to reduce verbosity of all.bash output.

By now, running go test on a Go package without tests can still produce useful information, such as the high-confidence vet checks that run (e.g., it would've caught an unused printf argument in TryBots), the fact that the package builds successfully, and so on. Furthermore, we have been generally wanting for all.bash not to miss any legitimate problems that a user running go test -short std cmd locally might find.

So we can revisit this. Some things worth considering here:

• measure a bit on how much time it adds to all.bash (both locally and on TryBots)
• think if the vendored packages should or shouldn't be included
  • their upstream tests do not get vendored into GOROOT, and by definition are already tested in the repos where their upstream source lives
  • having them in all.bash output adds quite a few lines

CC @golang/release, @aclements, @bcmills.

[gopherbot]

Change https://go.dev/cl/498270 mentions this issue: cmd/go/internal/modget: remove unused base.Errorf argument

[aclements]

One option, if we want to minimize output noise, is that dist could do some light filtering of the output to remove skipped package lines. It's a bit odd that the output of dist and go test diverge today, though. It seems annoying to have the skipped packages in the dist output, but I'm having trouble justifying why it would make a different decision about output than go test.

[aclements]

Good example of an issue this would have caught: https://go.dev/cl/501220 introduced a malformed fmt.Fprintf in cmd/cgo. Since cmd/cgo has no test files, dist test skips it, even though go test would have run vet and caught this. (This is also https://go.dev/cl/501224 and https://go.dev/cl/501216)

[dmitshur]

We discussed this earlier and there was agreement the high-confidence vet coverage will almost certainly warrant doing this now, just need to sort out some of the small implementation details here.

[aclements]

I just chatted with Russ about this and he thinks we should just go ahead and do this, and not worry about the increased noise or excluding vendored packages (vendored packages all pass go vet right now, and we probably want to know if they stop passing vet).