Permalink
Browse files

cmd/go: run vet automatically during go test

This CL adds an automatic, limited "go vet" to "go test".
If the building of a test package fails, vet is not run.
If vet fails, the test is not run.
The goal is that users don't notice vet as part of the "go test"
process at all, until vet speaks up and says something important.
This should help users find real problems in their code faster
(vet can just point to them instead of needing to debug a
test failure) and expands the scope of what kinds of things
vet can help with.

The "go vet" runs in parallel with the linking of the test binary,
so for incremental builds it typically does not slow the overall
"go test" at all: there's spare machine capacity during the link.

all.bash has less spare machine capacity. This CL increases
the time for all.bash on my laptop from 4m41s to 4m48s (+2.5%)

To opt out for a given run, use "go test -vet=off".

The vet checks used during "go test" are a subset of the full set,
restricted to ones that are 100% correct and therefore acceptable
to make mandatory. In this CL, that set is atomic, bool, buildtags,
nilfunc, and printf. Including printf is debatable, but I want to
include it for now and find out what needs to be scaled back.
(It already found one real problem in package os's tests that
previous go vet os had not turned up.)
Now that we can rely on type information it may be that printf
should make its function-name-based heuristic less aggressive
and have a whitelist of known print/printf functions.
Determining the exact set for Go 1.10 is #18085.

Running vet also means that programs now have to type-check
with both cmd/compile and go/types in order to pass "go test".
We don't start vet until cmd/compile has built the test package,
so normally the added go/types check doesn't find anything.
However, there is at least one instance where go/types is more
precise than cmd/compile: declared and not used errors involving
variables captured into closures.

This CL includes a printf fix to os/os_test.go and many declared
and not used fixes in the race detector tests.

Fixes #18084.

Change-Id: I353e00b9d1f9fec540c7557db5653e7501f5e1c9
Reviewed-on: https://go-review.googlesource.com/74356
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
  • Loading branch information...
rsc committed Oct 31, 2017
1 parent bd95f88 commit 0d188752524282496ebd0ab4b382bb4ff8750c90
@@ -504,7 +504,7 @@ func testABIHashNote(t *testing.T, f *elf.File, note *note) {
t.Errorf("%s has incorrect binding %v, want STB_LOCAL", hashbytes.Name, elf.ST_BIND(hashbytes.Info))
}
if f.Sections[hashbytes.Section] != note.section {
t.Errorf("%s has incorrect section %v, want %s", hashbytes.Name, f.Sections[hashbytes.Section].Name, note.section)
t.Errorf("%s has incorrect section %v, want %s", hashbytes.Name, f.Sections[hashbytes.Section].Name, note.section.Name)
}
if hashbytes.Value-note.section.Addr != 16 {
t.Errorf("%s has incorrect offset into section %d, want 16", hashbytes.Name, hashbytes.Value-note.section.Addr)
View
@@ -735,6 +735,13 @@
// The go tool will ignore a directory named "testdata", making it available
// to hold ancillary data needed by the tests.
//
// As part of building a test binary, go test runs go vet on the package
// and its test source files to identify significant problems. If go vet
// finds any problems, go test reports those and does not run the test binary.
// Only a high-confidence subset of the default go vet checks are used.
// To disable the running of go vet, use the -vet=off flag.
//
//
// Go test runs in two different modes: local directory mode when invoked with
// no package arguments (for example, 'go test'), and package list mode when
// invoked with package arguments (for example 'go test math', 'go test ./...',
@@ -1547,6 +1554,10 @@
// Verbose output: log all tests as they are run. Also print all
// text from Log and Logf calls even if the test succeeds.
//
// -vet mode
// Configure the invocation of "go vet" during "go test".
// The default is to run "go vet". If mode is "off", vet is disabled.
//
// The following flags are also recognized by 'go test' and can be used to
// profile the tests during execution:
//
View
@@ -4748,6 +4748,7 @@ func TestTestCache(t *testing.T) {
}
tg := testgo(t)
defer tg.cleanup()
tg.parallel()
tg.makeTempdir()
tg.setenv("GOPATH", tg.tempdir)
tg.setenv("GOCACHE", filepath.Join(tg.tempdir, "cache"))
@@ -4834,3 +4835,37 @@ func TestTestCache(t *testing.T) {
tg.grepStderr(`([\\/]link|gccgo).*t4\.test`, "did not relink t4_test")
tg.grepStdout(`ok \tt/t4\t\(cached\)`, "did not cache t/t4")
}
func TestTestVet(t *testing.T) {
tg := testgo(t)
defer tg.cleanup()
tg.parallel()
tg.tempFile("p1_test.go", `
package p
import "testing"
func Test(t *testing.T) {
t.Logf("%d") // oops
}
`)
tg.runFail("test", filepath.Join(tg.tempdir, "p1_test.go"))
tg.grepStderr(`Logf format %d`, "did not diagnose bad Logf")
tg.run("test", "-vet=off", filepath.Join(tg.tempdir, "p1_test.go"))
tg.grepStdout(`^ok`, "did not print test summary")
tg.tempFile("p1.go", `
package p
import "fmt"
func F() {
fmt.Printf("%d") // oops
}
`)
tg.runFail("test", filepath.Join(tg.tempdir, "p1.go"))
tg.grepStderr(`Printf format %d`, "did not diagnose bad Printf")
tg.run("test", "-x", "-vet=shift", filepath.Join(tg.tempdir, "p1.go"))
tg.grepStderr(`[\\/]vet.*-shift`, "did not run vet with -shift")
tg.grepStdout(`\[no test files\]`, "did not print test summary")
tg.run("test", "-vet=off", filepath.Join(tg.tempdir, "p1.go"))
tg.grepStdout(`\[no test files\]`, "did not print test summary")
}
@@ -69,6 +69,12 @@ separate package, and then linked and run with the main test binary.
The go tool will ignore a directory named "testdata", making it available
to hold ancillary data needed by the tests.
As part of building a test binary, go test runs go vet on the package
and its test source files to identify significant problems. If go vet
finds any problems, go test reports those and does not run the test binary.
Only a high-confidence subset of the default go vet checks are used.
To disable the running of go vet, use the -vet=off flag.
Go test runs in two different modes: local directory mode when invoked with
no package arguments (for example, 'go test'), and package list mode when
invoked with package arguments (for example 'go test math', 'go test ./...',
@@ -258,6 +264,13 @@ const testFlag2 = `
Verbose output: log all tests as they are run. Also print all
text from Log and Logf calls even if the test succeeds.
-vet list
Configure the invocation of "go vet" during "go test"
to use the comma-separated list of vet checks.
If list is empty, "go test" runs "go vet" with a curated list of
checks believed to be always worth addressing.
If list is "off", "go test" does not run "go vet" at all.
The following flags are also recognized by 'go test' and can be used to
profile the tests during execution:
@@ -446,7 +459,8 @@ var (
testArgs []string
testBench bool
testList bool
testShowPass bool // show passing output
testShowPass bool // show passing output
testVetList string // -vet flag
pkgArgs []string
pkgs []*load.Package
@@ -460,13 +474,41 @@ var testMainDeps = []string{
"testing/internal/testdeps",
}
// testVetFlags is the list of flags to pass to vet when invoked automatically during go test.
var testVetFlags = []string{
// TODO(rsc): Decide which tests are enabled by default.
// See golang.org/issue/18085.
// "-asmdecl",
// "-assign",
"-atomic",
"-bool",
"-buildtags",
// "-cgocall",
// "-composites",
// "-copylocks",
// "-httpresponse",
// "-lostcancel",
// "-methods",
"-nilfunc",
"-printf",
// "-rangeloops",
// "-shift",
// "-structtags",
// "-tests",
// "-unreachable",
// "-unsafeptr",
// "-unusedresult",
}
func runTest(cmd *base.Command, args []string) {
pkgArgs, testArgs = testFlags(args)
work.FindExecCmd() // initialize cached result
work.InstrumentInit()
work.BuildModeInit()
work.VetFlags = testVetFlags
pkgs = load.PackagesForBuild(pkgArgs)
if len(pkgs) == 0 {
base.Fatalf("no packages to test")
@@ -668,6 +710,7 @@ func builderTest(b *work.Builder, p *load.Package) (buildAction, runAction, prin
if len(p.TestGoFiles)+len(p.XTestGoFiles) == 0 {
build := b.CompileAction(work.ModeBuild, work.ModeBuild, p)
run := &work.Action{Mode: "test run", Package: p, Deps: []*work.Action{build}}
addTestVet(b, p, run, nil)
print := &work.Action{Mode: "test print", Func: builderNoTest, Package: p, Deps: []*work.Action{run}}
return build, run, print, nil
}
@@ -681,6 +724,7 @@ func builderTest(b *work.Builder, p *load.Package) (buildAction, runAction, prin
var imports, ximports []*load.Package
var stk load.ImportStack
stk.Push(p.ImportPath + " (test)")
rawTestImports := str.StringList(p.TestImports)
for i, path := range p.TestImports {
p1 := load.LoadImport(path, p.Dir, p, &stk, p.Internal.Build.TestImportPos[path], load.UseVendor)
if p1.Error != nil {
@@ -708,6 +752,7 @@ func builderTest(b *work.Builder, p *load.Package) (buildAction, runAction, prin
stk.Pop()
stk.Push(p.ImportPath + "_test")
pxtestNeedsPtest := false
rawXTestImports := str.StringList(p.XTestImports)
for i, path := range p.XTestImports {
p1 := load.LoadImport(path, p.Dir, p, &stk, p.Internal.Build.XTestImportPos[path], load.UseVendor)
if p1.Error != nil {
@@ -753,8 +798,20 @@ func builderTest(b *work.Builder, p *load.Package) (buildAction, runAction, prin
ptest.GoFiles = append(ptest.GoFiles, p.GoFiles...)
ptest.GoFiles = append(ptest.GoFiles, p.TestGoFiles...)
ptest.Target = ""
ptest.Imports = str.StringList(p.Imports, p.TestImports)
ptest.Internal.Imports = append(append([]*load.Package{}, p.Internal.Imports...), imports...)
// Note: The preparation of the vet config requires that common
// indexes in ptest.Imports, ptest.Internal.Imports, and ptest.Internal.RawImports
// all line up (but RawImports can be shorter than the others).
// That is, for 0 ≤ i < len(RawImports),
// RawImports[i] is the import string in the program text,
// Imports[i] is the expanded import string (vendoring applied or relative path expanded away),
// and Internal.Imports[i] is the corresponding *Package.
// Any implicitly added imports appear in Imports and Internal.Imports
// but not RawImports (because they were not in the source code).
// We insert TestImports, imports, and rawTestImports at the start of
// these lists to preserve the alignment.
ptest.Imports = str.StringList(p.TestImports, p.Imports)
ptest.Internal.Imports = append(imports, p.Internal.Imports...)
ptest.Internal.RawImports = str.StringList(rawTestImports, p.Internal.RawImports)
ptest.Internal.ForceLibrary = true
ptest.Internal.Build = new(build.Package)
*ptest.Internal.Build = *p.Internal.Build
@@ -794,7 +851,8 @@ func builderTest(b *work.Builder, p *load.Package) (buildAction, runAction, prin
Build: &build.Package{
ImportPos: p.Internal.Build.XTestImportPos,
},
Imports: ximports,
Imports: ximports,
RawImports: rawXTestImports,
},
}
if pxtestNeedsPtest {
@@ -942,6 +1000,7 @@ func builderTest(b *work.Builder, p *load.Package) (buildAction, runAction, prin
}
}
buildAction = a
var installAction *work.Action
if testC || testNeedBinary {
// -c or profiling flag: create action to copy binary to ./test.out.
@@ -953,14 +1012,15 @@ func builderTest(b *work.Builder, p *load.Package) (buildAction, runAction, prin
}
}
pmain.Target = target
buildAction = &work.Action{
installAction = &work.Action{
Mode: "test build",
Func: work.BuildInstallFunc,
Deps: []*work.Action{buildAction},
Package: pmain,
Target: target,
}
runAction = buildAction // make sure runAction != nil even if not running test
buildAction = installAction
runAction = installAction // make sure runAction != nil even if not running test
}
if testC {
printAction = &work.Action{Mode: "test print (nop)", Package: p, Deps: []*work.Action{runAction}} // nop
@@ -975,6 +1035,12 @@ func builderTest(b *work.Builder, p *load.Package) (buildAction, runAction, prin
IgnoreFail: true,
TryCache: c.tryCache,
}
if len(ptest.GoFiles)+len(ptest.CgoFiles) > 0 {
addTestVet(b, ptest, runAction, installAction)
}
if pxtest != nil {
addTestVet(b, pxtest, runAction, installAction)
}
cleanAction := &work.Action{
Mode: "test clean",
Func: builderCleanTest,
@@ -993,6 +1059,22 @@ func builderTest(b *work.Builder, p *load.Package) (buildAction, runAction, prin
return buildAction, runAction, printAction, nil
}
func addTestVet(b *work.Builder, p *load.Package, runAction, installAction *work.Action) {
if testVetList == "off" {
return
}
vet := b.VetAction(work.ModeBuild, work.ModeBuild, p)
runAction.Deps = append(runAction.Deps, vet)
// Install will clean the build directory.
// Make sure vet runs first.
// The install ordering in b.VetAction does not apply here
// because we are using a custom installAction (created above).
if installAction != nil {
installAction.Deps = append(installAction.Deps, vet)
}
}
func testImportStack(top string, p *load.Package, target string) []string {
stk := []string{top, p.ImportPath}
Search:
@@ -33,6 +33,7 @@ var testFlagDefn = []*cmdflag.Defn{
{Name: "covermode"},
{Name: "coverpkg"},
{Name: "exec"},
{Name: "vet"},
// Passed to 6.out, adding a "test." prefix to the name if necessary: -v becomes -test.v.
{Name: "bench", PassToTest: true},
@@ -175,6 +176,8 @@ func testFlags(args []string) (packageNames, passToTest []string) {
testCover = true
case "outputdir":
outputDir = value
case "vet":
testVetList = value
}
}
if extraWord {
@@ -193,6 +196,20 @@ func testFlags(args []string) (packageNames, passToTest []string) {
}
}
if testVetList != "" && testVetList != "off" {
if strings.Contains(testVetList, "=") {
base.Fatalf("-vet argument cannot contain equal signs")
}
if strings.Contains(testVetList, " ") {
base.Fatalf("-vet argument is comma-separated list, cannot contain spaces")
}
list := strings.Split(testVetList, ",")
for i, arg := range list {
list[i] = "-" + arg
}
testVetFlags = list
}
if cfg.BuildRace && testCoverMode != "atomic" {
base.Fatalf(`-covermode must be "atomic", not %q, when -race is enabled`, testCoverMode)
}
@@ -12,6 +12,7 @@ func TestRace(t *testing.T) {
}()
x = 3
<-c
_ = x
}
}
@@ -25,5 +26,6 @@ func BenchmarkRace(b *testing.B) {
}()
x = 3
<-c
_ = x
}
}
View
@@ -425,12 +425,19 @@ func doPackage(names []string, basePkg *Package) *Package {
pkg.path = astFiles[0].Name.Name
pkg.files = files
// Type check the package.
err := pkg.check(fs, astFiles)
if err != nil {
// Note that we only report this error when *verbose.
Println(err)
if mustTypecheck {
errorf("%v", err)
errs := pkg.check(fs, astFiles)
if errs != nil {
if *verbose || mustTypecheck {
for _, err := range errs {
fmt.Fprintf(os.Stderr, "%v\n", err)
}
if mustTypecheck {
// This message could be silenced, and we could just exit,
// but it might be helpful at least at first to make clear that the
// above errors are coming from vet and not the compiler
// (they often look like compiler errors, such as "declared but not used").
errorf("typecheck failures")
}
}
}
Oops, something went wrong.

0 comments on commit 0d18875

Please sign in to comment.