testing: 'go test' broken in go1.13

[gbbr]

With go1.13 the go test command broke due to the fix for #21051.

Consider the following program made of two files:

• file.go:

package main

import (
	"flag"
	"fmt"
)

var x string

func init() {
	flag.StringVar(&x, "x", "y", "sets a value for x")
	flag.Parse()
}

func getNumber() int { return 2 }

func main() {
	fmt.Println("Hello world")
}

• file_test.go:

package main

import "testing"

func TestGetNumber(t *testing.T) {
	if getNumber() != 2 {
		t.Fatal("did not get 2")
	}
}

Building the test binary fails to attach the test flags, resulting in an error of the type:

flag provided but not defined: -test.v
Usage of /var/folders/02/23s7r95n6t1739kdh0zc3trr0000gn/T/go-build752851519/b001/go1.13-flagbug.test:
  -x string
    	sets a value for x (default "y")
FAIL	_/Users/gabriel.aszalos/g/go1.13-flagbug	0.005s
FAIL

Consider the output of the below command with go1.13:

$ go test -o ./file.test -c && ./file.test --help
Usage of ./file.test:
  -x string
    	sets a value for x (default "y")

versus go1.12, which display the entire list of flags.

The issue is present because at the time the main package parses the flags, test flags are not yet attached.

[ALTree]

Please search the issue tracker before opening a new one

#31859
#33475
#33774
#33869
#34073
#34078
#34155

Or read the release notes:

Testing flags are now registered in the new Init function, which is invoked by the generated main function for the test. As a result, testing flags are now only registered when running a test binary, and packages that call flag.Parse during package initialization may cause tests to fail.

https://tip.golang.org/doc/go1.13#testing

[gbbr]

Sorry, I missed the release note, and I did search the tracker 😞

[gbbr]

Never the less, this is a terrible regression

[anphy]

terrible

[cespare]

@gbbr the reason we considered this an acceptable change is that calling flags.Parse in an init was always arguably incorrect. There might be other packages that have not yet registered their flags at that point.

I suggest you modify your code to avoid calling flags.Parse inside init.

[gbbr]

@cespare I definitely don't disagree with that. I guess I should've been more specific to say that what I meant by "terrible" was the fact that the output given by the go test command does not make the problem obvious at all and instead throws an apparently unrelated message, but I guess that's being addressed in another issue IIRC.