testing: Short() does not work in TestMain()

[umisama]

The testing.Short() have to report -test.short flag is set. But, it looks not working on TestMain(*testing.M).

You can see this behavior on this example.
https://gist.github.com/umisama/cebb1c8f9022e5402d21

Here is output on my environment.

$ go version
go version go1.4 linux/amd64
$ go test -short
false   on TestMain
true    on TestSomething
PASS
ok      _/home/umisama/testingshort  0.008s

In the example I expected to testing.Short() returns true for TestXxx() including TestMain(*testing.M), but it returns false on TestMain(*testing.M).

I'll create patch if this behavior is unexpected.

[minux]

TestMain is executed before flag.Parse in (*testing.M).Run.

I think this is intentional, because it provides a way to modify the
command line arguments. I you want testing.Short(), then you
should call flag.Parse() manually in TestMain.

In conclusion, I think a docs clarification is needed, it's not a bug.

[hraban]

Please include that testing.Verbose() also doesn't work in TestMain. Came here to post an issue about it, but now I see it's by design.

[gravis]

I really would like to see changed as well.
I was planning to use testMain as:

func TestMain(m *testing.M) {
    // Reset test db before running test.
    // The schema is only imported once for all test
    if !testing.Short() {
        fmt.Println("Initializing test DB for integration test (disable with `go test -short`)")
        initTestDB()
    }
    os.Exit(m.Run())
}

And initTestDB() is always run :(

[minux]

I don't think this is going to change.

If you want to use testing.Verbose or testing.Short, call flag.Parse in
TestMain before using them.

[gravis]

Ok, thanks for the info

[minux]

I've sent https://golang.org/cl/7604 to add the docs.

[gravis]

👍 👍