testing: parallel starts tests even when -parallel=1

[broady]

package main

import (
    "testing"
    "time"
)

func TestOne(t *testing.T) {
    tt(t)
}

func TestTwo(t *testing.T) {
    tt(t)
}

func TestThree(t *testing.T) {
    tt(t)
}

func tt(t *testing.T) {
    t.Parallel()
    time.Sleep(time.Second)
}

$ go test -timeout=1s -v
=== RUN   TestOne
=== RUN   TestTwo
=== RUN   TestThree
--- PASS: TestThree (1.00s)
--- PASS: TestTwo (1.00s)
--- PASS: TestOne (1.00s)
PASS
ok      github.com/broady/testtimeout   1.010s

$ go test -timeout=1s -parallel=1 -v
=== RUN   TestOne
=== RUN   TestTwo
=== RUN   TestThree
panic: test timed out after 1s
<snip>
exit status 2
FAIL    github.com/broady/testtimeout   1.016s

It looks like the test runner "starts" the timer for the tests even though the test code doesn't start executing. This results in a faulty test timeout.

go version devel +edb19aa Fri Mar 25 18:35:15 2016 +0000 linux/amd64

[bradfitz]

@robpike, @mpvl?

[robpike]

Looks like a bug to me.

[mpvl]

I touched this code last so I'll take it.

[mpvl]

This seems like intended behavior to me. Note that the timeout is defined for the aggregate time of all tests:

"if positive, sets an aggregate time limit for all tests".

In the "go test -timeout=1s -v" run, all your parallel tests get run in parallel and are able to complete within the given timeout (by chance, I guess).

In the "go test -timeout=1s -parallel=1 -v" run, as the parallel test are run in sequence, it is guaranteed to panic. On my install, I see a PASS for the first test (both for an old version of go and the one including my changes). But I guess it depends on how lucky you are, as the timeout is so close to the running time.

[mpvl]

working as intended.