Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go test -failfast doesn't stop immediately with t.Parallel() #30522

Open
timoha opened this issue Mar 1, 2019 · 4 comments · May be fixed by #53460
Open

cmd/go: go test -failfast doesn't stop immediately with t.Parallel() #30522

timoha opened this issue Mar 1, 2019 · 4 comments · May be fixed by #53460
Labels
NeedsInvestigation Performance
Milestone

Comments

@timoha
Copy link

@timoha timoha commented Mar 1, 2019

What version of Go are you using (go version)?

$ go version
go version go1.10.7 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jenkins/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build781955494=/tmp/go-build -gno-record-gcc-switches"

What did you do?

func TestParallel1(t *testing.T) {
    t.Parallel()
    t.FailNow()
}

func TestParallel2(t *testing.T) {
    t.Parallel()
}

run with go test -v -failfast -parallel 1 returns:

=== RUN   TestParallel1
=== PAUSE TestParallel1
=== RUN   TestParallel2
=== PAUSE TestParallel2
=== CONT  TestParallel1
=== CONT  TestParallel2
--- PASS: TestParallel2 (0.00s)
--- FAIL: TestParallel1 (0.00s)
FAIL
exit status 1
FAIL	test	0.075s

What did you expect to see?

TestParallel2 not continue.

What did you see instead?

TestParallel2 continued execution.

@bcmills bcmills added the NeedsInvestigation label Mar 1, 2019
@bcmills bcmills changed the title go test -failfast doesn't work with t.Parallel() cmd/go: go test -failfast doesn't stop immediately with t.Parallel() Mar 1, 2019
@bcmills
Copy link
Member

@bcmills bcmills commented Mar 1, 2019

CC @jayconrod

@bcmills
Copy link
Member

@bcmills bcmills commented Mar 1, 2019

I doubt we'll get to this in Go 1.13, but I plan to do some work on the testing package for 1.14.

@bcmills bcmills added this to the Unplanned milestone Mar 1, 2019
@gopherbot
Copy link

@gopherbot gopherbot commented Jul 12, 2019

Change https://golang.org/cl/185877 mentions this issue: integration_test.go: use custom parallel test code

gopherbot pushed a commit to protocolbuffers/protobuf-go that referenced this issue Jul 12, 2019
The t.Parallel method does not interact well with the -failfast flag.
See golang/go#30522.

Instead, start off tests ourselves in a goroutine.
Doing it this way allows the -failfast flag to take effect sooner.

One nice benefit of this change is that the test output is cleaner:

Before:
<<<
=== RUN   Test
=== RUN   Test/Go1.9.7
=== RUN   Test/Go1.9.7/TestNormal
=== PAUSE Test/Go1.9.7/TestNormal
=== RUN   Test/Go1.9.7/TestPureGo
=== PAUSE Test/Go1.9.7/TestPureGo
=== RUN   Test/Go1.9.7/TestReflect
=== PAUSE Test/Go1.9.7/TestReflect
=== CONT  Test/Go1.9.7/TestNormal
=== CONT  Test/Go1.9.7/TestReflect
=== CONT  Test/Go1.9.7/TestPureGo
=== RUN   Test/Go1.10.8
=== RUN   Test/Go1.10.8/TestNormal
=== PAUSE Test/Go1.10.8/TestNormal
=== RUN   Test/Go1.10.8/TestPureGo
=== PAUSE Test/Go1.10.8/TestPureGo
=== RUN   Test/Go1.10.8/TestReflect
=== PAUSE Test/Go1.10.8/TestReflect
=== CONT  Test/Go1.10.8/TestNormal
=== CONT  Test/Go1.10.8/TestReflect
=== CONT  Test/Go1.10.8/TestPureGo
=== RUN   Test/Go1.11.6
=== RUN   Test/Go1.11.6/TestNormal
=== PAUSE Test/Go1.11.6/TestNormal
=== RUN   Test/Go1.11.6/TestPureGo
=== PAUSE Test/Go1.11.6/TestPureGo
=== RUN   Test/Go1.11.6/TestReflect
=== PAUSE Test/Go1.11.6/TestReflect
=== CONT  Test/Go1.11.6/TestNormal
=== CONT  Test/Go1.11.6/TestReflect
=== CONT  Test/Go1.11.6/TestPureGo
=== RUN   Test/Go1.12.1
=== RUN   Test/Go1.12.1/TestNormal
=== PAUSE Test/Go1.12.1/TestNormal
=== RUN   Test/Go1.12.1/TestPureGo
=== PAUSE Test/Go1.12.1/TestPureGo
=== RUN   Test/Go1.12.1/TestReflect
=== PAUSE Test/Go1.12.1/TestReflect
=== RUN   Test/Go1.12.1/TestProto1Legacy
=== PAUSE Test/Go1.12.1/TestProto1Legacy
=== RUN   Test/Go1.12.1/TestProtocGenGo
=== PAUSE Test/Go1.12.1/TestProtocGenGo
=== RUN   Test/Go1.12.1/TestProtocGenGoGRPC
=== PAUSE Test/Go1.12.1/TestProtocGenGoGRPC
=== CONT  Test/Go1.12.1/TestNormal
=== CONT  Test/Go1.12.1/TestProto1Legacy
=== CONT  Test/Go1.12.1/TestProtocGenGoGRPC
=== CONT  Test/Go1.12.1/TestProtocGenGo
=== CONT  Test/Go1.12.1/TestReflect
=== CONT  Test/Go1.12.1/TestPureGo
=== RUN   Test/ConformanceTests
=== RUN   Test/GeneratedGoFiles
=== RUN   Test/FormattedGoFiles
=== RUN   Test/CommittedGitChanges
=== RUN   Test/TrackedGitFiles
--- PASS: Test (509.59s)
    --- PASS: Test/Go1.9.7 (0.00s)
        --- PASS: Test/Go1.9.7/TestReflect (100.81s)
        --- PASS: Test/Go1.9.7/TestNormal (100.88s)
        --- PASS: Test/Go1.9.7/TestPureGo (56.93s)
    --- PASS: Test/Go1.10.8 (0.00s)
        --- PASS: Test/Go1.10.8/TestNormal (64.85s)
        --- PASS: Test/Go1.10.8/TestReflect (65.20s)
        --- PASS: Test/Go1.10.8/TestPureGo (36.14s)
    --- PASS: Test/Go1.11.6 (0.00s)
        --- PASS: Test/Go1.11.6/TestReflect (59.69s)
        --- PASS: Test/Go1.11.6/TestNormal (60.25s)
        --- PASS: Test/Go1.11.6/TestPureGo (34.17s)
    --- PASS: Test/Go1.12.1 (0.00s)
        --- PASS: Test/Go1.12.1/TestProto1Legacy (59.41s)
        --- PASS: Test/Go1.12.1/TestNormal (59.87s)
        --- PASS: Test/Go1.12.1/TestProtocGenGo (3.55s)
        --- PASS: Test/Go1.12.1/TestProtocGenGoGRPC (4.12s)
        --- PASS: Test/Go1.12.1/TestReflect (60.10s)
        --- PASS: Test/Go1.12.1/TestPureGo (60.66s)
    --- PASS: Test/ConformanceTests (0.78s)
    --- PASS: Test/GeneratedGoFiles (8.99s)
    --- PASS: Test/FormattedGoFiles (2.64s)
    --- PASS: Test/CommittedGitChanges (0.10s)
    --- PASS: Test/TrackedGitFiles (0.00s)
PASS
>>>

After:
<<<
=== RUN   Test
=== RUN   Test/Go1.9.7/PureGo
=== RUN   Test/Go1.9.7/Normal
=== RUN   Test/Go1.9.7/Reflect
=== RUN   Test/Go1.10.8/Normal
=== RUN   Test/Go1.10.8/PureGo
=== RUN   Test/Go1.10.8/Reflect
=== RUN   Test/Go1.11.6/Normal
=== RUN   Test/Go1.11.6/PureGo
=== RUN   Test/Go1.11.6/Reflect
=== RUN   Test/Go1.12.1/Normal
=== RUN   Test/Go1.12.1/PureGo
=== RUN   Test/Go1.12.1/Reflect
=== RUN   Test/Go1.12.1/Proto1Legacy
=== RUN   Test/Go1.12.1/ProtocGenGo
=== RUN   Test/Go1.12.1/ProtocGenGoGRPC
=== RUN   Test/ConformanceTests
=== RUN   Test/GeneratedGoFiles
=== RUN   Test/FormattedGoFiles
=== RUN   Test/CommittedGitChanges
--- PASS: Test (182.87s)
    --- PASS: Test/Go1.9.7/PureGo (72.17s)
    --- PASS: Test/Go1.9.7/Normal (72.59s)
    --- PASS: Test/Go1.10.8/Normal (8.03s)
    --- PASS: Test/Go1.10.8/PureGo (8.03s)
    --- PASS: Test/Go1.10.8/Reflect (8.34s)
    --- PASS: Test/Go1.11.6/Normal (10.70s)
    --- PASS: Test/Go1.11.6/PureGo (8.21s)
    --- PASS: Test/Go1.11.6/Reflect (8.77s)
    --- PASS: Test/Go1.12.1/Normal (7.41s)
    --- PASS: Test/Go1.12.1/PureGo (6.52s)
    --- PASS: Test/Go1.12.1/Reflect (6.04s)
    --- PASS: Test/Go1.12.1/Proto1Legacy (6.96s)
    --- PASS: Test/Go1.12.1/ProtocGenGo (1.15s)
    --- PASS: Test/Go1.12.1/ProtocGenGoGRPC (2.43s)
    --- PASS: Test/Go1.9.7/Reflect (88.47s)
    --- PASS: Test/ConformanceTests (1.81s)
    --- PASS: Test/GeneratedGoFiles (16.28s)
    --- PASS: Test/FormattedGoFiles (3.17s)
    --- PASS: Test/CommittedGitChanges (0.17s)
PASS
>>>

Change-Id: I777ffe3f4d5f6407c87e3866cbaa890017204cba
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185877
Reviewed-by: Herbie Ong <herbie@google.com>
rubensayshi added a commit to rubensayshi/go that referenced this issue Jun 19, 2022
The t.Parallel method does not interact well with the -failfast flag.
See golang#30522.

With -failfast is enabled tests blocking on t.Parallel() can be skipped
after the first test failure.
This won't immediately kill any parallel tests which are already running
but it will prevent more tests from starting.
rubensayshi added a commit to rubensayshi/go that referenced this issue Jun 20, 2022
The t.Parallel method does not interact well with the -failfast flag.
See golang#30522.

With -failfast is enabled tests blocking on t.Parallel() can be skipped
after the first test failure.
This won't immediately kill any parallel tests which are already running
but it will prevent more tests from starting.
rubensayshi added a commit to rubensayshi/go that referenced this issue Jun 20, 2022
The t.Parallel method does not interact well with the -failfast flag.
See golang#30522.

With -failfast is enabled tests blocking on t.Parallel() can be skipped
after the first test failure.
This won't immediately kill any parallel tests which are already running
but it will prevent more tests from starting.
rubensayshi added a commit to rubensayshi/go that referenced this issue Jun 20, 2022
The t.Parallel method does not interact well with the -failfast flag.
See golang#30522.

With -failfast enabled tests blocking on t.Parallel() can be skipped
after the first test failure.
This won't immediately kill any parallel tests which are already running
but it will prevent more tests from starting.
rubensayshi added a commit to rubensayshi/go that referenced this issue Jun 20, 2022
The t.Parallel method does not interact well with the -failfast flag.
See golang#30522.

With -failfast enabled tests blocking on t.Parallel() can be skipped
after the first test failure.
This won't immediately kill any parallel tests which are already running
but it will prevent more tests from starting.
@gopherbot
Copy link

@gopherbot gopherbot commented Jun 20, 2022

Change https://go.dev/cl/413236 mentions this issue: testing: skip parallel tests when -failfast is enabled

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Performance
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants