Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: update documentation to be clear about when parallel subtests run #23368

Open
zwass opened this issue Jan 8, 2018 · 5 comments
Open

testing: update documentation to be clear about when parallel subtests run #23368

zwass opened this issue Jan 8, 2018 · 5 comments

Comments

@zwass
Copy link

@zwass zwass commented Jan 8, 2018

What version of Go are you using (go version)?

go version go1.9.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/zwass/dev/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.9.2/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.9.2/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/4k/2q3ctn7d5_xb03_zsb3v6lgr0000gn/T/go-build252941481=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

go test -v with the following code:

package main

import (
	"fmt"
	"sync"
	"testing"
)

func TestHang(t *testing.T) {
	t.Parallel()
	var wg sync.WaitGroup
	wg.Add(1)
	fmt.Println("added wg")
	t.Run("", func(t *testing.T) {
		fmt.Println("deferred done")
		t.Parallel()
		fmt.Println("set parallel")
		wg.Done()
	})

	fmt.Println("waiting")
	wg.Wait()
	fmt.Println("waiting complete")
}

What did you expect to see?

All statements print and test completes

What did you see instead?

Test hangs after "deferred done" and "waiting" are printed.

This is explained by the following comment in the blog post introducing subtests.

A parallel test never runs concurrently with a sequential test and its execution is suspended until its calling test function, that of the parent test, has returned.

But I cannot find this mentioned anywhere in the documentation for the testing package.

It seems like we at least need to update those docs with the appropriate details.

@bradfitz bradfitz changed the title Update testing documentation to be clear about when parallel subtests run testing: update documentation to be clear about when parallel subtests run Jan 8, 2018
@bradfitz bradfitz added this to the Go1.11 milestone Jan 8, 2018
@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Jan 8, 2018

Note that Go 1.10 has updated wording here:

current: https://golang.org/pkg/testing/#T.Run
go1.10: https://tip.golang.org/pkg/testing/#T.Run

go1.10 says:

Run runs f as a subtest of t called name. It runs f in a separate goroutine and blocks until f returns or calls t.Parallel to become a parallel test. Run reports whether f succeeded (or at least did not fail before calling t.Parallel).

Run may be called simultaneously from multiple goroutines, but all such calls must return before the outer test function for t returns.

That was added in d452986 for #22993.

Is that sufficient?

@zwass
Copy link
Author

@zwass zwass commented Jan 8, 2018

#22993 also bit me, and led to the code in this example that deadlocks.

I don't think the doc update is sufficient. There is still a critical bit of information missing:

its execution is suspended until its calling test function, that of the parent test, has returned.

@RaduBerinde
Copy link
Contributor

@RaduBerinde RaduBerinde commented Apr 24, 2019

Also hit this. There is definitely a lot more detail necessary in the doc for Parallel(). The fact that the test does not proceed until the parent finishes is very important and not at all obvious.

@tommyknows
Copy link

@tommyknows tommyknows commented Jun 4, 2019

Also, what may not be obvious on first look:

func TestMain(t *testing.T) {
    setup()
    defer teardown()
    for name, tt := range testCases {
        t.Run(name, func(t *testing.T) {
            t.Parallel()
            doStuff()
        })
    }
}

doStuff() will NEVER be executed before the defer teardown() kicks in.
Using sync.waitGroup doesn't help with this either, because the subtests only run once the parent has finished.

Is there any possibility to make this "work as expected"?

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 4, 2019

@tommyknows That is best discussed in a forum, not on this issue. Let's not clutter up the issue. Thanks. https://golang.org/wiki/Questions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.