Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: reject calls to Run within Cleanup callbacks #48515

Open
bcmills opened this issue Sep 21, 2021 · 2 comments
Open

testing: reject calls to Run within Cleanup callbacks #48515

bcmills opened this issue Sep 21, 2021 · 2 comments

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented Sep 21, 2021

What version of Go are you using (go version)?

go1.17.1

Does this issue reproduce with the latest release?

Yes.

What did you do?

https://play.golang.org/p/l8yW_gkioCx:

func TestCleanupOrder(t *testing.T) {
	t.Cleanup(func() {
		t.Logf("first cleanup")
	})

	t.Cleanup(func() {
		t.Logf("second cleanup")

		t.Run("in cleanup", func(t *testing.T) {
			t.Parallel()
			t.Logf("running parallel subtest")
		})
	})
}

What did you expect to see?

Per https://pkg.go.dev/testing#T.Cleanup:

Cleanup registers a function to be called when the test (or subtest) and all its subtests complete. Cleanup functions will be called in last added, first called order.

So a Cleanup callback should not be started while there are subtests of its parent test still outstanding. Thus, either:

  • the call to t.Run within a Cleanup callback panics the test, or
  • the first-added Cleanup callback is executed after all of the subtests spawned by the second-added Cleanup callback have completed.

That would give an output like:

=== RUN   TestCleanupOrder
    prog.go:11: second cleanup
=== RUN   TestCleanupOrder/in_cleanup
    prog.go:15: running parallel subtest
=== CONT  TestCleanupOrder
    prog.go:7: first cleanup
--- PASS: TestCleanupOrder (0.00s)
    --- PASS: TestCleanupOrder/in_cleanup (0.00s)
PASS

What did you see instead?

The first Cleanup callback is executed before the parallel subtests spawned by the second Cleanup callback are run.

=== RUN   TestCleanupOrder
    prog.go:11: second cleanup
=== RUN   TestCleanupOrder/in_cleanup
=== PAUSE TestCleanupOrder/in_cleanup
=== CONT  TestCleanupOrder
    prog.go:7: first cleanup
=== CONT  TestCleanupOrder/in_cleanup
    prog.go:15: running parallel subtest
--- PASS: TestCleanupOrder (0.00s)
    --- PASS: TestCleanupOrder/in_cleanup (0.00s)
PASS

CC @ianlancetaylor @changkun @rogpeppe @jayconrod

@bcmills bcmills added this to the Backlog milestone Sep 21, 2021
@rsc
Copy link
Contributor

@rsc rsc commented Sep 21, 2021

Don't think too hard about this. Make t.Run panic during t.Cleanup callbacks.

@bcmills bcmills changed the title testing: Cleanup runs in wrong order when a Cleanup function spawns a subtest testing: reject calls to Run within Cleanup callbacks Sep 21, 2021
@gopherbot
Copy link

@gopherbot gopherbot commented Sep 27, 2021

Change https://golang.org/cl/352349 mentions this issue: testing: reject calls to Run within Cleanup callbacks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants