Repository navigation
testing: reject calls to Run within Cleanup callbacks #48515
Description
What version of Go are you using (go version)?
go1.17.1
Does this issue reproduce with the latest release?
Yes.
What did you do?
https://play.golang.org/p/l8yW_gkioCx:
func TestCleanupOrder(t *testing.T) {
t.Cleanup(func() {
t.Logf("first cleanup")
})
t.Cleanup(func() {
t.Logf("second cleanup")
t.Run("in cleanup", func(t *testing.T) {
t.Parallel()
t.Logf("running parallel subtest")
})
})
}What did you expect to see?
Per https://pkg.go.dev/testing#T.Cleanup:
Cleanup registers a function to be called when the test (or subtest) and all its subtests complete. Cleanup functions will be called in last added, first called order.
So a Cleanup callback should not be started while there are subtests of its parent test still outstanding. Thus, either:
- the call to
t.Runwithin aCleanupcallback panics the test, or - the first-added
Cleanupcallback is executed after all of the subtests spawned by the second-addedCleanupcallback have completed.
That would give an output like:
=== RUN TestCleanupOrder
prog.go:11: second cleanup
=== RUN TestCleanupOrder/in_cleanup
prog.go:15: running parallel subtest
=== CONT TestCleanupOrder
prog.go:7: first cleanup
--- PASS: TestCleanupOrder (0.00s)
--- PASS: TestCleanupOrder/in_cleanup (0.00s)
PASS
What did you see instead?
The first Cleanup callback is executed before the parallel subtests spawned by the second Cleanup callback are run.
=== RUN TestCleanupOrder
prog.go:11: second cleanup
=== RUN TestCleanupOrder/in_cleanup
=== PAUSE TestCleanupOrder/in_cleanup
=== CONT TestCleanupOrder
prog.go:7: first cleanup
=== CONT TestCleanupOrder/in_cleanup
prog.go:15: running parallel subtest
--- PASS: TestCleanupOrder (0.00s)
--- PASS: TestCleanupOrder/in_cleanup (0.00s)
PASS
Activity
rsc commentedon Sep 21, 2021
Don't think too hard about this. Make t.Run panic during t.Cleanup callbacks.
[-]testing: Cleanup runs in wrong order when a Cleanup function spawns a subtest[/-][+]testing: reject calls to Run within Cleanup callbacks[/+]gopherbot commentedon Sep 27, 2021
Change https://golang.org/cl/352349 mentions this issue:
testing: reject calls to Run within Cleanup callbacksgopherbot commentedon Nov 16, 2022
Change https://go.dev/cl/451215 mentions this issue:
testing: in TestRacyOutput, ensure that the Run calls complete before cleanuptesting: in TestRacyOutput, ensure that the Run calls complete before…
nelsam commentedon Mar 9, 2023
FWIW, this broke a BDD-style structured testing library I maintain. It caught me off-guard because it wasn't mentioned in the go 1.20 release notes. As soon as I updated to
1.20all of my tests broke and I've had to fall back to1.19.x.In the library, for the moment I've had to add:
ianlancetaylor commentedon Mar 9, 2023
@nelsam Thanks. I sent https://go.dev/cl/475215 to mention this in the release notes.
gopherbot commentedon Mar 9, 2023
Change https://go.dev/cl/475215 mentions this issue:
doc/go1.20: mention that calling T.Run from T.Cleanup will panicnelsam commentedon Mar 9, 2023
Thanks! The docs weren't very clear if it was supported, so I tried it and it worked. And it felt a lot cleaner than our old
defer o.Run(t)(which wasn't hard to forget). We've been relying ont.Cleanupsince go 1.18.I still don't know how I'm going to fix it, but I'm at least glad that the breaking change is in the release notes now.
doc/go1.20: mention that calling T.Run from T.Cleanup will panic
testing: reject calls to Run within Cleanup callbacks