testing: surprising interaction of subtests with TempDir #46624
Description
What version of Go are you using (go version)?
I tested with Go 1.16.4 and tip (e1fa260).
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
linux/amd64
What did you do?
Here's a demo (playground):
package main
import (
"fmt"
"path/filepath"
"testing"
)
func TestTempdir(t *testing.T) {
testGlob(t)
t.Run("x=[1,2,3]", func(t *testing.T) {
testGlob(t)
})
t.Run("x=[]", func(t *testing.T) {
testGlob(t)
})
}
func testGlob(t *testing.T) {
dir := t.TempDir()
glob := filepath.Join(dir, "*.txt")
fmt.Println("glob:", glob)
if _, err := filepath.Glob(glob); err != nil {
t.Fatal("glob error:", err)
}
}
What did you expect to see?
It'd be nice if this just worked.
What did you see instead?
$ go test -v ./x
=== RUN TestTempdir
glob: /tmp/TestTempdir1889981746/001/*.txt
=== RUN TestTempdir/x=[1,2,3]
glob: /tmp/TestTempdir_x=[1,2,3]163097497/001/*.txt
=== RUN TestTempdir/x=[]
glob: /tmp/TestTempdir_x=[]3868477592/001/*.txt
x_test.go:24: glob error: syntax error in pattern
--- FAIL: TestTempdir (0.00s)
--- PASS: TestTempdir/x=[1,2,3] (0.00s)
--- FAIL: TestTempdir/x=[] (0.00s)
FAIL
When naming some subtest, you probably aren't considering that the name is going to become part of the path of any t.TempDir(). This can have some surprising interactions, such as this case where the name includes [], making any glob pattern inside the tempdir invalid.
I suspect there are other lurking surprises. For example, if you use a * inside the test name, that is passed on to os.MkdirTemp as-is where it has a special meaning (it indicates where the random number should be inserted). Not broken, but also not intended. Or your subtest name can include characters such as !"#$&'(),;<=>?[]^{|} and backtick which may require shell quoting.
Perhaps t.TempDir shouldn't name its directory after the name of the test, or else perhaps it should use a limited set of allowed symbols rather than trying to remove a few disallowed ones (like /).
(This is a continuation of the issues uncovered in #38465 -- cc @bradfitz @tklauser)