testing: add TB.TempDir() string

[bradfitz]

It's pretty usual to have tests make temp files.

And it's pretty usual to have tests leak temp files. (#23619 tracks detecting it for Go's own tests, as it keeps coming up)

Doing things properly also gets kinda boilerplatey.

I propose adding a method to testing.TB, *testing.T, and *testing.B something like this:

// TempDir returns a temporary directory for the test to use.
// It is lazily created on first access, and calls t.Fatal if the directory
// creation fails.
// Subsequent calls to t.TempDir return the same directory.
// The directory is automatically cleaned up when the test exits.
func (t *T) TempDir() string {
	t.tempDirOnce.Do(func() {
		t.tempDir, t.tempDirErr = ioutil.TempDir("", t.Name())
		if t.tempDirErr == nil {
			t.Cleanup(func() {
				if err := os.RemoveAll(t.tempDir); err != nil {
					t.Errorf("TempDir RemoveAll cleanup: %v", err)
				}
			})
		}
	})
	if t.tempDirErr != nil {
		t.Fatalf("TempDir: %v", err)
	}
	return t.tempDir
}

Is it possible that the artifacts in the temporary directory would be useful in the event of a test failure? In which case, the directory should not be removed. Or should it be expected that any t.Error calls contain all the necessary information to debug the failure?

[cherrymui]

Yeah, the artifacts could be useful for debugging. If there is an explicit os.RemoveAll, I can just comment that out when debugging. With the new function I guess I can do the same by digging into the testing package and commenting out that line, but that is more complicated. Maybe add a new helper that keeps the artifacts that can be used for debugging? t.Keep()? t.NoCleanup?

[bradfitz]

@cherrymui, no need. You can write:

   defer func() { if t.Failed() { os.Exit(1) } }

Or since you know it's already failing:

    defer log.Fatalf("check out the failing files in %v", t.TempDir())

... then no cleanup will happen due to the os.Exit(1) (which log.Fatalf also does).

[zikaeroh]

Why not have a flag which prevents tempdir deletion, something like -test.keeptmp? I'd think that'd be simpler than manually inserting defers (for most uses), and within this proposal the testing framework is already in charge of managing the directory's lifetime.

[cherrymui]

@bradfitz Good to know. Thanks!

[bradfitz]

@zikaeroh, flag/knob fatigue. Too much crap can be overwhelming. Especially for stuff that's rarely needed. "It might be useful sometimes" is not the bar to add stuff. You also have to consider the fatigue costs for more docs and more -help spew.

[dmitshur]

Since I’m not seeing it mentioned, have you considered a more verbose and explicit API variation that returns a cleanup closure?

Somewhat inspired by context.WithCancel, it may look like:

func (*T) TempDir() (dir string, cleanup func())

It would make idiomatic first usage require two extra lines:

func Test(t *testing.T) {
	// ...
	tempDir, cleanup := t.TempDir()
	defer cleanup()
	// ... use tempDir
}

The advantage would be that it’s easier to temporarily modify not to clean up, more readable that there’s no leak to first-time readers, possible to detect when cleanup isn’t called, but at the cost of some additional verbosity. But maybe two lines isn’t bad compared to the 4+ it requires now.

Throwing it out there for consideration, but I’m not convinced this would be better.

[bradfitz]

@dmitshur, if I wanted to write verbose code, I already can today? 🤷‍♂️

[bradfitz]

Also, if TempDir() returns two things, suddenly you need a variable to store it before you can use filepath.Join(). So that's another line.

[dmitshur]

So that's another line.

I think it’s a constant 2 extra lines, I don’t see why a third extra line would be required.

foo(filepath.Join(t.TempDir(), "foo"))
bar(filepath.Join(t.TempDir(), "bar"))

vs

tempDir, cleanup := t.TempDir()
defer cleanup()
foo(filepath.Join(tempDir, "foo"))
bar(filepath.Join(tempDir, "bar"))

[bradfitz]

Okay, constant two, compared to constant zero. I much prefer the foo(filepath.Join(t.TempDir(), "foo")) style with no variables.

[cespare]

I wrote some tempdir helpers that we use for a lot of tests at work. A trick I used (which I took from @tv42) is to detect whether the test is being run under go test and, if it is, locate the tempdir inside the go tool's own temp directory. (In this case the deferred td.Remove() does nothing.) Then if you want to inspect the tempdir, you use -work to print the directory and not delete it.

[bradfitz]

@cespare, @tv42, cute! (maybe a little fragile and thus gross, though?)