Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: testing: add TB.TempDir() string #35998

Open
bradfitz opened this issue Dec 5, 2019 · 14 comments
Open

proposal: testing: add TB.TempDir() string #35998

bradfitz opened this issue Dec 5, 2019 · 14 comments
Labels
Milestone

Comments

@bradfitz
Copy link
Member

@bradfitz bradfitz commented Dec 5, 2019

It's pretty usual to have tests make temp files.

And it's pretty usual to have tests leak temp files. (#23619 tracks detecting it for Go's own tests, as it keeps coming up)

Doing things properly also gets kinda boilerplatey.

I propose adding a method to testing.TB, *testing.T, and *testing.B something like this:

// TempDir returns a temporary directory for the test to use.
// It is lazily created on first access, and calls t.Fatal if the directory
// creation fails.
// Subsequent calls to t.TempDir return the same directory.
// The directory is automatically cleaned up when the test exits.
func (t *T) TempDir() string {
	t.tempDirOnce.Do(func() {
		t.tempDir, t.tempDirErr = ioutil.TempDir("", t.Name())
		if t.tempDirErr == nil {
			t.Cleanup(func() {
				if err := os.RemoveAll(t.tempDir); err != nil {
					t.Errorf("TempDir RemoveAll cleanup: %v", err)
				}
			})
		}
	})
	if t.tempDirErr != nil {
		t.Fatalf("TempDir: %v", err)
	}
	return t.tempDir
}
@gopherbot gopherbot added this to the Proposal milestone Dec 5, 2019
@gopherbot gopherbot added the Proposal label Dec 5, 2019
@bontibon

This comment has been minimized.

Copy link
Contributor

@bontibon bontibon commented Dec 5, 2019

Is it possible that the artifacts in the temporary directory would be useful in the event of a test failure? In which case, the directory should not be removed. Or should it be expected that any t.Error calls contain all the necessary information to debug the failure?

@cherrymui

This comment has been minimized.

Copy link
Contributor

@cherrymui cherrymui commented Dec 6, 2019

Yeah, the artifacts could be useful for debugging. If there is an explicit os.RemoveAll, I can just comment that out when debugging. With the new function I guess I can do the same by digging into the testing package and commenting out that line, but that is more complicated. Maybe add a new helper that keeps the artifacts that can be used for debugging? t.Keep()? t.NoCleanup?

@bradfitz

This comment has been minimized.

Copy link
Member Author

@bradfitz bradfitz commented Dec 6, 2019

@cherrymui, no need. You can write:

   defer func() { if t.Failed() { os.Exit(1) } }

Or since you know it's already failing:

    defer log.Fatalf("check out the failing files in %v", t.TempDir())

... then no cleanup will happen due to the os.Exit(1) (which log.Fatalf also does).

@zikaeroh

This comment has been minimized.

Copy link

@zikaeroh zikaeroh commented Dec 6, 2019

Why not have a flag which prevents tempdir deletion, something like -test.keeptmp? I'd think that'd be simpler than manually inserting defers (for most uses), and within this proposal the testing framework is already in charge of managing the directory's lifetime.

@cherrymui

This comment has been minimized.

Copy link
Contributor

@cherrymui cherrymui commented Dec 6, 2019

@bradfitz Good to know. Thanks!

@bradfitz

This comment has been minimized.

Copy link
Member Author

@bradfitz bradfitz commented Dec 6, 2019

@zikaeroh, flag/knob fatigue. Too much crap can be overwhelming. Especially for stuff that's rarely needed. "It might be useful sometimes" is not the bar to add stuff. You also have to consider the fatigue costs for more docs and more -help spew.

@dmitshur

This comment has been minimized.

Copy link
Member

@dmitshur dmitshur commented Dec 6, 2019

Since I’m not seeing it mentioned, have you considered a more verbose and explicit API variation that returns a cleanup closure?

Somewhat inspired by context.WithCancel, it may look like:

func (*T) TempDir() (dir string, cleanup func())

It would make idiomatic first usage require two extra lines:

func Test(t *testing.T) {
	// ...
	tempDir, cleanup := t.TempDir()
	defer cleanup()
	// ... use tempDir
}

The advantage would be that it’s easier to temporarily modify not to clean up, more readable that there’s no leak to first-time readers, possible to detect when cleanup isn’t called, but at the cost of some additional verbosity. But maybe two lines isn’t bad compared to the 4+ it requires now.

Throwing it out there for consideration, but I’m not convinced this would be better.

@bradfitz

This comment has been minimized.

Copy link
Member Author

@bradfitz bradfitz commented Dec 6, 2019

@dmitshur, if I wanted to write verbose code, I already can today? 🤷‍♂️

@bradfitz

This comment was marked as resolved.

Copy link
Member Author

@bradfitz bradfitz commented Dec 6, 2019

Also, if TempDir() returns two things, suddenly you need a variable to store it before you can use filepath.Join(). So that's another line.

@dmitshur

This comment was marked as resolved.

Copy link
Member

@dmitshur dmitshur commented Dec 6, 2019

So that's another line.

I think it’s a constant 2 extra lines, I don’t see why a third extra line would be required.

foo(filepath.Join(t.TempDir(), "foo"))
bar(filepath.Join(t.TempDir(), "bar"))

vs

tempDir, cleanup := t.TempDir()
defer cleanup()
foo(filepath.Join(tempDir, "foo"))
bar(filepath.Join(tempDir, "bar"))
@bradfitz

This comment was marked as resolved.

Copy link
Member Author

@bradfitz bradfitz commented Dec 6, 2019

Okay, constant two, compared to constant zero. I much prefer the foo(filepath.Join(t.TempDir(), "foo")) style with no variables.

@cespare

This comment has been minimized.

Copy link
Contributor

@cespare cespare commented Dec 6, 2019

I wrote some tempdir helpers that we use for a lot of tests at work. A trick I used (which I took from @tv42) is to detect whether the test is being run under go test and, if it is, locate the tempdir inside the go tool's own temp directory. (In this case the deferred td.Remove() does nothing.) Then if you want to inspect the tempdir, you use -work to print the directory and not delete it.

@bradfitz

This comment has been minimized.

Copy link
Member Author

@bradfitz bradfitz commented Dec 6, 2019

@cespare, @tv42, cute! (maybe a little fragile and thus gross, though?)

@tv42

This comment has been minimized.

Copy link

@tv42 tv42 commented Dec 7, 2019

Would love to have that possible without hacks like that!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.