Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: testing: add Defer method #32111

Open
rogpeppe opened this issue May 17, 2019 · 7 comments

Comments

Projects
None yet
6 participants
@rogpeppe
Copy link
Contributor

commented May 17, 2019

It's a common requirement to need to clean up resources at the end of a test - removing temporary directories, closing file servers, etc.

The standard Go idiom is to return a Closer or a cleanup function and defer a call to that, but in a testing context, that can be tedious and add cognitive overhead to reading otherwise simple test code. It is useful to be able to write a method that returns some domain object that can be used directly without worrying the caller about the resources that needed to be created (and later destroyed) in order to provide it.

Some test frameworks allow tear-down methods to be defined on "suite" types, but this does not compose well and doesn't feel Go-like.

Instead, I propose that we add a Defer method to the testing.B, testing.T and testing.TB types that will register a function to be called at the end of the test.

The implementation could be something like this:

type T struct {
	// etc
	mu sync.Mutex
	deferred func()
}

// Defer registers a function to be called at the end of the test.
// Deferred functions will be called in last added, first called order.
func (t *T) Defer(f func()) {
	t.mu.Lock()
	defer t.mu.Unlock()
	oldDeferred := t.deferred
	t.deferred = func() {
		defer oldDeferred()
		f()
	}
}

// done calls all the functions registered by Defer in reverse
// registration order.
func (t *T) done() {
	t.mu.Lock()
	deferred := t.deferred
	t.mu.Unlock()

	deferred()
}

The quicktest package uses this approach and it seems to work well.

@gopherbot gopherbot added this to the Proposal milestone May 17, 2019

@gopherbot gopherbot added the Proposal label May 17, 2019

@natefinch

This comment has been minimized.

Copy link
Contributor

commented May 17, 2019

I would use the heck out of this in my tests.

@ChrisHines

This comment has been minimized.

Copy link
Contributor

commented May 17, 2019

Can you provide an example of when this is better than a basic defer?

@natefinch

This comment has been minimized.

Copy link
Contributor

commented May 17, 2019

Because it means you can do the defer from inside a helper function.

So you get this:

func TestSomething(t *testing.T) {
    dir := mkDir(t)
    // use dir
}

func mkDir(t *testing.T) string {
    name, err := ioutil.TempDir("", "")
    if err != nil {
        t.Fatal(err)
    }
    t.Defer(func() {
        err := os.RemoveAll(name)
        if err != nil {
            t.Error(err)
	})
    return name
}

Without Defer, you have to return a cleanup function from mkDir that you then defer:

func TestSomething(t *testing.T) {
    dir, cleanup := mkDir(t)
    defer cleanup()
    // use dir
}

func mkDir(t *testing.T) (string, func()) {
    name, err := ioutil.TempDir("", "")
    if err != nil {
        t.Fatal(err)
    }
    return name, func() {
        err := os.RemoveAll(name)
        if err != nil {
            t.Error(err)
	})
}

If it's just one thing, like a single directory you're creating, it's not that bad. But when it's a whole bunch of things, it adds a lot of noise for little benefit.

@cespare

This comment has been minimized.

Copy link
Contributor

commented May 17, 2019

If there are many things, then you can create a test helper wrapping type that initializes them all for you at once and the close method can clean up/close/delete all of them.

This proposal adds an API that explicitly overlaps with a language feature, making tests look yet a bit more different from normal code. There is a nice symmetry (common in test and non-test code) where the creation and deferred cleanup of a thing are paired:

t := newThing()
defer t.cleanup()

(think files or mutexes); where it exists, this pattern is clear and obvious, and t.Defer breaks it by putting the cleanup elsewhere.

In the end, this is all to save a single line of code per test, which feels like an un-Go-like tradeoff to me. So initially I'm mildly opposed to this feature.

@rogpeppe

This comment has been minimized.

Copy link
Contributor Author

commented May 18, 2019

In the end, this is all to save a single line of code per test, which feels like an un-Go-like tradeoff to me. So initially I'm mildly opposed to this feature.

I understand where you're coming from, so let me try to explain one use case with reference to some existing tests that use this feature.

Firstly, tests already look different from normal code. In tests, unlike production code, we can concern ourselves with happy path only. There's always an easy solution when things don't happen the way they should - just abort the test. That's why we have T.Fatal in the first place. It makes it possible to have a straightforward sequence of operations with no significant logic associated with branches.

When testing more complex packages, it becomes useful to make test helper functions that set up or return one or more dependencies for the test. Sometimes helper functions run several levels deep, with a high level function calling various low level ones, all of which can call T.Fatal.

Here's an test helper function from some real test code:

https://github.com/juju/charmstore-client/blob/799c23e4ad134e4115837060a3f7831401a9097e/internal/ingest/testcs_test.go#L40-L76

It's building a *testCharmstore instance, but to do that, it has to put together various dependencies, any of which might fail. We want to guarantee that things are cleaned up even if a later dependency cannot be acquired. For example, if the call to charmstore.NewServer fails, we'd like to close the database connection and cs.discharger (another server instance).

Without the Defer method, that's not that easy to do. Even if you define a Close method on the returned *testCharmstore type, you'd have to write some code to clean up the earlier values if the later ones fail. Of course, that's entirely possible to do, as we do in production code.

However, given Defer, we have an easy option: we can write straight-line initialization code, failing hard if anything goes wrong, but still clean up properly if it does.

@beoran

This comment has been minimized.

Copy link

commented May 20, 2019

A simple way to solve this problem is to use a helper that opens and closes the resource and that takes the real test as a closure function parameter. This is a common idiom in Ruby, which I also use often in Go. Something like this:

func TestSomething(t *testing.T) {
    dir := withResourceHelper(t, func (r Resource) {
       // perform test in here, can use resource
    })
}

func withResourceHelper(t *testing.T, testFunc func(r Resource)) {
    resource, err := NewResource()
    if err != nil {
        t.Fatal(err)
    } else {
      defer resource.Close()
      testFunc(resource)
   }
}
@rogpeppe

This comment has been minimized.

Copy link
Contributor Author

commented May 20, 2019

A simple way to solve this problem is to use a helper that opens and closes the resource and that takes the real test as a closure function parameter.

Yes, that is a possible way of working around this, but as a personal preference in Go, I prefer to avoid this callback-oriented style of programming - it leads to more levels of indentation than I like, and I don't find it as clear. Rather than just asking for a resource to use, you have to give up your flow of control to another function which makes this style less orthogonal to other use cases. For example, callback style doesn't play well with goroutines - if you wanted to create two such resources concurrently, you'd need to break the implied invariant that you shouldn't use the resource outside of the passed function. It also means that if you do want to add some cleanup to an existing test helper, significant refactoring can be required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.