cleanUp stack for unittest

[voidspace]

BPO	5679
Nosy	@gpshead, @pitrou, @rbtcollins, @ezio-melotti, @ngie-eign, @voidspace
Files	unittest-cleanup.diff unittest-no-exit.patch unittest-cleanup.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/voidspace'
closed_at = <Date 2009-05-02.20:22:01.424>
created_at = <Date 2009-04-03.19:56:46.283>
labels = ['type-feature', 'library']
title = 'cleanUp stack for unittest'
updated_at = <Date 2011-03-09.19:32:54.415>
user = 'https://github.com/voidspace'

bugs.python.org fields:

activity = <Date 2011-03-09.19:32:54.415>
actor = 'ezio.melotti'
assignee = 'michael.foord'
closed = True
closed_date = <Date 2009-05-02.20:22:01.424>
closer = 'michael.foord'
components = ['Library (Lib)']
creation = <Date 2009-04-03.19:56:46.283>
creator = 'michael.foord'
dependencies = []
files = ['13612', '13787', '13791']
hgrepos = []
issue_num = 5679
keywords = ['patch']
message_count = 45.0
messages = ['85321', '85323', '85324', '85325', '85344', '85404', '85407', '85413', '85432', '85434', '85437', '85439', '85443', '85444', '85445', '85446', '85447', '85463', '85465', '85466', '85471', '85494', '85497', '85562', '85565', '85567', '85568', '85569', '85570', '85572', '85589', '85590', '85592', '85593', '85594', '85609', '85612', '85613', '86568', '86569', '86576', '86607', '86978', '86998', '130459']
nosy_count = 9.0
nosy_names = ['gregory.p.smith', 'pitrou', 'rbcollins', 'vdupras', 'jml', 'ezio.melotti', 'kumar303', 'ngie', 'michael.foord']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue5679'
versions = ['Python 2.7']

[voidspace]

Proposal to add a cleanUp stack to unittest.TestCase. This is a list of
callables to be called (LIFO) to cleanup resources. If there are items
on the stack it should be called even if setUp fails. Otherwise it
should be called after tearDown.

Similar functionality is in the testing frameworks used by bzr, Twisted,
and Zope.

I will create a patch similar to the bzr implementation.

The public API is via a new method:

    def addCleanUpItem(self, callable, *args, **kwargs):

Usage is:

    self.db = self.createDB()
    self.addCleanUpItem(self.db.close)

[gpshead]

I'm used to doing this using a finally clause in the setUp() method (if
I do it at all, I'm not used to setUp failing).

I guess this would be a convenience to avoid the need for this pattern?
I'm not sure we really need a list of cleanup callbacks. Got pointers
to good examples of this in bzr/twisted/zope?

def setUp(self):
  try:
    do_a_bunch_of_stuff_that_could_fail()
  finally:
    conditionally_undo_setup()
  do_more()

def conditionally_undo_setup(self):
  if self.foo: 
    self.foo.close()
  if self.bar:
    shutil.rmtree(self.bar)

def tearDown(self):
  conditionally_undo_setup()
  undo_more()

fwiw, in your self.db.close example, normally i'd also want to remove
and unlink the database as well as just closing it for a unittest.
maybe not the best example.

[voidspace]

This is a nice (simple to use and understand) pattern for resource
allocation / deallocation.

Supporting the cleaning up of resources when setUp fails (without
duplicating clean up code) is just one use case. (I agree setUp failure
is unusual.)

It provides a clean way to clean up just the resources you have
allocated in the event of test failure, without having to keep track
yourself of how far the test has got. Manually tracking resource usage
is easy to get wrong.

bzr:

http://bazaar.launchpad.net/~jml/testtools/trunk/annotate/head%3A/testtools/testcase.py

And from zope.testing:

The documentation:
http://svn.zope.org/zope.testing/trunk/src/zope/testing/setupstack.txt?rev=92340&view=auto

The module:
http://svn.zope.org/zope.testing/trunk/src/zope/testing/setupstack.py?rev=92340&view=auto

[voidspace]

From your example this would completely remove the need for the
conditional checking in the cleanup code. Only resources actually
allocated would be in the stack.

[voidspace]

And actually your conditionally_undo_setup() call in setUp is incorrect.
It should in an except block that re-raises rather than a finally block
(which will do the cleanup even in non-exceptional circumstances). Easy
code to get wrong...

[voidspace]

Patch attached.

No docs, if it is agreed I can apply I'll write docs.

After a long discussion we arrived at some semblance of consensus on the
Testing In Python mailing list that this was a good thing (tm).

Only one -1 (thought that cleanUp should be a method) and Holger Krekel
was only +0 because he thinks changes in unittest should be conservative
(but he isn't actually using it).

Many others were +1 - Fred Drake, Titus Brown, Laura Creighton, Robert
Collins, Jonathan Lange, Kumar McMillan and others.

It provides a clean and simple pattern for the cleaning up of resources

• avoiding the need for nested try finallys and the manual tracking of
  resources. It can also avoid the need for duplication of cleanup code.

[pitrou]

I'm not sure why it is called after tearDown. It would be better to call
it before tearDown, so that test cases can also use it to initialize
additional resources and finalize them in a LIFO way wrt the main tearDown.

[voidspace]

I'm agnostic on before / after tearDown, so happy to make that change.

[rbtcollins]

It should run after tearDown so that teardown can do actions that may
require cleanup; because the cleanups run in LIFO you can acquire
resources in setUp and have cleanups clean them up,

[rbtcollins]

Actually let me phrase that differently.
standard practice for setUp is
super.setUp()
my_setup_code()

and tearDown is
my_teardown_code()
super.tearDown()

because of the LIFO need.

If you imagine that clean ups are being done in the base class teardown,
at the end of that method, then it becomes a lot more obvious why it
should run after tearDown - because thats the right place, and because
many users may be failing to call the base class tearDown which has
historically done nothing.