Fail-fast behavior for unittest

[jcd]

BPO	8074
Nosy	@amauryfa, @voidspace

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/voidspace'
closed_at = <Date 2010-03-27.17:37:33.282>
created_at = <Date 2010-03-05.17:32:22.708>
labels = ['type-feature', 'library']
title = 'Fail-fast behavior for unittest'
updated_at = <Date 2010-03-27.17:37:33.280>
user = 'https://bugs.python.org/jcd'

bugs.python.org fields:

activity = <Date 2010-03-27.17:37:33.280>
actor = 'michael.foord'
assignee = 'michael.foord'
closed = True
closed_date = <Date 2010-03-27.17:37:33.282>
closer = 'michael.foord'
components = ['Library (Lib)']
creation = <Date 2010-03-05.17:32:22.708>
creator = 'jcd'
dependencies = []
files = []
hgrepos = []
issue_num = 8074
keywords = []
message_count = 6.0
messages = ['100493', '100494', '100792', '100799', '101473', '101832']
nosy_count = 3.0
nosy_names = ['amaury.forgeotdarc', 'jcd', 'michael.foord']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue8074'
versions = ['Python 2.7', 'Python 3.2']

[jcd]

When a long suite of tests is running, it would be nice to be able to exit out early to see the results of the tests that have run so far.

I would like to see a couple of features included in unittest to this effect that have recently been implemented in django's test runner. The first exits gracefully on ^C, the second allows the user to specify a --failfast option, which causes the test runner to exit after the first failure or error, rather than proceeding with the

If you hit ^C, rather than exit with a stack trace of whatever was happening at the time, unittest should complete the current test, and then exit with the results to that point. A second ^C would be treated as a normal KeyboardInterrupt, in case that last test were somehow running on too long.

{{{
$ python test_module.py
..F.s.^C <Test run halted by Ctrl-C> x
======================================================================
FAIL: test_multiline_equal (main.MyTest)
Unicode objects return diffs on failure
----------------------------------------------------------------------

Traceback (most recent call last):
  File "test_test.py", line 39, in test_multiline_equal
    self.assertEqual(original, revised)
AssertionError: 
- Lorem ipsum dolor sit amet, 
?             ^
+ Lorem ipsum color sit amet, 
?             ^
              consectetur adipisicing elit, 
+             that's the way the cookie crumbles,
              sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

---

Ran 7 tests in 12.010s

FAILED (failures=1, skipped=1, expected failures=1)
}}}

A failfast option could be specified as a keyword argument to unittest.main(), or unittest.main() could read it off the command line, according to the same mechanism used by, e.g., the verbosity option.

[jcd]

This feature was implemented in the django test runner as r12034 (^C override) and r11843 (failfast option)

[amauryfa]

fwiw, the similar bpo-2241 was rejected two years ago.

[voidspace]

I like the feature. I'm a bit concerned about the proliferation of command line options for unittest though.

Steve's rejection of the feature seemed to be on the basis that he didn't need it (or see a need for it). :-)

[voidspace]

Failfast option added, committed revision 79265. Still needs documentation. Ctrl-C handling not yet done. Slightly more complex as it needs to play well with tests that test SIGINT handling.

[voidspace]

Ctrl-C handling now implemented (-c command line option). Still needs documentation, but there is a bunch of this outstanding and I will do it in one go and rework the unittest docs at the same time.