unittest warnings counter

[giampaolo]

BPO	18381
Nosy	@pitrou, @giampaolo, @rbtcollins, @ezio-melotti, @voidspace, @berkerpeksag
Files	unittest.patch issue18381.diff issue18381_v2.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/voidspace'
closed_at = None
created_at = <Date 2013-07-06.12:37:39.799>
labels = ['type-feature', 'library']
title = 'unittest warnings counter'
updated_at = <Date 2014-08-13.14:02:20.158>
user = 'https://github.com/giampaolo'

bugs.python.org fields:

activity = <Date 2014-08-13.14:02:20.158>
actor = 'pitrou'
assignee = 'michael.foord'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2013-07-06.12:37:39.799>
creator = 'giampaolo.rodola'
dependencies = []
files = ['30812', '35351', '35487']
hgrepos = []
issue_num = 18381
keywords = ['patch', 'needs review']
message_count = 6.0
messages = ['192430', '194719', '219095', '219730', '219776', '225280']
nosy_count = 6.0
nosy_names = ['pitrou', 'giampaolo.rodola', 'rbcollins', 'ezio.melotti', 'michael.foord', 'berker.peksag']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue18381'
versions = ['Python 3.5']

[giampaolo]

...as per my original proposal on python-ideas:
http://mail.python.org/pipermail/python-ideas/2013-June/021450.html
Patch is in attachment.

[voidspace]

Looks good to me, except the backslash continuation is unneeded. Needs documentation.

[berkerpeksag]

I get a test failure when I run the test suite with unittest.patch:

test_Exit (unittest.test.test_program.Test_TestProgram) ... test test_unittest crashed -- Traceback (most recent call last):
  File "/home/berker/projects/cpython-default/Lib/test/regrtest.py", line 1278, in runtest_inner
    test_runner()
  File "/home/berker/projects/cpython-default/Lib/test/test_unittest.py", line 8, in test_main
    support.run_unittest(unittest.test.suite())
  File "/home/berker/projects/cpython-default/Lib/test/support/__init__.py", line 1764, in run_unittest
    _run_suite(suite)
  File "/home/berker/projects/cpython-default/Lib/test/support/__init__.py", line 1730, in _run_suite
    result = runner.run(suite)
  File "/home/berker/projects/cpython-default/Lib/unittest/runner.py", line 178, in run
    test(result)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/berker/projects/cpython-default/Lib/unittest/case.py", line 647, in __call__
    return self.run(*args, **kwds)
  File "/home/berker/projects/cpython-default/Lib/unittest/case.py", line 597, in run
    testMethod()
  File "/home/berker/projects/cpython-default/Lib/unittest/test/test_program.py", line 119, in test_Exit
    testLoader=self.FooBarLoader())
  File "/home/berker/projects/cpython-default/Lib/unittest/case.py", line 726, in assertRaises
    return context.handle('assertRaises', callableObj, args, kwargs)
  File "/home/berker/projects/cpython-default/Lib/unittest/case.py", line 173, in handle
    callable_obj(*args, **kwargs)
  File "/home/berker/projects/cpython-default/Lib/unittest/main.py", line 93, in __init__
    self.runTests()
  File "/home/berker/projects/cpython-default/Lib/unittest/main.py", line 244, in runTests
    self.result = testRunner.run(self.test)
  File "/home/berker/projects/cpython-default/Lib/unittest/runner.py", line 178, in run
    test(result)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/berker/projects/cpython-default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/berker/projects/cpython-default/Lib/unittest/case.py", line 647, in __call__
    return self.run(*args, **kwds)
  File "/home/berker/projects/cpython-default/Lib/unittest/case.py", line 597, in run
    testMethod()
  File "/home/berker/projects/cpython-default/Lib/unittest/test/test_program.py", line 60, in testFail
    assert False
AssertionError

The new patch (see bpo-18381.diff) fixes that failure. Other changes:

• Added documentation
• Added a test case for addWarning and TestResult.warnings
• Added "print warning" feature

[giampaolo]

FWICT Berker's patch LGTM. Michael, are you OK with committing this?

[berkerpeksag]

Thanks for the review, Giampaolo.

Attached a new patch which fixes text_logging and test_doctest tests.

However, this patch will broke the current behavior of test.support.check_warnings. See Lib/test/test_xml_etree.py for example:

test test_xml_etree crashed -- Traceback (most recent call last):
  File "/home/berker/projects/cpython-default/Lib/test/regrtest.py", line 1278, in runtest_inner
    test_runner()
  File "/home/berker/projects/cpython-default/Lib/test/test_xml_etree.py", line 2580, in test_main
    support.run_unittest(*test_classes)
  File "/home/berker/projects/cpython-default/Lib/test/test_xml_etree.py", line 2537, in __exit__
    self.checkwarnings.__exit__(*args)
  File "/home/berker/projects/cpython-default/Lib/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/home/berker/projects/cpython-default/Lib/test/support/__init__.py", line 1081, in _filterwarnings
    missing[0])
AssertionError: filter ("This search is broken in 1.3 and earlier, and will be fixed in a future version.  If you rely on the current behaviour, change it to '.+'", FutureWarning) did not catch any warning

[pitrou]

Ouch. I've just been bitten by the fact that unittest enables all warnings by default in 3.4, and this change would definitely make things worse. I would like to see a simple, documented way of disabling this new behaviour. I'm proposing a catch_warnings flag on both TestProgram and TestResult (defaulting to True if you like).

(it would be even nicer if this were more flexible, e.g. I want unittest to catch ResourceWarning but not NumbaWarning, but that's icing on the cake)

By the way, perhaps the tests should also exercise the interaction with assertWarns() calls.