Blacklist FunctionTestCase from test discovery

[pitrou]

BPO	22680
Nosy	@warsaw, @pitrou, @rbtcollins, @ezio-melotti, @voidspace, @vadmium
Dependencies	bpo-14534: Add means to mark unittest.TestCases as "do not load".
Files	issue22680.patch: Patch to fix issue 22680.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-08-20.23:30:49.588>
created_at = <Date 2014-10-20.17:14:21.911>
labels = ['easy', 'type-bug', 'library']
title = 'Blacklist FunctionTestCase from test discovery'
updated_at = <Date 2015-08-25.07:37:04.299>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2015-08-25.07:37:04.299>
actor = 'martin.panter'
assignee = 'none'
closed = True
closed_date = <Date 2015-08-20.23:30:49.588>
closer = 'rbcollins'
components = ['Library (Lib)']
creation = <Date 2014-10-20.17:14:21.911>
creator = 'pitrou'
dependencies = ['14534']
files = ['40040']
hgrepos = []
issue_num = 22680
keywords = ['patch', 'easy']
message_count = 17.0
messages = ['229731', '229732', '229733', '229734', '229735', '229737', '230367', '230369', '230370', '230372', '230373', '230374', '230629', '230672', '233275', '247488', '248923']
nosy_count = 8.0
nosy_names = ['barry', 'pitrou', 'rbcollins', 'ezio.melotti', 'Arfrever', 'michael.foord', 'martin.panter', 'Evan Hubinger']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'patch review'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22680'
versions = ['Python 3.5']

[pitrou]

I just got the following traceback when trying discover without 3.5. It runs fine under 3.4...

$ ~/cpython/default/python -m unittest discover -v
Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/antoine/cpython/default/Lib/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/antoine/cpython/default/Lib/unittest/__main__.py", line 18, in <module>
    main(module=None)
  File "/home/antoine/cpython/default/Lib/unittest/main.py", line 93, in __init__
    self.runTests()
  File "/home/antoine/cpython/default/Lib/unittest/main.py", line 244, in runTests
    self.result = testRunner.run(self.test)
  File "/home/antoine/cpython/default/Lib/unittest/runner.py", line 168, in run
    test(result)
  File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 125, in run
    test(result)
  File "/home/antoine/cpython/default/Lib/unittest/case.py", line 625, in __call__
    return self.run(*args, **kwds)
  File "/home/antoine/cpython/default/Lib/unittest/case.py", line 553, in run
    result.startTest(self)
  File "/home/antoine/cpython/default/Lib/unittest/runner.py", line 54, in startTest
    self.stream.write(self.getDescription(test))
  File "/home/antoine/cpython/default/Lib/unittest/runner.py", line 47, in getDescription
    return '\n'.join((str(test), doc_first_line))
  File "/home/antoine/cpython/default/Lib/unittest/case.py", line 1354, in __str__
    self._testFunc.__name__)
AttributeError: 'str' object has no attribute '__name__'

[voidspace]

I assume you mean you get the error *with* 3.5 (not "without"). Does this happen *every time* (i.e. is it trivially reproducible) - or can you provide a repro?

This is regression that I would *assume* (a totally lazy assumption) introduced by the new error handling code.

[pitrou]

I assume you mean you get the error *with* 3.5 (not "without").

Yes, sorry :-)

Does this happen *every time* (i.e. is it trivially reproducible)

It happens with the llvmlite repository: https://github.com/numba/llvmlite
Since using it requires some compiling, I could try to investigate myself if you tell me what to look for.

[voidspace]

As _testFunc is a string and shouldn't be, I'd be very interested to know *what* the string is. That may give us a clue as to where it has come from. (So a try...except...raise that also prints that value would be a good first step.)

[pitrou]

testFunc contains "runTest" (!).

[pitrou]

Ok, apparently it's because I have "from unittest import *" somewhere.

[ezio-melotti]

Does removing "from unittest import *" from somewhere fix the issue?
Is this a bug in your code, or did that import reveal a bug in unittest?

[pitrou]

Does removing "from unittest import *" from somewhere fix the issue?

Yes.

Is this a bug in your code, or did that import reveal a bug in unittest?

Your choice :-)

[pitrou]

btw, in case that's not clear from the traceback, discover was mistaking FunctionTestCase as one of my test classes (while it was just there because of the "import *").

[voidspace]

Ah. Test discovery is discovering FunctionTestCase and attempting to instantiate it as a test. And failing.

Maybe discovery should special case that class and not treat it as a normal TestCase.

[pitrou]

Or perhaps everything that has as a __module__ something which has the attribute __unittest = True?

(the attribute is there for a reason, right?)

[ethanfurman]

Michael Foord opined:
---------------------

Maybe discovery should special case that class and not treat it as a normal TestCase.

Not a bad idea. On the other hand, I don't believe unittest is advertised as supporting 'from ... import *', which *is* usually advised to be a bad idea.

[rbtcollins]

This was reported as https://code.google.com/p/unittest-ext/issues/detail?id=71 a while back. I think blacklisting FunctionTestCase in TestLoader is entirely reasonable.