unittest: if FunctionTestCase is imported, the loader loads "tests" from it

[abetkin]

BPO	40690
Nosy	@cjerdonek, @remilapeyre, @abetkin
PRs	bpo-40690: Check that testcase.__module__ != 'unittest.case' #20237
Files	test_functiontest.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-05-19.20:17:05.761>
labels = ['3.8', 'type-bug', 'tests', '3.9', '3.10']
title = 'unittest: if FunctionTestCase is imported, the loader loads "tests" from it'
updated_at = <Date 2020-05-27.21:04:36.962>
user = 'https://github.com/abetkin'

bugs.python.org fields:

activity = <Date 2020-05-27.21:04:36.962>
actor = 'remi.lapeyre'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests']
creation = <Date 2020-05-19.20:17:05.761>
creator = 'pwtail'
dependencies = []
files = ['49196']
hgrepos = []
issue_num = 40690
keywords = ['patch']
message_count = 6.0
messages = ['369386', '369524', '369547', '369571', '369575', '370135']
nosy_count = 3.0
nosy_names = ['chris.jerdonek', 'remi.lapeyre', 'pwtail']
pr_nums = ['20237']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40690'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

Linked PRs

• gh-84867: Do not load tests from TestCase and FunctionTestCase #100497
• [3.12] gh-84867: Do not load tests from TestCase and FunctionTestCase (GH-100497) #109327
• [3.11] gh-84867: Do not load tests from TestCase and FunctionTestCase (GH-100497) #109328

[abetkin]

In [2]: from unittest import FunctionTestCase

In [3]: loader.loadTestsFromTestCase(FunctionTestCase)
Out[3]: <unittest.suite.TestSuite tests=[<unittest.case.FunctionTestCase tec=runTest>]>

In [4]: test = _._tests[0]; test
Out[4]: <unittest.case.FunctionTestCase tec=runTest>

In [5]: test._testFunc
Out[5]: 'runTest'

In [6]: test._testFunc.__name__
*** AttributeError: 'str' object has no attribute '__name__'

[remilapeyre]

Hi Vitalii, can you give more context about why you are writing this code, what you expect it to do and why is there a bug?

[abetkin]

If you make an import in a module with your tests

From unittest import FunctionTestCase

then you will have 1 extra test in that module (the imported one), moreover, that test will be broken.

[cjerdonek]

then you will have 1 extra test in that module (the imported one), moreover, that test will be broken.

If this is true, then how is anyone able to be using FunctionTestCase in their tests today? Is the feature broken?

[abetkin]

I don't think anyone is using FunctionTestCase

[remilapeyre]

I checked and FunctionTestCase seems to completely break the loader. The tests for FunctionTestCase in the standard library instantiate the class from inside the method of a TestCase so the loader never see them but even the simple test file I attached completely breaks:

✗ python3 -m unittest
E
======================================================================

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/__main__.py", line 18, in <module>
    main(module=None)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/main.py", line 101, in __init__
    self.runTests()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/main.py", line 271, in runTests
    self.result = testRunner.run(self.test)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/runner.py", line 183, in run
    result.printErrors()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/runner.py", line 109, in printErrors
    self.printErrorList('ERROR', self.errors)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/runner.py", line 115, in printErrorList
    self.stream.writeln("%s: %s" % (flavour,self.getDescription(test)))
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/runner.py", line 47, in getDescription
    return '\n'.join((str(test), doc_first_line))
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/case.py", line 1472, in __str__
    self._testFunc.__name__)
AttributeError: 'str' object has no attribute '__name__'

I look at plenty of usages of FunctionTestCase on Github and all of them seemed to be false positive, they were copies of the unittest/test/test_functiontestcase.py file

The patch in the attached PR is not correct thought, it only fixes one of the loader and all of them suffer from the same issue.