Fix for issue #464 (@attr in subclasses of unittest.TestCase misbehaves) #500
Conversation
When determing what tests to run for a class, loader.py previously would walk the class's base classes and include any test that had the same name as a test being run for a base class. There is no reason to do this and it leads to the wrong behavior in cases where the parent should run the test but not the child, for example when the child has a test with the same name but different attributes and the attrib plugin is being used. Fixes nose-devs#464.
|
This is obviously a duplicate of #464. Not sure why my pull request showed up as an issue? |
| @@ -110,10 +110,6 @@ def wanted(attr, cls=testCaseClass, sel=self.selector): | |||
| return False | |||
| return sel.wantMethod(item) | |||
| cases = filter(wanted, dir(testCaseClass)) | |||
| for base in testCaseClass.__bases__: | |||
There was a problem hiding this comment.
This seems like a pretty heavy-weight solution for this issue, with lots of backwards-compatibility issues. I know it would break quite a few test suites of mine. So I'd like to see another solution, if possible.
There was a problem hiding this comment.
All the tests pass and as far as I can tell this doesn't break anything. I don't think this should change the behavior of anything other than fixing the cited bug. Can you give an example of code that will see a back-compat impact?
There was a problem hiding this comment.
Any time you have a set of test cases like:
class A(TestCase):
def test(self):
pass
class B(A):
pass
-- that will break. Current behavior would be to run A.test and B.test, but with this change, B.test would not be run (unless I'm mistaken and this block of loader code does nothing). This is a pretty common pattern for parameterizing batches of tests by adding flags or settings to the test case that the tests then use. For instance if you have a bunch of identical tests to run against several versions of an api, you might put them all in the base test class with the api version as a class-level attribute, and have a bunch of subclasses that just override that attribute without defining any tests themselves.
There was a problem hiding this comment.
Yes, that case works fine. I just tried it to verify. We get the initial list of test cases using dir() on the test class on the line immediately before the ones I removed. 'test' shows up in dir(B), so the test is run.
There was a problem hiding this comment.
Ok. That's some comfort, but I'm still really not enthusiastic about fixing an issue with the attr plugin by removing code from the test loader. I'd really like to find some other way.
There was a problem hiding this comment.
In my opinion, what the loader is doing here is clearly wrong and broken. It makes no sense to walk the list of base classes and re-add tests that the child class explicitly said it doesn't want. It is a no-op in the vast majority of cases, but makes the code more confusing through superflous use of __bases__ and recursion which is immediately scary. The only case in which the code actually has an effect is when it does something you don't want by re-adding tests that were explicitly filtered out for a subclass. In this case, that affects the attr plugin, but it could cause the same problem for any plugin that wants to filter out certain tests.
Nose's test coverage is good (especially around the attr plugin, where I added functional tests covering all the cases I could come up with when I fixed all the other issues with the plugin) and the fact that this change doesn't affect any tests makes me more confident it is the right thing to do.
|
Ok, I think we need a 2nd (or 3rd ;) opinion. Summoning @kumar303! Are you comfortable removing this code from the loader to fix the attr plugin issue? |
When determing what tests to run for a class, loader.py previously
would walk the class's base classes and include any test that had
the same name as a test being run for a base class.
There is no reason to do this and it leads to the wrong behavior
in cases where the parent should run the test but not the child,
for example when the child has a test with the same name but
different attributes and the attrib plugin is being used.
Fixes #464.