Mixin for abstract test classes

[Ginkss]

What's the problem this feature will solve?

Abstract test classes that use __test__=False to avoid collection require that the subclasses set __test__=True to be collected. If a user forgets to reset the attribute, the test they thought was running will be silently ignored. Difficult to catch in large projects.

Describe the solution you'd like

Have subclasses collected by pytest regardless of superclass attribute. I use a mixin class to achieve this:

class NotATest:
    def __init_subclass__(cls):
        cls.__test__ = NotATest not in cls.__bases__

class AbstractTest(NotATest):
    pass

class RealTest(AbstractTest):
    pass

Mixin clearly indicates what's not a test. No need to unset on subclasses. Safe to mixin on subclasses if they are also abstract. What I'm really asking is, would you like to add this mixin class to pytest.__init__?

Alternative Solutions

• Set abstract test class name so pytest will ignore it. Pycharm will also not reckonise it as a pytest class which means the IDE features like type hinting, navigating to the fixture function definition and find usage, don't work.
• Change pytest.Class.collect to access class __dict__ instead of using getattr. This ensures that the __test__ attribute is only considered if it is explicitly set for that class definition. However, I'm assuming that another use case for __test__=False is when an existing test set has classes who's name cause their collection even though they are completely unrelated and there subclasses should also never match.

Additional context

No

[Zac-HD]

What I'm really asking is, would you like to add this mixin class to pytest.__init__?

IMO this would be great as a 'recipe' in the documentation for the __test__ attribute, but doesn't quite meet the bar of generality-and-difficulty to justify adding it to our public API. Would you like to open a PR to add it to the docs?

[asottile]

mixins with side-effects are usually a smell, and I think this is not an exception here -- what if you have two levels of "abstract" you'd have to mixin the class twice (?) -- it's also just as much if not the same amount of typing as just putting __test__ = False -- admittedly, pytest is designed around avoiding test base classes and instead using fixtures and function-tests -- I usually consider inheritance-based testing a smell because it makes tests significantly harder to reason about and maintain