markers stains on all related classes #568

Closed
pytestbot opened this Issue Aug 23, 2014 · 14 comments

Comments

Projects
9 participants
@pytestbot

Originally reported by: Markus Unterwaditzer (BitBucket: untitaker, GitHub: untitaker)


Given this class hierarchy:

import pytest

class Foo(object):
    def test_lol(self):
        pass

@pytest.mark.skipif(False, reason='BECAUSE')
class TestBar(Foo):
    pass

@pytest.mark.skipif(True, reason='BECAUSE')
class TestBaz(Foo):
    pass

py.test will skip the single testcase of both TestBar and TestBaz.


@pytestbot pytestbot added the type: bug label Jun 15, 2015

@RonnyPfannschmidt

This comment has been minimized.

Show comment
Hide comment
@RonnyPfannschmidt

RonnyPfannschmidt Jul 25, 2015

Member

@untitaker this relates to a recent issue wrt marker transfer, current consent is that class level markers move to inherited methods

Member

RonnyPfannschmidt commented Jul 25, 2015

@untitaker this relates to a recent issue wrt marker transfer, current consent is that class level markers move to inherited methods

@pfctdayelise

This comment has been minimized.

Show comment
Hide comment
@pfctdayelise

pfctdayelise Jul 25, 2015

Member

Relates to #535

Member

pfctdayelise commented Jul 25, 2015

Relates to #535

@untitaker

This comment has been minimized.

Show comment
Hide comment
@untitaker

untitaker Jul 25, 2015

Contributor

Yeah I think this is a duplicate.

Ronny, I don't understand what you're saying. Are class markers discouraged? Is this expected behavior?

On 25 July 2015 19:16:40 CEST, Brianna Laugher notifications@github.com wrote:

Relates to #535


Reply to this email directly or view it on GitHub:
#568 (comment)

Sent from my phone. Please excuse my brevity.

Contributor

untitaker commented Jul 25, 2015

Yeah I think this is a duplicate.

Ronny, I don't understand what you're saying. Are class markers discouraged? Is this expected behavior?

On 25 July 2015 19:16:40 CEST, Brianna Laugher notifications@github.com wrote:

Relates to #535


Reply to this email directly or view it on GitHub:
#568 (comment)

Sent from my phone. Please excuse my brevity.

@nicoddemus

This comment has been minimized.

Show comment
Hide comment
@nicoddemus

nicoddemus Jul 25, 2015

Member

Here's the PR: #848

Member

nicoddemus commented Jul 25, 2015

Here's the PR: #848

@RonnyPfannschmidt

This comment has been minimized.

Show comment
Hide comment
@RonnyPfannschmidt

RonnyPfannschmidt Jul 26, 2015

Member

@untitaker py.test has a bug wrt marker transfer, we are on it

Member

RonnyPfannschmidt commented Jul 26, 2015

@untitaker py.test has a bug wrt marker transfer, we are on it

@untitaker

This comment has been minimized.

Show comment
Hide comment
@untitaker

untitaker Jul 26, 2015

Contributor

Ok, thanks!

On 26 July 2015 09:00:22 CEST, Ronny Pfannschmidt notifications@github.com wrote:

@untitaker py.test has a bug wrt marker transfer, we are on it


Reply to this email directly or view it on GitHub:
#568 (comment)

Sent from my phone. Please excuse my brevity.

Contributor

untitaker commented Jul 26, 2015

Ok, thanks!

On 26 July 2015 09:00:22 CEST, Ronny Pfannschmidt notifications@github.com wrote:

@untitaker py.test has a bug wrt marker transfer, we are on it


Reply to this email directly or view it on GitHub:
#568 (comment)

Sent from my phone. Please excuse my brevity.

jerrykan added a commit to jerrykan/herder that referenced this issue Sep 7, 2015

Work-around for pytest.mark.skipif() bug
There is a pytest bug that can cause all test classes marked with the
skipif() decorator that inherit a common class to be skipped if one of
the skipif() conditions is True. See:

    pytest-dev/pytest#568

@RonnyPfannschmidt RonnyPfannschmidt added this to the 3.0 milestone Sep 13, 2015

@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Nov 9, 2015

Any update on this issue?

laserson commented Nov 9, 2015

Any update on this issue?

@RonnyPfannschmidt

This comment has been minimized.

Show comment
Hide comment
@RonnyPfannschmidt

RonnyPfannschmidt Nov 10, 2015

Member

Unfortunately not, a backward compatible resolution is hard and needs time

Member

RonnyPfannschmidt commented Nov 10, 2015

Unfortunately not, a backward compatible resolution is hard and needs time

jerrykan added a commit to jerrykan/herder that referenced this issue Jan 6, 2016

Work-around for pytest.mark.skipif() bug
There is a pytest bug that can cause all test classes marked with the
skipif() decorator that inherit a common class to be skipped if one of
the skipif() conditions is True. See:

    pytest-dev/pytest#568

jerrykan added a commit to jerrykan/herder that referenced this issue Jan 12, 2016

Work-around for pytest.mark.skipif() bug
There is a pytest bug that can cause all test classes marked with the
skipif() decorator that inherit a common class to be skipped if one of
the skipif() conditions is True. See:

    pytest-dev/pytest#568

jerrykan added a commit to jerrykan/herder that referenced this issue Jan 12, 2016

Work-around for pytest.mark.skipif() bug
There is a pytest bug that can cause all test classes marked with the
skipif() decorator that inherit a common class to be skipped if one of
the skipif() conditions is True. See:

    pytest-dev/pytest#568

@nicoddemus nicoddemus changed the title from skipif marker stains on all related classes to markers stains on all related classes Mar 18, 2016

@untitaker

This comment has been minimized.

Show comment
Hide comment
@untitaker

untitaker May 3, 2016

Contributor

Here's a ugly workaround:

def mark_class(marker):
    '''Workaround for https://github.com/pytest-dev/pytest/issues/568'''
    import types
    def copy_func(f):
        try:
            return types.FunctionType(f.__code__, f.__globals__,
                                      name=f.__name__, argdefs=f.__defaults__,
                                      closure=f.__closure__)
        except AttributeError:
            return types.FunctionType(f.func_code, f.func_globals,
                                      name=f.func_name,
                                      argdefs=f.func_defaults,
                                      closure=f.func_closure)

    def mark(cls):
        for method in dir(cls):
            if method.startswith('test_'):
                f = copy_func(getattr(cls, method))
                setattr(cls, method, marker(f))
        return cls
    return mark


import pytest

class Foo(object):
    def test_lol(self):
        pass

@mark_class(pytest.mark.skipif(False, reason='BECAUSE'))
class TestBar(Foo):
    pass

@mark_class(pytest.mark.skipif(True, reason='BECAUSE'))
class TestBaz(Foo):
    pass
Contributor

untitaker commented May 3, 2016

Here's a ugly workaround:

def mark_class(marker):
    '''Workaround for https://github.com/pytest-dev/pytest/issues/568'''
    import types
    def copy_func(f):
        try:
            return types.FunctionType(f.__code__, f.__globals__,
                                      name=f.__name__, argdefs=f.__defaults__,
                                      closure=f.__closure__)
        except AttributeError:
            return types.FunctionType(f.func_code, f.func_globals,
                                      name=f.func_name,
                                      argdefs=f.func_defaults,
                                      closure=f.func_closure)

    def mark(cls):
        for method in dir(cls):
            if method.startswith('test_'):
                f = copy_func(getattr(cls, method))
                setattr(cls, method, marker(f))
        return cls
    return mark


import pytest

class Foo(object):
    def test_lol(self):
        pass

@mark_class(pytest.mark.skipif(False, reason='BECAUSE'))
class TestBar(Foo):
    pass

@mark_class(pytest.mark.skipif(True, reason='BECAUSE'))
class TestBaz(Foo):
    pass
@emmett9001

This comment has been minimized.

Show comment
Hide comment
@emmett9001

emmett9001 May 31, 2016

Here's how PyKafka works around this issue in its test suite

Here's how PyKafka works around this issue in its test suite

@RonnyPfannschmidt

This comment has been minimized.

Show comment
Hide comment
@RonnyPfannschmidt

RonnyPfannschmidt May 31, 2016

Member

@emmett9001 there is a pending branch for the initial mark refactor in #1534

after it is merged the base for a backward compatible implementation of non-smearing marker combinations wil lbe in place

Member

RonnyPfannschmidt commented May 31, 2016

@emmett9001 there is a pending branch for the initial mark refactor in #1534

after it is merged the base for a backward compatible implementation of non-smearing marker combinations wil lbe in place

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jun 25, 2016

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jun 25, 2016

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jun 27, 2016

nicoddemus added a commit that referenced this issue Jun 27, 2016

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Aug 5, 2016

Member

Since we have #1670 to track all marker staining related issues and the fix for that unfortunately won't make it to 3.0, I'm removing the milestone for now.

Member

The-Compiler commented Aug 5, 2016

Since we have #1670 to track all marker staining related issues and the fix for that unfortunately won't make it to 3.0, I'm removing the milestone for now.

@tetianakh

This comment has been minimized.

Show comment
Hide comment
@tetianakh

tetianakh May 18, 2017

I've tried the workaround posted by @untitaker and it works like a charm. Thanks @untitaker!

I've tried the workaround posted by @untitaker and it works like a charm. Thanks @untitaker!

@RonnyPfannschmidt RonnyPfannschmidt moved this from todo to triage in resolve the mark fallout Jun 23, 2017

@RonnyPfannschmidt RonnyPfannschmidt moved this from triage to todo in resolve the mark fallout Jun 23, 2017

@shoyer shoyer referenced this issue in pydata/xarray Sep 7, 2017

Merged

fix unintentional skipped tests #1557

4 of 4 tasks complete

alimanfoo added a commit to zarr-developers/zarr that referenced this issue Jan 3, 2018

@alimanfoo alimanfoo referenced this issue in zarr-developers/zarr Jan 3, 2018

Merged

Documentation about contributing #227

5 of 5 tasks complete

@jhamman jhamman referenced this issue in pydata/xarray Jan 13, 2018

Closed

Testing deprecations #1825

@TomAugspurger TomAugspurger referenced this issue in pandas-dev/pandas Feb 22, 2018

Merged

Test Decorators and Better Pytest Integration in 'test_excel' #19829

3 of 4 tasks complete

@RonnyPfannschmidt RonnyPfannschmidt moved this from todo to in progress in resolve the mark fallout Mar 27, 2018

@RonnyPfannschmidt RonnyPfannschmidt moved this from in progress to done in resolve the mark fallout Apr 10, 2018

@RonnyPfannschmidt

This comment has been minimized.

Show comment
Hide comment
Member

RonnyPfannschmidt commented Apr 10, 2018

fixed in #3317

@emmett9001 emmett9001 referenced this issue in Parsely/pykafka Apr 10, 2018

Open

simplify test skipping logic #782

llazzaro pushed a commit to infobyte/faraday that referenced this issue Jun 28, 2018

Fix bug with pytest that was unexpectedly skipping some tests
Because of a bug with pytest, I can't simply mark TestListCommandView
with @pytest.mark.skip. I had to made it inherit from object instad of
ReadOnlyAPITests, and to manually skip the extra tests inside the class.
See https://docs.pytest.org/en/latest/skipping.html#skip-all-test-functions-of-a-class-or-module
and pytest-dev/pytest#568 for more information

emmett9001 added a commit to Parsely/pykafka that referenced this issue Jul 19, 2018

upgrade pytest dependency
remove hacky test-skipping workaround now that pytest-dev/pytest#568 has been fixed

@emmett9001 emmett9001 referenced this issue in Parsely/pykafka Jul 19, 2018

Open

upgrade pytest dependency #839

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment