Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.6.0 breaks usefixtures when using unittest/unittest2 #3498

Closed
owenleonard opened this Issue May 23, 2018 · 7 comments

Comments

Projects
None yet
3 participants
@owenleonard
Copy link

owenleonard commented May 23, 2018

Thanks for submitting an issue!

Here's a quick checklist in what to include:

  • Include a detailed description of the bug or suggestion
    @pytest.mark.usefixtures() on test cases in classes that inherit from unittest.TestCase and unittest2.TestCase no longer works. The fixtures are completely ignored.

Given your README.rst says Can run unittest (or trial), nose test suites out of the box;, I'm assuming this is an appropriate bug to file.

  • pip list of the virtual environment you are using
    In both venvs, I installed pytest and unittest2. The remaining packages are just dependencies

python 3.6.3

Owens-MacBook-Pro:bug owen$ pip list
Package        Version
-------------- -------
atomicwrites   1.1.5
attrs          18.1.0
linecache2     1.0.0
more-itertools 4.1.0
pip            10.0.1
pluggy         0.6.0
py             1.5.3
pytest         3.6.0
setuptools     39.2.0
six            1.11.0
traceback2     1.4.0
unittest2      1.1.0
wheel          0.31.1

python 2.7.10

Owens-MacBook-Pro:bug owen$ pip list
Package        Version
-------------- -------
atomicwrites   1.1.5
attrs          18.1.0
funcsigs       1.0.2
linecache2     1.0.0
more-itertools 4.1.0
pip            10.0.1
pluggy         0.6.0
py             1.5.3
pytest         3.6.0
setuptools     39.2.0
six            1.11.0
traceback2     1.4.0
unittest2      1.1.0
wheel          0.31.1
  • pytest and operating system versions
    pytest 3.6.0
    macOS High Sierra 10.13.4

  • Minimal example if possible

conftest.py

import pytest


@pytest.fixture(scope='function')
def fixture1():
    print("in fixture1")
    yield


@pytest.fixture(scope='function')
def fixture2():
    print("in fixture2")
    yield

tests.py

import pytest
# import unittest
# import unittest2


# class Tests(unittest.TestCase):
# class Tests(unittest2.TestCase):
class Tests(object):

    @pytest.mark.usefixtures("fixture1")
    def test_one(self):
        print("inside test_one")

    @pytest.mark.usefixtures("fixture1", "fixture2")
    def test_two(self):
        print("inside test_two")

Results identical between python 2.7.10 and 3.6.3:

using class Tests(object)

Owens-MacBook-Pro:bug owen$ pytest -s tests.py
=============================== test session starts ===============================
platform darwin -- Python 2.7.10, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: /Users/owen/repos/sandbox/bug, inifile:
collected 2 items

tests.py in fixture1
inside test_one
.in fixture1
in fixture2
inside test_two
.

============================ 2 passed in 0.02 seconds =============================

using class Tests(unittest.TestCase)

Owens-MacBook-Pro:bug owen$ pytest -s tests.py
=============================== test session starts ===============================
platform darwin -- Python 2.7.10, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: /Users/owen/repos/sandbox/bug, inifile:
collected 2 items

tests.py inside test_one
.inside test_two
.

============================ 2 passed in 0.01 seconds =============================

Notice in fixture1 and in fixture2 is not printed

using class Tests(unittest2.TestCase)

Owens-MacBook-Pro:bug owen$ pytest -s tests.py
=============================== test session starts ===============================
platform darwin -- Python 2.7.10, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: /Users/owen/repos/sandbox/bug, inifile:
collected 2 items

tests.py inside test_one
.inside test_two
.

============================ 2 passed in 0.04 seconds =============================

Notice in fixture1 and in fixture2 is not printed

I also installed pytest 3.5.1 instead of 3.6.0 to show it was working before using unittest2:

Owens-MacBook-Pro:bug owen$ pytest -s tests.py
=============================== test session starts ===============================
platform darwin -- Python 2.7.10, pytest-3.5.1, py-1.5.3, pluggy-0.6.0
rootdir: /Users/owen/repos/sandbox/bug, inifile:
collected 2 items

tests.py running fixture1
inside test_one
.running fixture1
running fixture2
inside test_two
.

============================ 2 passed in 0.02 seconds =============================
@pytestbot

This comment has been minimized.

Copy link

pytestbot commented May 23, 2018

GitMate.io thinks possibly related issues are #3181 (3.4.0 logging changes break devpi tests), #1862 (pytest 3.0 breaks Jenkins' "Publish JUnit test result report" for doctests), #2968 (Memory usage in 3.3.0), #386 (tests fail with pexpect-3.0), and #1367 (Support for Python 3.4 unittest subtests).

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented May 24, 2018

this issue has 2 strange parts

  1. marks where not correctly added from unit-test tests - which is a oversight on my part (i have a fix for that by now)
  2. the marks for filling fixtures are not considered correctly
    (i have no idea yet how to sort that part out
@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented May 24, 2018

it seems that at getfixtureinfo time there are no markers on the objects

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented May 24, 2018

due to a simply rotten implicit dependency in various parts of the node structure, the marker bugs where simply hiding the unit-test fixture initialization bug

now that markers get loaded correctly, the bad way the unittest plugin handles the actual test objects hides the markers until its too late

a fix may take a bit

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented May 24, 2018

found a plausible place for a quick-fix - getfixtureinfo is broken for Function due to using parent instead of the actual function

in parametrized test support this however is replaced by a FunctionDefinition providing the correct details

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue May 24, 2018

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented May 24, 2018

@owenleonard #3500 should fix your issue, i hope you can verify soon

@owenleonard

This comment has been minimized.

Copy link
Author

owenleonard commented May 24, 2018

Commented on the PR, but I'll comment here, too. The changes work for my use cases. Thanks for the quick response and fix!

nicoddemus added a commit that referenced this issue May 24, 2018

Merge pull request #3500 from RonnyPfannschmidt/fix-3498-unittest-marks
fix #3498 - correctly consider marks on unittest classes

guykisel added a commit to guykisel/inline-plz that referenced this issue Jul 6, 2018

Update pytest to 3.6.3 (#302)
This PR updates [pytest](https://pypi.org/project/pytest) from **3.5.1** to **3.6.3**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.6.2
   ```
   =========================

Bug Fixes
---------

- Fix regression in ``Node.add_marker`` by extracting the mark object of a
  ``MarkDecorator``. (`3555
  &lt;https://github.com/pytest-dev/pytest/issues/3555&gt;`_)

- Warnings without ``location`` were reported as ``None``. This is corrected to
  now report ``&lt;undetermined location&gt;``. (`3563
  &lt;https://github.com/pytest-dev/pytest/issues/3563&gt;`_)

- Continue to call finalizers in the stack when a finalizer in a former scope
  raises an exception. (`3569
  &lt;https://github.com/pytest-dev/pytest/issues/3569&gt;`_)

- Fix encoding error with `print` statements in doctests (`3583
  &lt;https://github.com/pytest-dev/pytest/issues/3583&gt;`_)


Improved Documentation
----------------------

- Add documentation for the ``--strict`` flag. (`3549
  &lt;https://github.com/pytest-dev/pytest/issues/3549&gt;`_)


Trivial/Internal Changes
------------------------

- Update old quotation style to parens in fixture.rst documentation. (`3525
  &lt;https://github.com/pytest-dev/pytest/issues/3525&gt;`_)

- Improve display of hint about ``--fulltrace`` with ``KeyboardInterrupt``.
  (`3545 &lt;https://github.com/pytest-dev/pytest/issues/3545&gt;`_)

- pytest&#39;s testsuite is no longer runnable through ``python setup.py test`` --
  instead invoke ``pytest`` or ``tox`` directly. (`3552
  &lt;https://github.com/pytest-dev/pytest/issues/3552&gt;`_)

- Fix typo in documentation (`3567
  &lt;https://github.com/pytest-dev/pytest/issues/3567&gt;`_)
   ```
   
  
  
   ### 3.6.1
   ```
   =========================

Bug Fixes
---------

- Fixed a bug where stdout and stderr were logged twice by junitxml when a test
  was marked xfail. (`3491
  &lt;https://github.com/pytest-dev/pytest/issues/3491&gt;`_)

- Fix ``usefixtures`` mark applyed to unittest tests by correctly instantiating
  ``FixtureInfo``. (`3498
  &lt;https://github.com/pytest-dev/pytest/issues/3498&gt;`_)

- Fix assertion rewriter compatibility with libraries that monkey patch
  ``file`` objects. (`3503
  &lt;https://github.com/pytest-dev/pytest/issues/3503&gt;`_)


Improved Documentation
----------------------

- Added a section on how to use fixtures as factories to the fixture
  documentation. (`3461 &lt;https://github.com/pytest-dev/pytest/issues/3461&gt;`_)


Trivial/Internal Changes
------------------------

- Enable caching for pip/pre-commit in order to reduce build time on
  travis/appveyor. (`3502
  &lt;https://github.com/pytest-dev/pytest/issues/3502&gt;`_)

- Switch pytest to the src/ layout as we already suggested it for good practice
  - now we implement it as well. (`3513
  &lt;https://github.com/pytest-dev/pytest/issues/3513&gt;`_)

- Fix if in tests to support 3.7.0b5, where a docstring handling in AST got
  reverted. (`3530 &lt;https://github.com/pytest-dev/pytest/issues/3530&gt;`_)

- Remove some python2.5 compatibility code. (`3529
  &lt;https://github.com/pytest-dev/pytest/issues/3529&gt;`_)
   ```
   
  
  
   ### 3.6.0
   ```
   =========================

Features
--------

- Revamp the internals of the ``pytest.mark`` implementation with correct per
  node handling which fixes a number of long standing bugs caused by the old
  design. This introduces new ``Node.iter_markers(name)`` and
  ``Node.get_closest_mark(name)`` APIs. Users are **strongly encouraged** to
  read the `reasons for the revamp in the docs
  &lt;https://docs.pytest.org/en/latest/mark.htmlmarker-revamp-and-iteration&gt;`_,
  or jump over to details about `updating existing code to use the new APIs
  &lt;https://docs.pytest.org/en/latest/mark.htmlupdating-code&gt;`_. (`3317
  &lt;https://github.com/pytest-dev/pytest/issues/3317&gt;`_)

- Now when ``pytest.fixture`` is applied more than once to the same function a
  ``ValueError`` is raised. This buggy behavior would cause surprising problems
  and if was working for a test suite it was mostly by accident. (`2334
  &lt;https://github.com/pytest-dev/pytest/issues/2334&gt;`_)

- Support for Python 3.7&#39;s builtin ``breakpoint()`` method, see `Using the
  builtin breakpoint function
  &lt;https://docs.pytest.org/en/latest/usage.htmlbreakpoint-builtin&gt;`_ for
  details. (`3180 &lt;https://github.com/pytest-dev/pytest/issues/3180&gt;`_)

- ``monkeypatch`` now supports a ``context()`` function which acts as a context
  manager which undoes all patching done within the ``with`` block. (`3290
  &lt;https://github.com/pytest-dev/pytest/issues/3290&gt;`_)

- The ``--pdb`` option now causes KeyboardInterrupt to enter the debugger,
  instead of stopping the test session. On python 2.7, hitting CTRL+C again
  exits the debugger. On python 3.2 and higher, use CTRL+D. (`3299
  &lt;https://github.com/pytest-dev/pytest/issues/3299&gt;`_)

- pytest not longer changes the log level of the root logger when the
  ``log-level`` parameter has greater numeric value than that of the level of
  the root logger, which makes it play better with custom logging configuration
  in user code. (`3307 &lt;https://github.com/pytest-dev/pytest/issues/3307&gt;`_)


Bug Fixes
---------

- A rare race-condition which might result in corrupted ``.pyc`` files on
  Windows has been hopefully solved. (`3008
  &lt;https://github.com/pytest-dev/pytest/issues/3008&gt;`_)

- Also use iter_marker for discovering the marks applying for marker
  expressions from the cli to avoid the bad data from the legacy mark storage.
  (`3441 &lt;https://github.com/pytest-dev/pytest/issues/3441&gt;`_)

- When showing diffs of failed assertions where the contents contain only
  whitespace, escape them using ``repr()`` first to make it easy to spot the
  differences. (`3443 &lt;https://github.com/pytest-dev/pytest/issues/3443&gt;`_)


Improved Documentation
----------------------

- Change documentation copyright year to a range which auto-updates itself each
  time it is published. (`3303
  &lt;https://github.com/pytest-dev/pytest/issues/3303&gt;`_)


Trivial/Internal Changes
------------------------

- ``pytest`` now depends on the `python-atomicwrites
  &lt;https://github.com/untitaker/python-atomicwrites&gt;`_ library. (`3008
  &lt;https://github.com/pytest-dev/pytest/issues/3008&gt;`_)

- Update all pypi.python.org URLs to pypi.org. (`3431
  &lt;https://github.com/pytest-dev/pytest/issues/3431&gt;`_)

- Detect `pytest_` prefixed hooks using the internal plugin manager since
  ``pluggy`` is deprecating the ``implprefix`` argument to ``PluginManager``.
  (`3487 &lt;https://github.com/pytest-dev/pytest/issues/3487&gt;`_)

- Import ``Mapping`` and ``Sequence`` from ``_pytest.compat`` instead of
  directly from ``collections`` in ``python_api.py::approx``. Add ``Mapping``
  to ``_pytest.compat``, import it from ``collections`` on python 2, but from
  ``collections.abc`` on Python 3 to avoid a ``DeprecationWarning`` on Python
  3.7 or newer. (`3497 &lt;https://github.com/pytest-dev/pytest/issues/3497&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Repo: https://github.com/pytest-dev/pytest/issues
  - Homepage: http://pytest.org
</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.