@@ -15,7 +15,7 @@ For basic examples, see

- :doc:`../getting-started` for basic introductory examples
- :ref:`assert` for basic assertion examples
- :ref:`fixtures` for basic fixture/setup examples
- :ref:`Fixtures <fixtures>` for basic fixture/setup examples
- :ref:`parametrize` for basic test function parametrization
- :doc:`../unittest` for basic unittest integration
- :doc:`../nose` for basic nosetests integration

Large diffs are not rendered by default.

@@ -29,7 +29,7 @@ now execute the test specification:

nonpython $ pytest test_simple.yaml
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
collected 2 items
@@ -66,7 +66,7 @@ consulted when reporting in ``verbose`` mode:

nonpython $ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
collecting ... collected 2 items
@@ -92,11 +92,12 @@ interesting to just look at the collection tree:

nonpython $ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
collected 2 items
<Package $REGENDOC_TMPDIR/nonpython>

<Package nonpython>
<YamlFile test_simple.yaml>
<YamlItem hello>
<YamlItem ok>
@@ -160,10 +160,11 @@ objects, they are still using the default pytest representation:

$ pytest test_time.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 8 items

<Module test_time.py>
<Function test_timedistance_v0[a0-b0-expected0]>
<Function test_timedistance_v0[a1-b1-expected1]>
@@ -224,7 +225,7 @@ this is a fully self-contained example which you can run with:

$ pytest test_scenarios.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
@@ -239,10 +240,11 @@ If you just collect tests you'll also nicely see 'advanced' and 'basic' as varia

$ pytest --collect-only test_scenarios.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items

<Module test_scenarios.py>
<Class TestSampleWithScenarios>
<Function test_demo1[basic]>
@@ -317,10 +319,11 @@ Let's first see how it looks like at collection time:

$ pytest test_backends.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items

<Module test_backends.py>
<Function test_db_initialized[d1]>
<Function test_db_initialized[d2]>
@@ -351,6 +354,30 @@ And then when we run the test:

The first invocation with ``db == "DB1"`` passed while the second with ``db == "DB2"`` failed. Our ``db`` fixture function has instantiated each of the DB values during the setup phase while the ``pytest_generate_tests`` generated two according calls to the ``test_db_initialized`` during the collection phase.

Indirect parametrization
---------------------------------------------------

Using the ``indirect=True`` parameter when parametrizing a test allows to
parametrize a test with a fixture receiving the values before passing them to a
test:

.. code-block:: python
import pytest
@pytest.fixture
def fixt(request):
return request.param * 3
@pytest.mark.parametrize("fixt", ["a", "b"], indirect=True)
def test_indirect(fixt):
assert len(fixt) == 3
This can be used, for example, to do more expensive setup at test run time in
the fixture, rather than having to run those setup steps at collection time.

.. regendoc:wipe
Apply indirect on particular arguments
@@ -389,21 +416,18 @@ The result of this test will be successful:

.. code-block:: pytest

$ pytest test_indirect_list.py --collect-only
$ pytest -v test_indirect_list.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
<Module test_indirect_list.py>
<Function test_indirect[a-b]>
collecting ... collected 1 item

========================== no tests ran in 0.12s ===========================
test_indirect_list.py::test_indirect[a-b] PASSED [100%]

.. regendoc:wipe
============================ 1 passed in 0.12s =============================

Note, that each argument in `parametrize` list should be explicitly declared in corresponding
python test function or via `indirect`.
.. regendoc:wipe
Parametrizing test methods through per-class configuration
--------------------------------------------------------------
@@ -486,8 +510,8 @@ Running it results in some skips if we don't have all the python interpreters in
. $ pytest -rs -q multipython.py
ssssssssssss...ssssssssssss [100%]
========================= short test summary info ==========================
SKIPPED [12] $REGENDOC_TMPDIR/CWD/multipython.py:29: 'python3.5' not found
SKIPPED [12] $REGENDOC_TMPDIR/CWD/multipython.py:29: 'python3.7' not found
SKIPPED [12] multipython.py:29: 'python3.5' not found
SKIPPED [12] multipython.py:29: 'python3.7' not found
3 passed, 24 skipped in 0.12s

Indirect parametrization of optional implementations/imports
@@ -548,15 +572,15 @@ If you run this with reporting for skips enabled:

$ pytest -rs test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items

test_module.py .s [100%]

========================= short test summary info ==========================
SKIPPED [1] $REGENDOC_TMPDIR/conftest.py:12: could not import 'opt2': No module named 'opt2'
SKIPPED [1] conftest.py:12: could not import 'opt2': No module named 'opt2'
======================= 1 passed, 1 skipped in 0.12s =======================

You'll see that we don't have an ``opt2`` module and thus the second test run
@@ -610,7 +634,7 @@ Then run ``pytest`` with verbose mode and with only the ``basic`` marker:

$ pytest -v -m basic
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 14 items / 11 deselected / 3 selected
@@ -115,15 +115,13 @@ Changing naming conventions

You can configure different naming conventions by setting
the :confval:`python_files`, :confval:`python_classes` and
:confval:`python_functions` configuration options.
:confval:`python_functions` in your :ref:`configuration file <config file formats>`.
Here is an example:

.. code-block:: ini

# content of pytest.ini
# Example 1: have pytest look for "check" instead of "test"
# can also be defined in tox.ini or setup.cfg file, although the section
# name in setup.cfg files should be "tool:pytest"
[pytest]
python_files = check_*.py
python_classes = Check
@@ -149,10 +147,11 @@ The test collection would look like this:

$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
rootdir: $REGENDOC_TMPDIR, configfile: pytest.ini
collected 2 items

<Module check_myapp.py>
<Class CheckMyApp>
<Function simple_check>
@@ -165,8 +164,7 @@ You can check for multiple glob patterns by adding a space between the patterns:
.. code-block:: ini

# Example 2: have pytest look for files with "test" and "example"
# content of pytest.ini, tox.ini, or setup.cfg file (replace "pytest"
# with "tool:pytest" for setup.cfg)
# content of pytest.ini
[pytest]
python_files = test_*.py example_*.py

@@ -211,10 +209,11 @@ You can always peek at the collection tree without running tests like this:

. $ pytest --collect-only pythoncollection.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
rootdir: $REGENDOC_TMPDIR, configfile: pytest.ini
collected 3 items

<Module CWD/pythoncollection.py>
<Function test_function>
<Class TestClass>
@@ -292,15 +291,15 @@ file will be left out:

$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
rootdir: $REGENDOC_TMPDIR, configfile: pytest.ini
collected 0 items

========================== no tests ran in 0.12s ===========================

It's also possible to ignore files based on Unix shell-style wildcards by adding
patterns to ``collect_ignore_glob``.
patterns to :globalvar:`collect_ignore_glob`.

The following example ``conftest.py`` ignores the file ``setup.py`` and in
addition all files that end with ``*_py2.py`` when executed with a Python 3

Large diffs are not rendered by default.

@@ -3,6 +3,50 @@
Basic patterns and examples
==========================================================

How to change command line options defaults
-------------------------------------------

It can be tedious to type the same series of command line options
every time you use ``pytest``. For example, if you always want to see
detailed info on skipped and xfailed tests, as well as have terser "dot"
progress output, you can write it into a configuration file:

.. code-block:: ini

# content of pytest.ini
[pytest]
addopts = -ra -q


Alternatively, you can set a ``PYTEST_ADDOPTS`` environment variable to add command
line options while the environment is in use:

.. code-block:: bash
export PYTEST_ADDOPTS="-v"
Here's how the command-line is built in the presence of ``addopts`` or the environment variable:

.. code-block:: text

<pytest.ini:addopts> $PYTEST_ADDOPTS <extra command-line arguments>

So if the user executes in the command-line:

.. code-block:: bash
pytest -m slow
The actual command line executed is:

.. code-block:: bash
pytest -ra -q -v -m slow
Note that as usual for other command-line applications, in case of conflicting options the last one wins, so the example
above will show verbose output because ``-v`` overwrites ``-q``.


.. _request example:

Pass different values to a test function, depending on command line options
@@ -131,7 +175,7 @@ directory with the above conftest.py:

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 0 items
@@ -196,7 +240,7 @@ and when running it will see a skipped "slow" test:

$ pytest -rs # "-rs" means report details on the little 's'
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -213,7 +257,7 @@ Or run it including the ``slow`` marked test:

$ pytest --runslow
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -222,8 +266,10 @@ Or run it including the ``slow`` marked test:

============================ 2 passed in 0.12s =============================

.. _`__tracebackhide__`:

Writing well integrated assertion helpers
--------------------------------------------------
-----------------------------------------

.. regendoc:wipe
@@ -355,7 +401,7 @@ which will add the string to the test header accordingly:

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
project deps: mylib-1.1
rootdir: $REGENDOC_TMPDIR
@@ -384,7 +430,7 @@ which will add info only when run with "--v":

$ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
info1: did you know that ...
did you?
@@ -399,7 +445,7 @@ and nothing when run plainly:

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 0 items
@@ -439,14 +485,14 @@ Now we can profile which test functions execute the slowest:

$ pytest --durations=3
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 3 items

test_some_are_slow.py ... [100%]

========================= slowest 3 test durations =========================
=========================== slowest 3 durations ============================
0.30s call test_some_are_slow.py::test_funcslow2
0.20s call test_some_are_slow.py::test_funcslow1
0.10s call test_some_are_slow.py::test_funcfast
@@ -545,7 +591,7 @@ If we run this:

$ pytest -rx
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
@@ -629,7 +675,7 @@ We can run this:

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 7 items
@@ -748,7 +794,7 @@ and run them:

$ pytest test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -855,7 +901,7 @@ and run it:

$ pytest -s test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 3 items
@@ -908,9 +954,9 @@ information.

Sometimes a test session might get stuck and there might be no easy way to figure out
which test got stuck, for example if pytest was run in quiet mode (``-q``) or you don't have access to the console
output. This is particularly a problem if the problem helps only sporadically, the famous "flaky" kind of tests.
output. This is particularly a problem if the problem happens only sporadically, the famous "flaky" kind of tests.

``pytest`` sets a ``PYTEST_CURRENT_TEST`` environment variable when running tests, which can be inspected
``pytest`` sets the :envvar:`PYTEST_CURRENT_TEST` environment variable when running tests, which can be inspected
by process monitoring utilities or libraries like `psutil <https://pypi.org/project/psutil/>`_ to discover which
test got stuck if necessary:

@@ -924,8 +970,8 @@ test got stuck if necessary:
print(f'pytest process {pid} running: {environ["PYTEST_CURRENT_TEST"]}')
During the test session pytest will set ``PYTEST_CURRENT_TEST`` to the current test
:ref:`nodeid <nodeids>` and the current stage, which can be ``setup``, ``call``
and ``teardown``.
:ref:`nodeid <nodeids>` and the current stage, which can be ``setup``, ``call``,
or ``teardown``.

For example, when running a single test function named ``test_foo`` from ``foo_module.py``,
``PYTEST_CURRENT_TEST`` will be set to:
@@ -32,7 +32,7 @@ just run your tests even if you return Deferreds. In addition,
there also is a dedicated `pytest-twisted
<https://pypi.org/project/pytest-twisted/>`_ plugin which allows you to
return deferreds from pytest-style tests, allowing the use of
:ref:`fixtures` and other features.
:ref:`fixtures <fixtures>` and other features.

how does pytest work with Django?
++++++++++++++++++++++++++++++++++++++++++++++
@@ -151,7 +151,7 @@ marked ``smtp_connection`` fixture function. Running the test looks like this:

$ pytest test_smtpsimple.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -179,7 +179,7 @@ In the failure traceback we see that the test function was called with a
function. The test function fails on our deliberate ``assert 0``. Here is
the exact protocol used by ``pytest`` to call the test function this way:

1. pytest :ref:`finds <test discovery>` the ``test_ehlo`` because
1. pytest :ref:`finds <test discovery>` the test ``test_ehlo`` because
of the ``test_`` prefix. The test function needs a function argument
named ``smtp_connection``. A matching fixture function is discovered by
looking for a fixture-marked function named ``smtp_connection``.
@@ -244,8 +244,8 @@ and `pytest-datafiles <https://pypi.org/project/pytest-datafiles/>`__.

.. _smtpshared:

Scope: sharing a fixture instance across tests in a class, module or session
----------------------------------------------------------------------------
Scope: sharing fixtures across classes, modules, packages or session
--------------------------------------------------------------------

.. regendoc:wipe
@@ -303,7 +303,7 @@ inspect what is going on and can now run the tests:

$ pytest test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -356,29 +356,23 @@ instance, you can simply declare it:
# all tests needing it
...
Finally, the ``class`` scope will invoke the fixture once per test *class*.

.. note::

Pytest will only cache one instance of a fixture at a time.
This means that when using a parametrized fixture, pytest may invoke a fixture more than once in the given scope.


``package`` scope (experimental)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Fixture scopes
^^^^^^^^^^^^^^

In pytest 3.7 the ``package`` scope has been introduced. Package-scoped fixtures
are finalized when the last test of a *package* finishes.
Fixtures are created when first requested by a test, and are destroyed based on their ``scope``:

.. warning::
This functionality is considered **experimental** and may be removed in future
versions if hidden corner-cases or serious problems with this functionality
are discovered after it gets more usage in the wild.
* ``function``: the default scope, the fixture is destroyed at the end of the test.
* ``class``: the fixture is destroyed during teardown of the last test in the class.
* ``module``: the fixture is destroyed during teardown of the last test in the module.
* ``package``: the fixture is destroyed during teardown of the last test in the package.
* ``session``: the fixture is destroyed at the end of the test session.

Use this new feature sparingly and please make sure to report any issues you find.
.. note::

Pytest only caches one instance of a fixture at a time, which
means that when using a parametrized fixture, pytest may invoke a fixture more than once in
the given scope.

.. _dynamic scope:

@@ -415,7 +409,7 @@ Order: Higher-scoped fixtures are instantiated first



Within a function request for features, fixture of higher-scopes (such as ``session``) are instantiated first than
Within a function request for fixtures, those of higher-scopes (such as ``session``) are instantiated before
lower-scoped fixtures (such as ``function`` or ``class``). The relative order of fixtures of same scope follows
the declared order in the test function and honours dependencies between fixtures. Autouse fixtures will be
instantiated before explicitly used fixtures.
@@ -665,6 +659,37 @@ Running it:
voila! The ``smtp_connection`` fixture function picked up our mail server name
from the module namespace.

.. _`using-markers`:

Using markers to pass data to fixtures
-------------------------------------------------------------

Using the :py:class:`request <FixtureRequest>` object, a fixture can also access
markers which are applied to a test function. This can be useful to pass data
into a fixture from a test:

.. code-block:: python
import pytest
@pytest.fixture
def fixt(request):
marker = request.node.get_closest_marker("fixt_data")
if marker is None:
# Handle missing marker in some way...
data = None
else:
data = marker.args[0]
# Do something with the data
return data
@pytest.mark.fixt_data(42)
def test_fixt(fixt):
assert fixt == 42
.. _`fixture-factory`:

Factories as fixtures
@@ -828,7 +853,7 @@ be used with ``-k`` to select specific cases to run, and they will
also identify the specific case when one is failing. Running pytest
with ``--collect-only`` will show the generated IDs.

Numbers, strings, booleans and None will have their usual string
Numbers, strings, booleans and ``None`` will have their usual string
representation used in the test ID. For other objects, pytest will
make a string based on the argument name. It is possible to customise
the string used in a test ID for a certain fixture value by using the
@@ -867,18 +892,19 @@ the string used in a test ID for a certain fixture value by using the
The above shows how ``ids`` can be either a list of strings to use or
a function which will be called with the fixture value and then
has to return a string to use. In the latter case if the function
return ``None`` then pytest's auto-generated ID will be used.
returns ``None`` then pytest's auto-generated ID will be used.

Running the above tests results in the following test IDs being used:

.. code-block:: pytest

$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 10 items

<Module test_anothersmtp.py>
<Function test_showhelo[smtp.gmail.com]>
<Function test_showhelo[mail.python.org]>
@@ -925,7 +951,7 @@ Running this test will *skip* the invocation of ``data_set`` with value ``2``:

$ pytest test_fixture_marks.py -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 3 items
@@ -975,7 +1001,7 @@ Here we declare an ``app`` fixture which receives the previously defined

$ pytest -v test_appsetup.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 2 items
@@ -1055,7 +1081,7 @@ Let's run the tests in verbose mode and with looking at the print-output:

$ pytest -v -s test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 8 items
@@ -1181,15 +1207,12 @@ You can specify multiple fixtures like this:
def test():
...
and you may specify fixture usage at the test module level, using
a generic feature of the mark mechanism:
and you may specify fixture usage at the test module level using :globalvar:`pytestmark`:

.. code-block:: python
pytestmark = pytest.mark.usefixtures("cleandir")
Note that the assigned variable *must* be called ``pytestmark``, assigning e.g.
``foomark`` will not activate the fixtures.
It is also possible to put fixtures required by all tests in your project
into an ini-file:
@@ -28,7 +28,7 @@ Flaky tests sometimes appear when a test suite is run in parallel (such as use o
Overly strict assertion
~~~~~~~~~~~~~~~~~~~~~~~

Overly strict assertions can cause problems with floating point comparison as well as timing issues. `pytest.approx <https://docs.pytest.org/en/latest/reference.html#pytest-approx>`_ is useful here.
Overly strict assertions can cause problems with floating point comparison as well as timing issues. `pytest.approx <https://docs.pytest.org/en/stable/reference.html#pytest-approx>`_ is useful here.


Pytest features
@@ -43,7 +43,8 @@ Xfail strict
PYTEST_CURRENT_TEST
~~~~~~~~~~~~~~~~~~~

:ref:`pytest current test env` may be useful for figuring out "which test got stuck".
:envvar:`PYTEST_CURRENT_TEST` may be useful for figuring out "which test got stuck".
See :ref:`pytest current test env` for more details.


Plugins
@@ -7,7 +7,7 @@ pytest-2.3: reasoning for fixture/funcarg evolution

**Target audience**: Reading this document requires basic knowledge of
python testing, xUnit setup methods and the (previous) basic pytest
funcarg mechanism, see https://docs.pytest.org/en/latest/historical-notes.html#funcargs-and-pytest-funcarg.
funcarg mechanism, see https://docs.pytest.org/en/stable/historical-notes.html#funcargs-and-pytest-funcarg.
If you are new to pytest, then you can simply ignore this
section and read the other sections.

@@ -170,7 +170,7 @@ several problems:

1. in distributed testing the master process would setup test resources
that are never needed because it only co-ordinates the test run
activities of the slave processes.
activities of the worker processes.

2. if you only perform a collection (with "--collect-only")
resource-setup will still be executed.
@@ -1,7 +1,7 @@
Installation and Getting Started
===================================

**Pythons**: Python 3.5, 3.6, 3.7, PyPy3
**Pythons**: Python 3.5, 3.6, 3.7, 3.8, 3.9, PyPy3

**Platforms**: Linux and Windows

@@ -28,7 +28,7 @@ Install ``pytest``
.. code-block:: bash
$ pytest --version
This is pytest version 5.x.y, imported from $PYTHON_PREFIX/lib/python3.8/site-packages/pytest/__init__.py
pytest 6.0.0
.. _`simpletest`:

@@ -53,7 +53,7 @@ That’s it. You can now execute the test function:

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -73,7 +73,7 @@ That’s it. You can now execute the test function:
FAILED test_sample.py::test_answer - assert 4 == 5
============================ 1 failed in 0.12s =============================

This test returns a failure report because ``func(3)`` does not return ``5``.
The ``[100%]`` refers to the overall progress of running all test cases. After it finishes, pytest then shows a failure report because ``func(3)`` does not return ``5``.

.. note::

@@ -112,9 +112,15 @@ Execute the test function with “quiet” reporting mode:
. [100%]
1 passed in 0.12s

.. note::

The ``-q/--quiet`` flag keeps the output brief in this and following examples.

Group multiple tests in a class
--------------------------------------------------------------

.. regendoc:wipe
Once you develop multiple tests, you may want to group them into a class. pytest makes it easy to create a class containing more than one test:

.. code-block:: python
@@ -153,10 +159,61 @@ Once you develop multiple tests, you may want to group them into a class. pytest

The first test passed and the second failed. You can easily see the intermediate values in the assertion to help you understand the reason for the failure.

Grouping tests in classes can be beneficial for the following reasons:

* Test organization
* Sharing fixtures for tests only in that particular class
* Applying marks at the class level and having them implicitly apply to all tests

Something to be aware of when grouping tests inside classes is that each test has a unique instance of the class.
Having each test share the same class instance would be very detrimental to test isolation and would promote poor test practices.
This is outlined below:

.. regendoc:wipe
.. code-block:: python
# content of test_class_demo.py
class TestClassDemoInstance:
def test_one(self):
assert 0
def test_two(self):
assert 0
.. code-block:: pytest

$ pytest -k TestClassDemoInstance -q
FF [100%]
================================= FAILURES =================================
______________________ TestClassDemoInstance.test_one ______________________

self = <test_class_demo.TestClassDemoInstance object at 0xdeadbeef>

def test_one(self):
> assert 0
E assert 0

test_class_demo.py:3: AssertionError
______________________ TestClassDemoInstance.test_two ______________________

self = <test_class_demo.TestClassDemoInstance object at 0xdeadbeef>

def test_two(self):
> assert 0
E assert 0

test_class_demo.py:6: AssertionError
========================= short test summary info ==========================
FAILED test_class_demo.py::TestClassDemoInstance::test_one - assert 0
FAILED test_class_demo.py::TestClassDemoInstance::test_two - assert 0
2 failed in 0.12s

Request a unique temporary directory for functional tests
--------------------------------------------------------------

``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/latest/builtin.html>`_ to request arbitrary resources, like a unique temporary directory:
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/stable/builtin.html>`_ to request arbitrary resources, like a unique temporary directory:

.. code-block:: python
@@ -1,4 +1,4 @@
.. highlightlang:: python
.. highlight:: python
.. _`goodpractices`:

Good Integration Practices
@@ -91,7 +91,8 @@ This has the following benefits:
See :ref:`pytest vs python -m pytest` for more information about the difference between calling ``pytest`` and
``python -m pytest``.

Note that using this scheme your test files must have **unique names**, because
Note that this scheme has a drawback if you are using ``prepend`` :ref:`import mode <import-modes>`
(which is the default): your test files must have **unique names**, because
``pytest`` will import them as *top-level* modules since there are no packages
to derive a full package name from. In other words, the test files in the example above will
be imported as ``test_app`` and ``test_view`` top-level modules by adding ``tests/`` to
@@ -118,9 +119,12 @@ Now pytest will load the modules as ``tests.foo.test_view`` and ``tests.bar.test
you to have modules with the same name. But now this introduces a subtle problem: in order to load
the test modules from the ``tests`` directory, pytest prepends the root of the repository to
``sys.path``, which adds the side-effect that now ``mypkg`` is also importable.

This is problematic if you are using a tool like `tox`_ to test your package in a virtual environment,
because you want to test the *installed* version of your package, not the local code from the repository.

.. _`src-layout`:

In this situation, it is **strongly** suggested to use a ``src`` layout where application root package resides in a
sub-directory of your root:

@@ -145,6 +149,15 @@ sub-directory of your root:
This layout prevents a lot of common pitfalls and has many benefits, which are better explained in this excellent
`blog post by Ionel Cristian Mărieș <https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure>`_.

.. note::
The new ``--import-mode=importlib`` (see :ref:`import-modes`) doesn't have
any of the drawbacks above because ``sys.path`` and ``sys.modules`` are not changed when importing
test modules, so users that run
into this issue are strongly encouraged to try it and report if the new option works well for them.

The ``src`` directory layout is still strongly recommended however.


Tests as part of application code
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

@@ -190,8 +203,8 @@ Note that this layout also works in conjunction with the ``src`` layout mentione

.. note::

If ``pytest`` finds an "a/b/test_module.py" test file while
recursing into the filesystem it determines the import name
In ``prepend`` and ``append`` import-modes, if pytest finds a ``"a/b/test_module.py"``
test file while recursing into the filesystem it determines the import name
as follows:

* determine ``basedir``: this is the first "upward" (towards the root)
@@ -212,6 +225,10 @@ Note that this layout also works in conjunction with the ``src`` layout mentione
from each other and thus deriving a canonical import name helps
to avoid surprises such as a test module getting imported twice.

With ``--import-mode=importlib`` things are less convoluted because
pytest doesn't need to change ``sys.path`` or ``sys.modules``, making things
much less surprising.


.. _`virtualenv`: https://pypi.org/project/virtualenv/
.. _`buildout`: http://www.buildout.org/
@@ -28,7 +28,7 @@ To execute it:

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -250,6 +250,9 @@ made in ``3.4`` after community feedback:

* Log levels are no longer changed unless explicitly requested by the :confval:`log_level` configuration
or ``--log-level`` command-line options. This allows users to configure logger objects themselves.
Setting :confval:`log_level` will set the level that is captured globally so if a specific test requires
a lower level than this, use the ``caplog.set_level()`` functionality otherwise that test will be prone to
failure.
* :ref:`Live Logs <live_logs>` is now disabled by default and can be enabled setting the
:confval:`log_cli` configuration option to ``true``. When enabled, the verbosity is increased so logging for each
test is visible.
@@ -4,14 +4,19 @@ Marking test functions with attributes
======================================

By using the ``pytest.mark`` helper you can easily set
metadata on your test functions. There are
some builtin markers, for example:
metadata on your test functions. You can find the full list of builtin markers
in the :ref:`API Reference<marks ref>`. Or you can list all the markers, including
builtin and custom, using the CLI - :code:`pytest --markers`.

Here are some of the builtin markers:

* :ref:`usefixtures <usefixtures>` - use fixtures on a test function or class
* :ref:`filterwarnings <filterwarnings>` - filter certain warnings of a test function
* :ref:`skip <skip>` - always skip a test function
* :ref:`skipif <skipif>` - skip a test function if a certain condition is met
* :ref:`xfail <xfail>` - produce an "expected failure" outcome if a certain
condition is met
* :ref:`parametrize <parametrizemark>` to perform multiple calls
* :ref:`parametrize <parametrizemark>` - perform multiple calls
to the same test function.

It's easy to create custom markers or to apply markers
@@ -268,7 +268,7 @@ to do this using the ``setenv`` and ``delenv`` method. Our example code to test:
def get_os_user_lower():
"""Simple retrieval function.
Returns lowercase USER or raises EnvironmentError."""
Returns lowercase USER or raises OSError."""
username = os.getenv("USER")
if username is None:
@@ -293,7 +293,7 @@ both paths can be safely tested without impacting the running environment:
def test_raise_exception(monkeypatch):
"""Remove the USER env var and assert EnvironmentError is raised."""
"""Remove the USER env var and assert OSError is raised."""
monkeypatch.delenv("USER", raising=False)
with pytest.raises(OSError):
@@ -26,7 +26,6 @@ Supported nose Idioms
* setup and teardown at module/class/method level
* SkipTest exceptions and markers
* setup/teardown decorators
* ``yield``-based tests and their setup (considered deprecated as of pytest 3.0)
* ``__test__`` attribute on modules/classes/functions
* general usage of nose utilities

@@ -65,10 +64,8 @@ Unsupported idioms / known issues

- no nose-configuration is recognized.

- ``yield``-based methods don't support ``setup`` properly because
the ``setup`` method is always called in the same class instance.
There are no plans to fix this currently because ``yield``-tests
are deprecated in pytest 3.0, with ``pytest.mark.parametrize``
being the recommended alternative.
- ``yield``-based methods are unsupported as of pytest 4.1.0. They are
fundamentally incompatible with pytest because they don't support fixtures
properly since collection and test execution are separated.

.. _nose: https://nose.readthedocs.io/en/latest/
@@ -56,7 +56,7 @@ them in turn:

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 3 items
@@ -79,19 +79,30 @@ them in turn:
FAILED test_expectation.py::test_eval[6*9-42] - AssertionError: assert 54...
======================= 1 failed, 2 passed in 0.12s ========================
.. note::

Parameter values are passed as-is to tests (no copy whatsoever).

For example, if you pass a list or a dict as a parameter value, and
the test case code mutates it, the mutations will be reflected in subsequent
test case calls.

.. note::

pytest by default escapes any non-ascii characters used in unicode strings
for the parametrization because it has several downsides.
If however you would like to use unicode strings in parametrization and see them in the terminal as is (non-escaped), use this option in your ``pytest.ini``:
If however you would like to use unicode strings in parametrization
and see them in the terminal as is (non-escaped), use this option
in your ``pytest.ini``:

.. code-block:: ini

[pytest]
disable_test_id_escaping_and_forfeit_all_rights_to_community_support = True

Keep in mind however that this might cause unwanted side effects and
even bugs depending on the OS used and plugins currently installed, so use it at your own risk.
even bugs depending on the OS used and plugins currently installed,
so use it at your own risk.


As designed in this example, only one pair of input/output values fails
@@ -123,7 +134,7 @@ Let's run this:

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 3 items
@@ -133,7 +144,7 @@ Let's run this:
======================= 2 passed, 1 xfailed in 0.12s =======================

The one parameter set which caused a failure previously now
shows up as an "xfailed (expected to fail)" test.
shows up as an "xfailed" (expected to fail) test.

In case the values provided to ``parametrize`` result in an empty list - for
example, if they're dynamically generated by some function - the behaviour of
@@ -70,7 +70,7 @@ You may also discover more plugins through a `pytest- pypi.org search`_.
Requiring/Loading plugins in a test module or conftest file
-----------------------------------------------------------

You can require plugins in a test module or a conftest file like this:
You can require plugins in a test module or a conftest file using :globalvar:`pytest_plugins`:

.. code-block:: python
@@ -80,6 +80,7 @@ When the test module or conftest plugin is loaded the specified plugins
will be loaded as well.

.. note::

Requiring plugins using a ``pytest_plugins`` variable in non-root
``conftest.py`` files is deprecated. See
:ref:`full explanation <requiring plugins in non-root conftests>`
@@ -9,7 +9,7 @@ In case of Python 2 and 3, the difference between the languages makes it even mo
because many new Python 3 features cannot be used in a Python 2/3 compatible code base.

Python 2.7 EOL has been reached `in 2020 <https://legacy.python.org/dev/peps/pep-0373/#id4>`__, with
the last release planned for mid-April, 2020.
the last release made in April, 2020.

Python 3.4 EOL has been reached `in 2019 <https://www.python.org/dev/peps/pep-0429/#release-schedule>`__, with the last release made in March, 2019.

@@ -3,11 +3,65 @@
pytest import mechanisms and ``sys.path``/``PYTHONPATH``
========================================================

Here's a list of scenarios where pytest may need to change ``sys.path`` in order
to import test modules or ``conftest.py`` files.
.. _`import-modes`:

Import modes
------------

pytest as a testing framework needs to import test modules and ``conftest.py`` files for execution.

Importing files in Python (at least until recently) is a non-trivial processes, often requiring
changing `sys.path <https://docs.python.org/3/library/sys.html#sys.path>`__. Some aspects of the
import process can be controlled through the ``--import-mode`` command-line flag, which can assume
these values:

* ``prepend`` (default): the directory path containing each module will be inserted into the *beginning*
of ``sys.path`` if not already there, and then imported with the `__import__ <https://docs.python.org/3/library/functions.html#__import__>`__ builtin.

This requires test module names to be unique when the test directory tree is not arranged in
packages, because the modules will put in ``sys.modules`` after importing.

This is the classic mechanism, dating back from the time Python 2 was still supported.

* ``append``: the directory containing each module is appended to the end of ``sys.path`` if not already
there, and imported with ``__import__``.

This better allows to run test modules against installed versions of a package even if the
package under test has the same import root. For example:

::

testing/__init__.py
testing/test_pkg_under_test.py
pkg_under_test/

the tests will run against the installed version
of ``pkg_under_test`` when ``--import-mode=append`` is used whereas
with ``prepend`` they would pick up the local version. This kind of confusion is why
we advocate for using :ref:`src <src-layout>` layouts.

Same as ``prepend``, requires test module names to be unique when the test directory tree is
not arranged in packages, because the modules will put in ``sys.modules`` after importing.

* ``importlib``: new in pytest-6.0, this mode uses `importlib <https://docs.python.org/3/library/importlib.html>`__ to import test modules. This gives full control over the import process, and doesn't require
changing ``sys.path`` or ``sys.modules`` at all.

For this reason this doesn't require test module names to be unique at all, but also makes test
modules non-importable by each other. This was made possible in previous modes, for tests not residing
in Python packages, because of the side-effects of changing ``sys.path`` and ``sys.modules``
mentioned above. Users which require this should turn their tests into proper packages instead.

We intend to make ``importlib`` the default in future releases.

``prepend`` and ``append`` import modes scenarios
-------------------------------------------------

Here's a list of scenarios when using ``prepend`` or ``append`` import modes where pytest needs to
change ``sys.path`` in order to import test modules or ``conftest.py`` files, and the issues users
might encounter because of that.

Test modules / ``conftest.py`` files inside packages
----------------------------------------------------
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Consider this file and directory layout::

@@ -28,8 +82,6 @@ When executing:
pytest root/
pytest will find ``foo/bar/tests/test_foo.py`` and realize it is part of a package given that
there's an ``__init__.py`` file in the same folder. It will then search upwards until it can find the
last folder which still contains an ``__init__.py`` file in order to find the package *root* (in
@@ -44,7 +96,7 @@ and allow test modules to have duplicated names. This is also discussed in detai
:ref:`test discovery`.

Standalone test modules / ``conftest.py`` files
-----------------------------------------------
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Consider this file and directory layout::

Large diffs are not rendered by default.

@@ -1,4 +1,5 @@
pallets-sphinx-themes
pygments-pytest>=1.1.0
sphinx>=1.8.2,<2.1
sphinxcontrib-trio
sphinx-removed-in>=0.2.0
sphinx>=3.1,<4
sphinxcontrib-trio
@@ -14,7 +14,7 @@ otherwise pytest should skip running the test altogether. Common examples are sk
windows-only tests on non-windows platforms, or skipping tests that depend on an external
resource which is not available at the moment (for example a database).

A **xfail** means that you expect a test to fail for some reason.
An **xfail** means that you expect a test to fail for some reason.
A common example is a test for a feature not yet implemented, or a bug not yet fixed.
When a test passes despite being expected to fail (marked with ``pytest.mark.xfail``),
it's an **xpass** and will be reported in the test summary.
@@ -152,8 +152,8 @@ You can use the ``skipif`` marker (as any other marker) on classes:
If the condition is ``True``, this marker will produce a skip result for
each of the test methods of that class.

If you want to skip all test functions of a module, you may use
the ``pytestmark`` name on the global level:
If you want to skip all test functions of a module, you may use the
:globalvar:`pytestmark` global:

.. code-block:: python
@@ -265,43 +265,29 @@ internally by raising a known exception.
**Reference**: :ref:`pytest.mark.xfail ref`


.. _`xfail strict tutorial`:

``strict`` parameter
~~~~~~~~~~~~~~~~~~~~


``condition`` parameter
~~~~~~~~~~~~~~~~~~~~~~~

Both ``XFAIL`` and ``XPASS`` don't fail the test suite by default.
You can change this by setting the ``strict`` keyword-only parameter to ``True``:
If a test is only expected to fail under a certain condition, you can pass
that condition as the first parameter:

.. code-block:: python
@pytest.mark.xfail(strict=True)
@pytest.mark.xfail(sys.platform == "win32", reason="bug in a 3rd party library")
def test_function():
...
This will make ``XPASS`` ("unexpectedly passing") results from this test to fail the test suite.

You can change the default value of the ``strict`` parameter using the
``xfail_strict`` ini option:

.. code-block:: ini

[pytest]
xfail_strict=true

Note that you have to pass a reason as well (see the parameter description at
:ref:`pytest.mark.xfail ref`).

``reason`` parameter
~~~~~~~~~~~~~~~~~~~~

As with skipif_ you can also mark your expectation of a failure
on a particular platform:
You can specify the motive of an expected failure with the ``reason`` parameter:

.. code-block:: python
@pytest.mark.xfail(sys.version_info >= (3, 6), reason="python3.6 api changes")
@pytest.mark.xfail(reason="known parser issue")
def test_function():
...
@@ -336,6 +322,31 @@ even executed, use the ``run`` parameter as ``False``:
This is specially useful for xfailing tests that are crashing the interpreter and should be
investigated later.

.. _`xfail strict tutorial`:

``strict`` parameter
~~~~~~~~~~~~~~~~~~~~

Both ``XFAIL`` and ``XPASS`` don't fail the test suite by default.
You can change this by setting the ``strict`` keyword-only parameter to ``True``:

.. code-block:: python
@pytest.mark.xfail(strict=True)
def test_function():
...
This will make ``XPASS`` ("unexpectedly passing") results from this test to fail the test suite.

You can change the default value of the ``strict`` parameter using the
``xfail_strict`` ini option:

.. code-block:: ini

[pytest]
xfail_strict=true


Ignoring xfail
~~~~~~~~~~~~~~
@@ -362,7 +373,7 @@ Running it with the report-on-xfail option gives this output:

example $ pytest -rx xfail_demo.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/example
collected 7 items
@@ -2,6 +2,11 @@
Talks and Tutorials
==========================

.. sidebar:: Next Open Trainings

- `Free 1h webinar: "pytest: Test Driven Development für Python" <https://mylearning.ch/kurse/online-kurse/tech-webinar/>`_ (German), online, August 18 2020.
- `"pytest: Test Driven Development (nicht nur) für Python" <https://workshoptage.ch/workshops/2020/pytest-test-driven-development-nicht-nur-fuer-python/>`_ (German) at the `CH Open Workshoptage <https://workshoptage.ch/>`_, September 8 2020, HSLU Campus Rotkreuz (ZG), Switzerland.

.. _`funcargs`: funcargs.html

Books
@@ -41,7 +41,7 @@ Running this would result in a passed test except for the last

$ pytest test_tmp_path.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -114,7 +114,7 @@ Running this would result in a passed test except for the last

$ pytest test_tmpdir.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -137,7 +137,7 @@ the ``self.db`` values in the traceback:

$ pytest test_unittest_db.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -216,7 +216,7 @@ Example:

$ pytest -ra
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 6 items
@@ -241,7 +241,7 @@ Example:

test_example.py:14: AssertionError
========================= short test summary info ==========================
SKIPPED [1] $REGENDOC_TMPDIR/test_example.py:22: skipping this test
SKIPPED [1] test_example.py:22: skipping this test
XFAIL test_example.py::test_xfail
reason: xfailing this test
XPASS test_example.py::test_xpass always xfail
@@ -274,7 +274,7 @@ More than one character can be used, so for example to only see failed and skipp

$ pytest -rfs
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 6 items
@@ -300,7 +300,7 @@ More than one character can be used, so for example to only see failed and skipp
test_example.py:14: AssertionError
========================= short test summary info ==========================
FAILED test_example.py::test_fail - assert 0
SKIPPED [1] $REGENDOC_TMPDIR/test_example.py:22: skipping this test
SKIPPED [1] test_example.py:22: skipping this test
== 1 failed, 1 passed, 1 skipped, 1 xfailed, 1 xpassed, 1 error in 0.12s ===

Using ``p`` lists the passing tests, whilst ``P`` adds an extra section "PASSES" with those tests that passed but had
@@ -310,7 +310,7 @@ captured output:

$ pytest -rpP
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 6 items
@@ -698,7 +698,7 @@ by the `PyPy-test`_ web page to show test results over several revisions.

If you use this option, consider using the new `pytest-reportlog <https://github.com/pytest-dev/pytest-reportlog>`__ plugin instead.

See `the deprecation docs <https://docs.pytest.org/en/latest/deprecations.html#result-log-result-log>`__
See `the deprecation docs <https://docs.pytest.org/en/stable/deprecations.html#result-log-result-log>`__
for more information.


@@ -28,7 +28,7 @@ Running pytest now produces this output:

$ pytest test_show_warnings.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -40,7 +40,7 @@ Running pytest now produces this output:
$REGENDOC_TMPDIR/test_show_warnings.py:5: UserWarning: api v1, should use functions from v2
warnings.warn(UserWarning("api v1, should use functions from v2"))

-- Docs: https://docs.pytest.org/en/latest/warnings.html
-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================= 1 passed, 1 warning in 0.12s =======================

The ``-W`` flag can be passed to control which warnings will be displayed or even turn
@@ -117,7 +117,7 @@ Filters applied using a mark take precedence over filters passed on the command
by the ``filterwarnings`` ini option.

You may apply a filter to all tests of a class by using the ``filterwarnings`` mark as a class
decorator or to all tests in a module by setting the ``pytestmark`` variable:
decorator or to all tests in a module by setting the :globalvar:`pytestmark` variable:

.. code-block:: python
@@ -381,8 +381,6 @@ custom error message.
Internal pytest warnings
------------------------



pytest may generate its own warnings in some situations, such as improper usage or deprecated features.

For example, pytest will emit a warning if it encounters a class that matches :confval:`python_classes` but also
@@ -407,30 +405,12 @@ defines an ``__init__`` constructor, as this prevents the class from being insta
$REGENDOC_TMPDIR/test_pytest_warnings.py:1: PytestCollectionWarning: cannot collect test class 'Test' because it has a __init__ constructor (from: test_pytest_warnings.py)
class Test:

-- Docs: https://docs.pytest.org/en/latest/warnings.html
-- Docs: https://docs.pytest.org/en/stable/warnings.html
1 warning in 0.12s

These warnings might be filtered using the same builtin mechanisms used to filter other types of warnings.

Please read our :ref:`backwards-compatibility` to learn how we proceed about deprecating and eventually removing
features.

The following warning types are used by pytest and are part of the public API:

.. autoclass:: pytest.PytestWarning

.. autoclass:: pytest.PytestAssertRewriteWarning

.. autoclass:: pytest.PytestCacheWarning

.. autoclass:: pytest.PytestCollectionWarning

.. autoclass:: pytest.PytestConfigWarning

.. autoclass:: pytest.PytestDeprecationWarning

.. autoclass:: pytest.PytestExperimentalApiWarning

.. autoclass:: pytest.PytestUnhandledCoroutineWarning

.. autoclass:: pytest.PytestUnknownMarkWarning
The full list of warnings is listed in :ref:`the reference documentation <warnings ref>`.
@@ -52,7 +52,7 @@ Plugin discovery order at tool startup
your ``conftest.py`` file in the top level test or project root directory.

* by recursively loading all plugins specified by the
``pytest_plugins`` variable in ``conftest.py`` files
:globalvar:`pytest_plugins` variable in ``conftest.py`` files


.. _`pytest/plugin`: http://bitbucket.org/pytest-dev/pytest/src/tip/pytest/plugin/
@@ -227,7 +227,7 @@ import ``helper.py`` normally. The contents of
Requiring/Loading plugins in a test module or conftest file
-----------------------------------------------------------

You can require plugins in a test module or a ``conftest.py`` file like this:
You can require plugins in a test module or a ``conftest.py`` file using :globalvar:`pytest_plugins`:

.. code-block:: python
@@ -241,31 +241,31 @@ application modules:
pytest_plugins = "myapp.testsupport.myplugin"
``pytest_plugins`` variables are processed recursively, so note that in the example above
if ``myapp.testsupport.myplugin`` also declares ``pytest_plugins``, the contents
:globalvar:`pytest_plugins` are processed recursively, so note that in the example above
if ``myapp.testsupport.myplugin`` also declares :globalvar:`pytest_plugins`, the contents
of the variable will also be loaded as plugins, and so on.

.. _`requiring plugins in non-root conftests`:

.. note::
Requiring plugins using a ``pytest_plugins`` variable in non-root
Requiring plugins using :globalvar:`pytest_plugins` variable in non-root
``conftest.py`` files is deprecated.

This is important because ``conftest.py`` files implement per-directory
hook implementations, but once a plugin is imported, it will affect the
entire directory tree. In order to avoid confusion, defining
``pytest_plugins`` in any ``conftest.py`` file which is not located in the
:globalvar:`pytest_plugins` in any ``conftest.py`` file which is not located in the
tests root directory is deprecated, and will raise a warning.

This mechanism makes it easy to share fixtures within applications or even
external applications without the need to create external plugins using
the ``setuptools``'s entry point technique.

Plugins imported by ``pytest_plugins`` will also automatically be marked
Plugins imported by :globalvar:`pytest_plugins` will also automatically be marked
for assertion rewriting (see :func:`pytest.register_assert_rewrite`).
However for this to have any effect the module must not be
imported already; if it was already imported at the time the
``pytest_plugins`` statement is processed, a warning will result and
:globalvar:`pytest_plugins` statement is processed, a warning will result and
assertions inside the plugin will not be rewritten. To fix this you
can either call :func:`pytest.register_assert_rewrite` yourself before
the module is imported, or you can arrange the code to delay the
@@ -430,9 +430,9 @@ additionally it is possible to copy examples for an example folder before runnin

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
rootdir: $REGENDOC_TMPDIR, configfile: pytest.ini
collected 2 items

test_example.py .. [100%]
@@ -442,13 +442,8 @@ additionally it is possible to copy examples for an example folder before runnin
$REGENDOC_TMPDIR/test_example.py:4: PytestExperimentalApiWarning: testdir.copy_example is an experimental api that may change over time
testdir.copy_example("test_example.py")

test_example.py::test_plugin
$PYTHON_PREFIX/lib/python3.8/site-packages/_pytest/terminal.py:287: PytestDeprecationWarning: TerminalReporter.writer attribute is deprecated, use TerminalReporter._tw instead at your own risk.
See https://docs.pytest.org/en/latest/deprecations.html#terminalreporter-writer for more information.
warnings.warn(

-- Docs: https://docs.pytest.org/en/latest/warnings.html
====================== 2 passed, 2 warnings in 0.12s =======================
-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================= 2 passed, 1 warning in 0.12s =======================

For more information about the result object that ``runpytest()`` returns, and
the methods that it provides please check out the :py:class:`RunResult
@@ -513,6 +508,7 @@ call only executes until the first of N registered functions returns a
non-None result which is then taken as result of the overall hook call.
The remaining hook functions will not be called in this case.

.. _`hookwrapper`:

hookwrapper: executing around other hooks
-------------------------------------------------
@@ -557,8 +553,10 @@ perform tracing or other side effects around the actual hook implementations.
If the result of the underlying hook is a mutable object, they may modify
that result but it's probably better to avoid it.

For more information, consult the `pluggy documentation <http://pluggy.readthedocs.io/en/latest/#wrappers>`_.
For more information, consult the
:ref:`pluggy documentation about hookwrappers <pluggy:hookwrappers>`.

.. _plugin-hookorder:

Hook function ordering / call example
-------------------------------------
@@ -45,7 +45,7 @@ def main(args):


def _get_kind(issue):
labels = [l["name"] for l in issue["labels"]]
labels = [label["name"] for label in issue["labels"]]
for key in ("bug", "enhancement", "proposal"):
if key in labels:
return key
@@ -7,6 +7,49 @@ requires = [
]
build-backend = "setuptools.build_meta"

[tool.pytest.ini_options]
minversion = "2.0"
addopts = "-rfEX -p pytester --strict-markers"
python_files = ["test_*.py", "*_test.py", "testing/*/*.py"]
python_classes = ["Test", "Acceptance"]
python_functions = ["test"]
# NOTE: "doc" is not included here, but gets tested explicitly via "doctesting".
testpaths = ["testing"]
norecursedirs = ["testing/example_scripts"]
xfail_strict = true
filterwarnings = [
"error",
"default:Using or importing the ABCs:DeprecationWarning:unittest2.*",
"default:the imp module is deprecated in favour of importlib:DeprecationWarning:nose.*",
"ignore:Module already imported so cannot be rewritten:pytest.PytestWarning",
# produced by python3.6/site.py itself (3.6.7 on Travis, could not trigger it with 3.6.8)."
"ignore:.*U.*mode is deprecated:DeprecationWarning:(?!(pytest|_pytest))",
# produced by pytest-xdist
"ignore:.*type argument to addoption.*:DeprecationWarning",
# produced by python >=3.5 on execnet (pytest-xdist)
"ignore:.*inspect.getargspec.*deprecated, use inspect.signature.*:DeprecationWarning",
# pytest's own futurewarnings
"ignore::pytest.PytestExperimentalApiWarning",
# Do not cause SyntaxError for invalid escape sequences in py37.
# Those are caught/handled by pyupgrade, and not easy to filter with the
# module being the filename (with .py removed).
"default:invalid escape sequence:DeprecationWarning",
# ignore use of unregistered marks, because we use many to test the implementation
"ignore::_pytest.warning_types.PytestUnknownMarkWarning",
]
pytester_example_dir = "testing/example_scripts"
markers = [
# dummy markers for testing
"foo",
"bar",
"baz",
# conftest.py reorders tests moving slow ones to the end of the list
"slow",
# experimental mark for all tests using pexpect
"uses_pexpect",
]


[tool.towncrier]
package = "pytest"
package_dir = "src"
@@ -2,20 +2,21 @@
This script is part of the pytest release process which is triggered by comments
in issues.
This script is started by the `prepare_release.yml` workflow, which is triggered by two comment
This script is started by the `release-on-comment.yml` workflow, which is triggered by two comment
related events:
* https://help.github.com/en/actions/reference/events-that-trigger-workflows#issue-comment-event-issue_comment
* https://help.github.com/en/actions/reference/events-that-trigger-workflows#issues-event-issues
This script receives the payload and a secrets on the command line.
The payload must contain a comment with a phrase matching this regular expression:
The payload must contain a comment with a phrase matching this pseudo-regular expression:
@pytestbot please prepare release from <branch name>
@pytestbot please prepare (major )? release from <branch name>
Then the appropriate version will be obtained based on the given branch name:
* a major release from master if "major" appears in the phrase in that position
* a feature or bug fix release from master (based if there are features in the current changelog
folder)
* a bug fix from a maintenance branch
@@ -31,8 +32,10 @@
import re
import sys
from pathlib import Path
from subprocess import CalledProcessError
from subprocess import check_call
from subprocess import check_output
from subprocess import run
from textwrap import dedent
from typing import Dict
from typing import Optional
@@ -74,15 +77,15 @@ def get_comment_data(payload: Dict) -> str:

def validate_and_get_issue_comment_payload(
issue_payload_path: Optional[Path],
) -> Tuple[str, str]:
) -> Tuple[str, str, bool]:
payload = json.loads(issue_payload_path.read_text(encoding="UTF-8"))
body = get_comment_data(payload)["body"]
m = re.match(r"@pytestbot please prepare release from ([\w\-_\.]+)", body)
m = re.match(r"@pytestbot please prepare (major )?release from ([\w\-_\.]+)", body)
if m:
base_branch = m.group(1)
is_major, base_branch = m.group(1) is not None, m.group(2)
else:
base_branch = None
return payload, base_branch
is_major, base_branch = False, None
return payload, base_branch, is_major


def print_and_exit(msg) -> None:
@@ -91,7 +94,10 @@ def print_and_exit(msg) -> None:


def trigger_release(payload_path: Path, token: str) -> None:
payload, base_branch = validate_and_get_issue_comment_payload(payload_path)
error_contents = "" # to be used to store error output in case any command fails
payload, base_branch, is_major = validate_and_get_issue_comment_payload(
payload_path
)
if base_branch is None:
url = get_comment_data(payload)["html_url"]
print_and_exit(
@@ -106,10 +112,9 @@ def trigger_release(payload_path: Path, token: str) -> None:
issue = repo.issue(issue_number)

check_call(["git", "checkout", f"origin/{base_branch}"])
print("DEBUG:", check_output(["git", "rev-parse", "HEAD"]))

try:
version = find_next_version(base_branch)
version = find_next_version(base_branch, is_major)
except InvalidFeatureRelease as e:
issue.create_comment(str(e))
print_and_exit(f"{Fore.RED}{e}")
@@ -119,17 +124,42 @@ def trigger_release(payload_path: Path, token: str) -> None:

release_branch = f"release-{version}"

check_call(["git", "config", "user.name", "pytest bot"])
check_call(["git", "config", "user.email", "pytestbot@gmail.com"])
run(
["git", "config", "user.name", "pytest bot"],
text=True,
check=True,
capture_output=True,
)
run(
["git", "config", "user.email", "pytestbot@gmail.com"],
text=True,
check=True,
capture_output=True,
)

check_call(["git", "checkout", "-b", release_branch, f"origin/{base_branch}"])
run(
["git", "checkout", "-b", release_branch, f"origin/{base_branch}"],
text=True,
check=True,
capture_output=True,
)

print(f"Branch {Fore.CYAN}{release_branch}{Fore.RESET} created.")

check_call([sys.executable, "scripts/release.py", version])
run(
[sys.executable, "scripts/release.py", version, "--skip-check-links"],
text=True,
check=True,
capture_output=True,
)

oauth_url = f"https://{token}:x-oauth-basic@github.com/{SLUG}.git"
check_call(["git", "push", oauth_url, f"HEAD:{release_branch}", "--force"])
run(
["git", "push", oauth_url, f"HEAD:{release_branch}", "--force"],
text=True,
check=True,
capture_output=True,
)
print(f"Branch {Fore.CYAN}{release_branch}{Fore.RESET} pushed.")

body = PR_BODY.format(
@@ -149,7 +179,10 @@ def trigger_release(payload_path: Path, token: str) -> None:
print(f"Notified in original comment {Fore.CYAN}{comment.url}{Fore.RESET}.")

print(f"{Fore.GREEN}Success.")
except CalledProcessError as e:
error_contents = e.output
except Exception as e:
error_contents = str(e)
link = f"https://github.com/{SLUG}/actions/runs/{os.environ['GITHUB_RUN_ID']}"
issue.create_comment(
dedent(
@@ -166,8 +199,25 @@ def trigger_release(payload_path: Path, token: str) -> None:
)
print_and_exit(f"{Fore.RED}{e}")

if error_contents:
link = f"https://github.com/{SLUG}/actions/runs/{os.environ['GITHUB_RUN_ID']}"
issue.create_comment(
dedent(
f"""
Sorry, the request to prepare release `{version}` from {base_branch} failed with:
```
{error_contents}
```
See: {link}.
"""
)
)
print_and_exit(f"{Fore.RED}{error_contents}")


def find_next_version(base_branch: str) -> str:
def find_next_version(base_branch: str, is_major: bool) -> str:
output = check_output(["git", "tag"], encoding="UTF-8")
valid_versions = []
for v in output.splitlines():
@@ -194,7 +244,9 @@ def find_next_version(base_branch: str) -> str:
msg += "\n".join(f"* `{x.name}`" for x in sorted(features + breaking))
raise InvalidFeatureRelease(msg)

if is_feature_release:
if is_major:
return f"{last_version[0]+1}.0.0"
elif is_feature_release:
return f"{last_version[0]}.{last_version[1] + 1}.0"
else:
return f"{last_version[0]}.{last_version[1]}.{last_version[2] + 1}"
@@ -2,6 +2,7 @@
Invoke development tasks.
"""
import argparse
import os
from pathlib import Path
from subprocess import call
from subprocess import check_call
@@ -65,10 +66,13 @@ def announce(version):
check_call(["git", "add", str(target)])


def regen():
def regen(version):
"""Call regendoc tool to update examples and pytest output in the docs."""
print(f"{Fore.CYAN}[generate.regen] {Fore.RESET}Updating docs")
check_call(["tox", "-e", "regen"])
check_call(
["tox", "-e", "regen"],
env={**os.environ, "SETUPTOOLS_SCM_PRETEND_VERSION": version},
)


def fix_formatting():
@@ -88,13 +92,13 @@ def check_links():
def pre_release(version, *, skip_check_links):
"""Generates new docs, release announcements and creates a local tag."""
announce(version)
regen()
regen(version)
changelog(version, write_out=True)
fix_formatting()
if not skip_check_links:
check_links()

msg = "Preparing release version {}".format(version)
msg = "Prepare release version {}".format(version)
check_call(["git", "commit", "-a", "-m", msg])

print()
@@ -0,0 +1,15 @@
import sys
from subprocess import call


def main():
"""
Platform agnostic wrapper script for towncrier.
Fixes the issue (#7251) where windows users are unable to natively run tox -e docs to build pytest docs.
"""
with open("doc/en/_changelog_towncrier_draft.rst", "w") as draft_file:
return call(("towncrier", "--draft"), stdout=draft_file)


if __name__ == "__main__":
sys.exit(main())
@@ -2,34 +2,35 @@
name = pytest
description = pytest: simple powerful testing with Python
long_description = file: README.rst
long_description_content_type = text/x-rst
url = https://docs.pytest.org/en/latest/
project_urls =
Source=https://github.com/pytest-dev/pytest
Tracker=https://github.com/pytest-dev/pytest/issues

author = Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, Florian Bruhin and others

license = MIT license
keywords = test, unittest
license = MIT
license_file = LICENSE
platforms = unix, linux, osx, cygwin, win32
classifiers =
Development Status :: 6 - Mature
Intended Audience :: Developers
License :: OSI Approved :: MIT License
Operating System :: POSIX
Operating System :: Microsoft :: Windows
Operating System :: MacOS :: MacOS X
Topic :: Software Development :: Testing
Topic :: Software Development :: Libraries
Topic :: Utilities
Operating System :: Microsoft :: Windows
Operating System :: POSIX
Programming Language :: Python :: 3
Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3.5
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
platforms = unix, linux, osx, cygwin, win32
Programming Language :: Python :: 3.9
Topic :: Software Development :: Libraries
Topic :: Software Development :: Testing
Topic :: Utilities
keywords = test, unittest
project_urls =
Source=https://github.com/pytest-dev/pytest
Tracker=https://github.com/pytest-dev/pytest/issues

[options]
zip_safe = no
packages =
_pytest
_pytest._code
@@ -38,13 +39,45 @@ packages =
_pytest.config
_pytest.mark
pytest

install_requires =
attrs>=17.4.0
iniconfig
more-itertools>=4.0.0
packaging
pluggy>=0.12,<1.0
py>=1.8.2
toml
atomicwrites>=1.0;sys_platform=="win32"
colorama;sys_platform=="win32"
importlib-metadata>=0.12;python_version<"3.8"
pathlib2>=2.2.0;python_version<"3.6"
python_requires = >=3.5
package_dir =
=src
setup_requires =
setuptools>=40.0
setuptools-scm
zip_safe = no

[options.entry_points]
console_scripts =
pytest=pytest:main
py.test=pytest:main
pytest=pytest:console_main
py.test=pytest:console_main

[options.extras_require]
checkqa-mypy =
mypy==0.780
testing =
argcomplete
hypothesis>=3.56
mock
nose
requests
xmlschema

[options.package_data]
_pytest = py.typed
pytest = py.typed

[build_sphinx]
source-dir = doc/en/
@@ -56,13 +89,14 @@ upload-dir = doc/en/build/html

[check-manifest]
ignore =
src/_pytest/_version.py
src/_pytest/_version.py

[devpi:upload]
formats = sdist.tgz,bdist_wheel

[mypy]
mypy_path = src
check_untyped_defs = True
ignore_missing_imports = True
no_implicit_optional = True
show_error_codes = True
@@ -1,41 +1,8 @@
from setuptools import setup

# TODO: if py gets upgrade to >=1.6,
# remove _width_of_current_line in terminal.py
INSTALL_REQUIRES = [
"py>=1.5.0",
"packaging",
"attrs>=17.4.0", # should match oldattrs tox env.
"more-itertools>=4.0.0",
'atomicwrites>=1.0;sys_platform=="win32"',
'pathlib2>=2.2.0;python_version<"3.6"',
'colorama;sys_platform=="win32"',
"pluggy>=0.12,<1.0",
'importlib-metadata>=0.12;python_version<"3.8"',
"wcwidth",
]


def main():
setup(
use_scm_version={"write_to": "src/_pytest/_version.py"},
setup_requires=["setuptools-scm", "setuptools>=40.0"],
package_dir={"": "src"},
extras_require={
"testing": [
"argcomplete",
"hypothesis>=3.56",
"mock",
"nose",
"requests",
"xmlschema",
],
"checkqa-mypy": [
"mypy==v0.761", # keep this in sync with .pre-commit-config.yaml.
],
},
install_requires=INSTALL_REQUIRES,
)
setup(use_scm_version={"write_to": "src/_pytest/_version.py"})


if __name__ == "__main__":
@@ -1,10 +1,22 @@
""" python inspection/code generation API """
from .code import Code # noqa
from .code import ExceptionInfo # noqa
from .code import filter_traceback # noqa
from .code import Frame # noqa
from .code import getrawcode # noqa
from .code import Traceback # noqa
from .source import compile_ as compile # noqa
from .source import getfslineno # noqa
from .source import Source # noqa
"""Python inspection/code generation API."""
from .code import Code
from .code import ExceptionInfo
from .code import filter_traceback
from .code import Frame
from .code import getfslineno
from .code import getrawcode
from .code import Traceback
from .code import TracebackEntry
from .source import Source

__all__ = [
"Code",
"ExceptionInfo",
"filter_traceback",
"Frame",
"getfslineno",
"getrawcode",
"Traceback",
"TracebackEntry",
"Source",
]

Large diffs are not rendered by default.

@@ -1,63 +1,43 @@
import ast
import inspect
import linecache
import sys
import textwrap
import tokenize
import warnings
from bisect import bisect_right
from types import CodeType
from types import FrameType
from typing import Any
from typing import Iterable
from typing import Iterator
from typing import List
from typing import Optional
from typing import Sequence
from typing import Tuple
from typing import Union

import py

from _pytest.compat import get_real_func
from _pytest.compat import overload
from _pytest.compat import TYPE_CHECKING

if TYPE_CHECKING:
from typing_extensions import Literal


class Source:
""" an immutable object holding a source code fragment,
possibly deindenting it.
"""An immutable object holding a source code fragment.
When using Source(...), the source lines are deindented.
"""

_compilecounter = 0

def __init__(self, *parts, **kwargs) -> None:
self.lines = lines = [] # type: List[str]
de = kwargs.get("deindent", True)
for part in parts:
if not part:
partlines = [] # type: List[str]
elif isinstance(part, Source):
partlines = part.lines
elif isinstance(part, (tuple, list)):
partlines = [x.rstrip("\n") for x in part]
elif isinstance(part, str):
partlines = part.split("\n")
else:
partlines = getsource(part, deindent=de).lines
if de:
partlines = deindent(partlines)
lines.extend(partlines)

def __eq__(self, other):
try:
return self.lines == other.lines
except AttributeError:
if isinstance(other, str):
return str(self) == other
return False
def __init__(self, obj: object = None) -> None:
if not obj:
self.lines = [] # type: List[str]
elif isinstance(obj, Source):
self.lines = obj.lines
elif isinstance(obj, (tuple, list)):
self.lines = deindent(x.rstrip("\n") for x in obj)
elif isinstance(obj, str):
self.lines = deindent(obj.split("\n"))
else:
rawcode = getrawcode(obj)
src = inspect.getsource(rawcode)
self.lines = deindent(src.split("\n"))

def __eq__(self, other: object) -> bool:
if not isinstance(other, Source):
return NotImplemented
return self.lines == other.lines

# Ignore type because of https://github.com/python/mypy/issues/4266.
__hash__ = None # type: ignore
@@ -99,19 +79,6 @@ def strip(self) -> "Source":
source.lines[:] = self.lines[start:end]
return source

def putaround(
self, before: str = "", after: str = "", indent: str = " " * 4
) -> "Source":
""" return a copy of the source object with
'before' and 'after' wrapped around it.
"""
beforesource = Source(before)
aftersource = Source(after)
newsource = Source()
lines = [(indent + line) for line in self.lines]
newsource.lines = beforesource.lines + lines + aftersource.lines
return newsource

def indent(self, indent: str = " " * 4) -> "Source":
""" return a copy of the source object with
all lines indented by the given indent-string.
@@ -142,177 +109,9 @@ def deindent(self) -> "Source":
newsource.lines[:] = deindent(self.lines)
return newsource

def isparseable(self, deindent: bool = True) -> bool:
""" return True if source is parseable, heuristically
deindenting it by default.
"""
if deindent:
source = str(self.deindent())
else:
source = str(self)
try:
ast.parse(source)
except (SyntaxError, ValueError, TypeError):
return False
else:
return True

def __str__(self) -> str:
return "\n".join(self.lines)

@overload
def compile(
self,
filename: Optional[str] = ...,
mode: str = ...,
flag: "Literal[0]" = ...,
dont_inherit: int = ...,
_genframe: Optional[FrameType] = ...,
) -> CodeType:
raise NotImplementedError()

@overload # noqa: F811
def compile( # noqa: F811
self,
filename: Optional[str] = ...,
mode: str = ...,
flag: int = ...,
dont_inherit: int = ...,
_genframe: Optional[FrameType] = ...,
) -> Union[CodeType, ast.AST]:
raise NotImplementedError()

def compile( # noqa: F811
self,
filename: Optional[str] = None,
mode: str = "exec",
flag: int = 0,
dont_inherit: int = 0,
_genframe: Optional[FrameType] = None,
) -> Union[CodeType, ast.AST]:
""" return compiled code object. if filename is None
invent an artificial filename which displays
the source/line position of the caller frame.
"""
if not filename or py.path.local(filename).check(file=0):
if _genframe is None:
_genframe = sys._getframe(1) # the caller
fn, lineno = _genframe.f_code.co_filename, _genframe.f_lineno
base = "<%d-codegen " % self._compilecounter
self.__class__._compilecounter += 1
if not filename:
filename = base + "%s:%d>" % (fn, lineno)
else:
filename = base + "%r %s:%d>" % (filename, fn, lineno)
source = "\n".join(self.lines) + "\n"
try:
co = compile(source, filename, mode, flag)
except SyntaxError as ex:
# re-represent syntax errors from parsing python strings
msglines = self.lines[: ex.lineno]
if ex.offset:
msglines.append(" " * ex.offset + "^")
msglines.append("(code was compiled probably from here: %s)" % filename)
newex = SyntaxError("\n".join(msglines))
newex.offset = ex.offset
newex.lineno = ex.lineno
newex.text = ex.text
raise newex
else:
if flag & ast.PyCF_ONLY_AST:
assert isinstance(co, ast.AST)
return co
assert isinstance(co, CodeType)
lines = [(x + "\n") for x in self.lines]
# Type ignored because linecache.cache is private.
linecache.cache[filename] = (1, None, lines, filename) # type: ignore
return co


#
# public API shortcut functions
#


@overload
def compile_(
source: Union[str, bytes, ast.mod, ast.AST],
filename: Optional[str] = ...,
mode: str = ...,
flags: "Literal[0]" = ...,
dont_inherit: int = ...,
) -> CodeType:
raise NotImplementedError()


@overload # noqa: F811
def compile_( # noqa: F811
source: Union[str, bytes, ast.mod, ast.AST],
filename: Optional[str] = ...,
mode: str = ...,
flags: int = ...,
dont_inherit: int = ...,
) -> Union[CodeType, ast.AST]:
raise NotImplementedError()


def compile_( # noqa: F811
source: Union[str, bytes, ast.mod, ast.AST],
filename: Optional[str] = None,
mode: str = "exec",
flags: int = 0,
dont_inherit: int = 0,
) -> Union[CodeType, ast.AST]:
""" compile the given source to a raw code object,
and maintain an internal cache which allows later
retrieval of the source code for the code object
and any recursively created code objects.
"""
if isinstance(source, ast.AST):
# XXX should Source support having AST?
assert filename is not None
co = compile(source, filename, mode, flags, dont_inherit)
assert isinstance(co, (CodeType, ast.AST))
return co
_genframe = sys._getframe(1) # the caller
s = Source(source)
return s.compile(filename, mode, flags, _genframe=_genframe)


def getfslineno(obj: Any) -> Tuple[Union[str, py.path.local], int]:
""" Return source location (path, lineno) for the given object.
If the source cannot be determined return ("", -1).
The line number is 0-based.
"""
from .code import Code

# xxx let decorators etc specify a sane ordering
# NOTE: this used to be done in _pytest.compat.getfslineno, initially added
# in 6ec13a2b9. It ("place_as") appears to be something very custom.
obj = get_real_func(obj)
if hasattr(obj, "place_as"):
obj = obj.place_as

try:
code = Code(obj)
except TypeError:
try:
fn = inspect.getsourcefile(obj) or inspect.getfile(obj)
except TypeError:
return "", -1

fspath = fn and py.path.local(fn) or ""
lineno = -1
if fspath:
try:
_, lineno = findsource(obj)
except IOError:
pass
return fspath, lineno
else:
return code.path, code.firstlineno


#
# helper functions
@@ -329,25 +128,26 @@ def findsource(obj) -> Tuple[Optional[Source], int]:
return source, lineno


def getsource(obj, **kwargs) -> Source:
from .code import getrawcode

obj = getrawcode(obj)
def getrawcode(obj, trycall: bool = True):
""" return code object for given function. """
try:
strsrc = inspect.getsource(obj)
except IndentationError:
strsrc = '"Buggy python version consider upgrading, cannot get source"'
assert isinstance(strsrc, str)
return Source(strsrc, **kwargs)


def deindent(lines: Sequence[str]) -> List[str]:
return obj.__code__
except AttributeError:
obj = getattr(obj, "f_code", obj)
obj = getattr(obj, "__code__", obj)
if trycall and not hasattr(obj, "co_firstlineno"):
if hasattr(obj, "__call__") and not inspect.isclass(obj):
x = getrawcode(obj.__call__, trycall=False)
if hasattr(x, "co_firstlineno"):
return x
return obj


def deindent(lines: Iterable[str]) -> List[str]:
return textwrap.dedent("\n".join(lines)).splitlines()


def get_statement_startend2(lineno: int, node: ast.AST) -> Tuple[int, Optional[int]]:
import ast

# flatten all statements and except handlers into one lineno-list
# AST's line numbers start indexing at 1
values = [] # type: List[int]
@@ -1,39 +1,8 @@
from typing import List
from typing import Sequence
from .terminalwriter import get_terminal_width
from .terminalwriter import TerminalWriter

from py.io import TerminalWriter as BaseTerminalWriter # noqa: F401


class TerminalWriter(BaseTerminalWriter):
def _write_source(self, lines: List[str], indents: Sequence[str] = ()) -> None:
"""Write lines of source code possibly highlighted.
Keeping this private for now because the API is clunky. We should discuss how
to evolve the terminal writer so we can have more precise color support, for example
being able to write part of a line in one color and the rest in another, and so on.
"""
if indents and len(indents) != len(lines):
raise ValueError(
"indents size ({}) should have same size as lines ({})".format(
len(indents), len(lines)
)
)
if not indents:
indents = [""] * len(lines)
source = "\n".join(lines)
new_lines = self._highlight(source).splitlines()
for indent, new_line in zip(indents, new_lines):
self.line(indent + new_line)

def _highlight(self, source):
"""Highlight the given source code according to the "code_highlight" option"""
if not self.hasmarkup:
return source
try:
from pygments.formatters.terminal import TerminalFormatter
from pygments.lexers.python import PythonLexer
from pygments import highlight
except ImportError:
return source
else:
return highlight(source, PythonLexer(), TerminalFormatter(bg="dark"))
__all__ = [
"TerminalWriter",
"get_terminal_width",
]
@@ -1,9 +1,12 @@
import pprint
import reprlib
from typing import Any
from typing import Dict
from typing import IO
from typing import Optional


def _try_repr_or_str(obj):
def _try_repr_or_str(obj: object) -> str:
try:
return repr(obj)
except (KeyboardInterrupt, SystemExit):
@@ -12,15 +15,15 @@ def _try_repr_or_str(obj):
return '{}("{}")'.format(type(obj).__name__, obj)


def _format_repr_exception(exc: BaseException, obj: Any) -> str:
def _format_repr_exception(exc: BaseException, obj: object) -> str:
try:
exc_info = _try_repr_or_str(exc)
except (KeyboardInterrupt, SystemExit):
raise
except BaseException as exc:
exc_info = "unpresentable exception ({})".format(_try_repr_or_str(exc))
return "<[{} raised in repr()] {} object at 0x{:x}>".format(
exc_info, obj.__class__.__name__, id(obj)
exc_info, type(obj).__name__, id(obj)
)


@@ -42,7 +45,7 @@ def __init__(self, maxsize: int) -> None:
self.maxstring = maxsize
self.maxsize = maxsize

def repr(self, x: Any) -> str:
def repr(self, x: object) -> str:
try:
s = super().repr(x)
except (KeyboardInterrupt, SystemExit):
@@ -51,7 +54,7 @@ def repr(self, x: Any) -> str:
s = _format_repr_exception(exc, x)
return _ellipsize(s, self.maxsize)

def repr_instance(self, x: Any, level: int) -> str:
def repr_instance(self, x: object, level: int) -> str:
try:
s = repr(x)
except (KeyboardInterrupt, SystemExit):
@@ -61,7 +64,7 @@ def repr_instance(self, x: Any, level: int) -> str:
return _ellipsize(s, self.maxsize)


def safeformat(obj: Any) -> str:
def safeformat(obj: object) -> str:
"""return a pretty printed string for the given object.
Failing __repr__ functions of user instances will be represented
with a short exception info.
@@ -72,7 +75,7 @@ def safeformat(obj: Any) -> str:
return _format_repr_exception(exc, obj)


def saferepr(obj: Any, maxsize: int = 240) -> str:
def saferepr(obj: object, maxsize: int = 240) -> str:
"""return a size-limited safe repr-string for the given object.
Failing __repr__ functions of user instances will be represented
with a short exception info and 'saferepr' generally takes
@@ -85,19 +88,39 @@ def saferepr(obj: Any, maxsize: int = 240) -> str:
class AlwaysDispatchingPrettyPrinter(pprint.PrettyPrinter):
"""PrettyPrinter that always dispatches (regardless of width)."""

def _format(self, object, stream, indent, allowance, context, level):
p = self._dispatch.get(type(object).__repr__, None)
def _format(
self,
object: object,
stream: IO[str],
indent: int,
allowance: int,
context: Dict[int, Any],
level: int,
) -> None:
# Type ignored because _dispatch is private.
p = self._dispatch.get(type(object).__repr__, None) # type: ignore[attr-defined]

objid = id(object)
if objid in context or p is None:
return super()._format(object, stream, indent, allowance, context, level)
# Type ignored because _format is private.
super()._format( # type: ignore[misc]
object, stream, indent, allowance, context, level,
)
return

context[objid] = 1
p(self, object, stream, indent, allowance, context, level + 1)
del context[objid]


def _pformat_dispatch(object, indent=1, width=80, depth=None, *, compact=False):
def _pformat_dispatch(
object: object,
indent: int = 1,
width: int = 80,
depth: Optional[int] = None,
*,
compact: bool = False
) -> str:
return AlwaysDispatchingPrettyPrinter(
indent=indent, width=width, depth=depth, compact=compact
).pformat(object)