@@ -160,10 +160,11 @@ objects, they are still using the default pytest representation:
$ pytest test_time.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 8 items
<Module test_time.py>
<Function test_timedistance_v0[a0-b0-expected0]>
<Function test_timedistance_v0[a1-b1-expected1]>
@@ -224,7 +225,7 @@ this is a fully self-contained example which you can run with:
$ pytest test_scenarios.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
@@ -239,10 +240,11 @@ If you just collect tests you'll also nicely see 'advanced' and 'basic' as varia
$ pytest --collect-only test_scenarios.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
<Module test_scenarios.py>
<Class TestSampleWithScenarios>
<Function test_demo1[basic]>
@@ -317,10 +319,11 @@ Let's first see how it looks like at collection time:
$ pytest test_backends.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
<Module test_backends.py>
<Function test_db_initialized[d1]>
<Function test_db_initialized[d2]>
@@ -351,6 +354,30 @@ And then when we run the test:
The first invocation with ``db == "DB1"`` passed while the second with ``db == "DB2"`` failed. Our ``db`` fixture function has instantiated each of the DB values during the setup phase while the ``pytest_generate_tests`` generated two according calls to the ``test_db_initialized`` during the collection phase.
@@ -28,7 +28,7 @@ Flaky tests sometimes appear when a test suite is run in parallel (such as use o
Overly strict assertion
~~~~~~~~~~~~~~~~~~~~~~~
Overly strict assertions can cause problems with floating point comparison as well as timing issues. `pytest.approx <https://docs.pytest.org/en/latest/reference.html#pytest-approx>`_ is useful here.
Overly strict assertions can cause problems with floating point comparison as well as timing issues. `pytest.approx <https://docs.pytest.org/en/stable/reference.html#pytest-approx>`_ is useful here.
Pytest features
@@ -43,7 +43,8 @@ Xfail strict
PYTEST_CURRENT_TEST
~~~~~~~~~~~~~~~~~~~
:ref:`pytest current test env` may be useful for figuring out "which test got stuck".
:envvar:`PYTEST_CURRENT_TEST` may be useful for figuring out "which test got stuck".
See :ref:`pytest current test env` for more details.
============================ 1 failed in 0.12s =============================
This test returns a failure report because ``func(3)`` does not return ``5``.
The ``[100%]`` refers to the overall progress of running all test cases. After it finishes, pytest then shows a failure report because ``func(3)`` does not return ``5``.
.. note::
@@ -112,9 +112,15 @@ Execute the test function with “quiet” reporting mode:
. [100%]
1 passed in 0.12s
.. note::
The ``-q/--quiet`` flag keeps the output brief in this and following examples.
Once you develop multiple tests, you may want to group them into a class. pytest makes it easy to create a class containing more than one test:
.. code-block:: python
@@ -153,10 +159,61 @@ Once you develop multiple tests, you may want to group them into a class. pytest
The first test passed and the second failed. You can easily see the intermediate values in the assertion to help you understand the reason for the failure.
Grouping tests in classes can be beneficial for the following reasons:
* Test organization
* Sharing fixtures for tests only in that particular class
* Applying marks at the class level and having them implicitly apply to all tests
Something to be aware of when grouping tests inside classes is that each test has a unique instance of the class.
Having each test share the same class instance would be very detrimental to test isolation and would promote poor test practices.
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/latest/builtin.html>`_ to request arbitrary resources, like a unique temporary directory:
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/stable/builtin.html>`_ to request arbitrary resources, like a unique temporary directory:
======================= 1 failed, 2 passed in 0.12s ========================
.. note::
Parameter values are passed as-is to tests (no copy whatsoever).
For example, if you pass a list or a dict as a parameter value, and
the test case code mutates it, the mutations will be reflected in subsequent
test case calls.
.. note::
pytest by default escapes any non-ascii characters used in unicode strings
for the parametrization because it has several downsides.
If however you would like to use unicode strings in parametrization and see them in the terminal as is (non-escaped), use this option in your ``pytest.ini``:
If however you would like to use unicode strings in parametrization
and see them in the terminal as is (non-escaped), use this option
Here's a list of scenarios where pytest may need to change ``sys.path`` in order
to import test modules or ``conftest.py`` files.
.. _`import-modes`:
Import modes
------------
pytest as a testing framework needs to import test modules and ``conftest.py`` files for execution.
Importing files in Python (at least until recently) is a non-trivial processes, often requiring
changing `sys.path <https://docs.python.org/3/library/sys.html#sys.path>`__. Some aspects of the
import process can be controlled through the ``--import-mode`` command-line flag, which can assume
these values:
* ``prepend`` (default): the directory path containing each module will be inserted into the *beginning*
of ``sys.path`` if not already there, and then imported with the `__import__ <https://docs.python.org/3/library/functions.html#__import__>`__ builtin.
This requires test module names to be unique when the test directory tree is not arranged in
packages, because the modules will put in ``sys.modules`` after importing.
This is the classic mechanism, dating back from the time Python 2 was still supported.
* ``append``: the directory containing each module is appended to the end of ``sys.path`` if not already
there, and imported with ``__import__``.
This better allows to run test modules against installed versions of a package even if the
package under test has the same import root. For example:
::
testing/__init__.py
testing/test_pkg_under_test.py
pkg_under_test/
the tests will run against the installed version
of ``pkg_under_test`` when ``--import-mode=append`` is used whereas
with ``prepend`` they would pick up the local version. This kind of confusion is why
we advocate for using :ref:`src <src-layout>` layouts.
Same as ``prepend``, requires test module names to be unique when the test directory tree is
not arranged in packages, because the modules will put in ``sys.modules`` after importing.
* ``importlib``: new in pytest-6.0, this mode uses `importlib <https://docs.python.org/3/library/importlib.html>`__ to import test modules. This gives full control over the import process, and doesn't require
changing ``sys.path`` or ``sys.modules`` at all.
For this reason this doesn't require test module names to be unique at all, but also makes test
modules non-importable by each other. This was made possible in previous modes, for tests not residing
in Python packages, because of the side-effects of changing ``sys.path`` and ``sys.modules``
mentioned above. Users which require this should turn their tests into proper packages instead.
We intend to make ``importlib`` the default in future releases.
``prepend`` and ``append`` import modes scenarios
-------------------------------------------------
Here's a list of scenarios when using ``prepend`` or ``append`` import modes where pytest needs to
change ``sys.path`` in order to import test modules or ``conftest.py`` files, and the issues users
might encounter because of that.
Test modules / ``conftest.py`` files inside packages
- `Free 1h webinar: "pytest: Test Driven Development für Python" <https://mylearning.ch/kurse/online-kurse/tech-webinar/>`_ (German), online, August 18 2020.
- `"pytest: Test Driven Development (nicht nur) für Python" <https://workshoptage.ch/workshops/2020/pytest-test-driven-development-nicht-nur-fuer-python/>`_ (German) at the `CH Open Workshoptage <https://workshoptage.ch/>`_, September 8 2020, HSLU Campus Rotkreuz (ZG), Switzerland.
======================= 1 passed, 1 warning in 0.12s =======================
The ``-W`` flag can be passed to control which warnings will be displayed or even turn
@@ -117,7 +117,7 @@ Filters applied using a mark take precedence over filters passed on the command
by the ``filterwarnings`` ini option.
You may apply a filter to all tests of a class by using the ``filterwarnings`` mark as a class
decorator or to all tests in a module by setting the ``pytestmark`` variable:
decorator or to all tests in a module by setting the :globalvar:`pytestmark` variable:
.. code-block:: python
@@ -381,8 +381,6 @@ custom error message.
Internal pytest warnings
------------------------
pytest may generate its own warnings in some situations, such as improper usage or deprecated features.
For example, pytest will emit a warning if it encounters a class that matches :confval:`python_classes` but also
@@ -407,30 +405,12 @@ defines an ``__init__`` constructor, as this prevents the class from being insta
$REGENDOC_TMPDIR/test_pytest_warnings.py:1: PytestCollectionWarning: cannot collect test class 'Test' because it has a __init__ constructor (from: test_pytest_warnings.py)
You can require plugins in a test module or a ``conftest.py`` file like this:
You can require plugins in a test module or a ``conftest.py`` file using :globalvar:`pytest_plugins`:
.. code-block:: python
@@ -241,31 +241,31 @@ application modules:
pytest_plugins ="myapp.testsupport.myplugin"
``pytest_plugins`` variables are processed recursively, so note that in the example above
if ``myapp.testsupport.myplugin`` also declares ``pytest_plugins``, the contents
:globalvar:`pytest_plugins` are processed recursively, so note that in the example above
if ``myapp.testsupport.myplugin`` also declares :globalvar:`pytest_plugins`, the contents
of the variable will also be loaded as plugins, and so on.
.. _`requiring plugins in non-root conftests`:
.. note::
Requiring plugins using a ``pytest_plugins`` variable in non-root
Requiring plugins using :globalvar:`pytest_plugins` variable in non-root
``conftest.py`` files is deprecated.
This is important because ``conftest.py`` files implement per-directory
hook implementations, but once a plugin is imported, it will affect the
entire directory tree. In order to avoid confusion, defining
``pytest_plugins`` in any ``conftest.py`` file which is not located in the
:globalvar:`pytest_plugins` in any ``conftest.py`` file which is not located in the
tests root directory is deprecated, and will raise a warning.
This mechanism makes it easy to share fixtures within applications or even
external applications without the need to create external plugins using
the ``setuptools``'s entry point technique.
Plugins imported by ``pytest_plugins`` will also automatically be marked
Plugins imported by :globalvar:`pytest_plugins` will also automatically be marked
for assertion rewriting (see :func:`pytest.register_assert_rewrite`).
However for this to have any effect the module must not be
imported already; if it was already imported at the time the
``pytest_plugins`` statement is processed, a warning will result and
:globalvar:`pytest_plugins` statement is processed, a warning will result and
assertions inside the plugin will not be rewritten. To fix this you
can either call :func:`pytest.register_assert_rewrite` yourself before
the module is imported, or you can arrange the code to delay the
@@ -430,9 +430,9 @@ additionally it is possible to copy examples for an example folder before runnin
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
rootdir: $REGENDOC_TMPDIR, configfile: pytest.ini
collected 2 items
test_example.py .. [100%]
@@ -442,13 +442,8 @@ additionally it is possible to copy examples for an example folder before runnin
$REGENDOC_TMPDIR/test_example.py:4: PytestExperimentalApiWarning: testdir.copy_example is an experimental api that may change over time
testdir.copy_example("test_example.py")
test_example.py::test_plugin
$PYTHON_PREFIX/lib/python3.8/site-packages/_pytest/terminal.py:287: PytestDeprecationWarning: TerminalReporter.writer attribute is deprecated, use TerminalReporter._tw instead at your own risk.
See https://docs.pytest.org/en/latest/deprecations.html#terminalreporter-writer for more information.