-
-
Notifications
You must be signed in to change notification settings - Fork 2.6k
/
usage.rst
251 lines (142 loc) · 6.61 KB
/
usage.rst
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
.. _usage:
How to invoke pytest
==========================================
.. seealso:: :ref:`Complete pytest command-line flag reference <command-line-flags>`
In general, pytest is invoked with the command ``pytest`` (see below for :ref:`other ways to invoke pytest
<invoke-other>`). This will execute all tests in all files whose names follow the form ``test_*.py`` or ``\*_test.py``
in the current directory and its subdirectories. More generally, pytest follows :ref:`standard test discovery rules
<test discovery>`.
.. _select-tests:
Specifying which tests to run
------------------------------
Pytest supports several ways to run and select tests from the command-line or from a file
(see below for :ref:`reading arguments from file <args-from-file>`).
**Run tests in a module**
.. code-block:: bash
pytest test_mod.py
**Run tests in a directory**
.. code-block:: bash
pytest testing/
**Run tests by keyword expressions**
.. code-block:: bash
pytest -k 'MyClass and not method'
This will run tests which contain names that match the given *string expression* (case-insensitive),
which can include Python operators that use filenames, class names and function names as variables.
The example above will run ``TestMyClass.test_something`` but not ``TestMyClass.test_method_simple``.
Use ``""`` instead of ``''`` in expression when running this on Windows
.. _nodeids:
**Run tests by collection arguments**
Pass the module filename relative to the working directory, followed by specifiers like the class name and function name
separated by ``::`` characters, and parameters from parameterization enclosed in ``[]``.
To run a specific test within a module:
.. code-block:: bash
pytest tests/test_mod.py::test_func
To run all tests in a class:
.. code-block:: bash
pytest tests/test_mod.py::TestClass
Specifying a specific test method:
.. code-block:: bash
pytest tests/test_mod.py::TestClass::test_method
Specifying a specific parametrization of a test:
.. code-block:: bash
pytest tests/test_mod.py::test_func[x1,y2]
**Run tests by marker expressions**
.. code-block:: bash
pytest -m slow
Will run all tests which are decorated with the ``@pytest.mark.slow`` decorator.
For more information see :ref:`marks <mark>`.
**Run tests from packages**
.. code-block:: bash
pytest --pyargs pkg.testing
This will import ``pkg.testing`` and use its filesystem location to find and run tests from.
.. _args-from-file:
**Read arguments from file**
.. versionadded:: 8.2
All of the above can be read from a file using the ``@`` prefix:
.. code-block:: bash
pytest @tests_to_run.txt
where ``tests_to_run.txt`` contains an entry per line, e.g.:
.. code-block:: text
tests/test_file.py
tests/test_mod.py::test_func[x1,y2]
tests/test_mod.py::TestClass
-m slow
This file can also be generated using ``pytest --collect-only -q`` and modified as needed.
Getting help on version, option names, environment variables
--------------------------------------------------------------
.. code-block:: bash
pytest --version # shows where pytest was imported from
pytest --fixtures # show available builtin function arguments
pytest -h | --help # show help on command line and config file options
.. _durations:
Profiling test execution duration
-------------------------------------
.. versionchanged:: 6.0
To get a list of the slowest 10 test durations over 1.0s long:
.. code-block:: bash
pytest --durations=10 --durations-min=1.0
By default, pytest will not show test durations that are too small (<0.005s) unless ``-vv`` is passed on the command-line.
Managing loading of plugins
-------------------------------
Early loading plugins
~~~~~~~~~~~~~~~~~~~~~~~
You can early-load plugins (internal and external) explicitly in the command-line with the ``-p`` option::
pytest -p mypluginmodule
The option receives a ``name`` parameter, which can be:
* A full module dotted name, for example ``myproject.plugins``. This dotted name must be importable.
* The entry-point name of a plugin. This is the name passed to ``setuptools`` when the plugin is
registered. For example to early-load the :pypi:`pytest-cov` plugin you can use::
pytest -p pytest_cov
Disabling plugins
~~~~~~~~~~~~~~~~~~
To disable loading specific plugins at invocation time, use the ``-p`` option
together with the prefix ``no:``.
Example: to disable loading the plugin ``doctest``, which is responsible for
executing doctest tests from text files, invoke pytest like this:
.. code-block:: bash
pytest -p no:doctest
.. _invoke-other:
Other ways of calling pytest
-----------------------------------------------------
.. _invoke-python:
Calling pytest through ``python -m pytest``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can invoke testing through the Python interpreter from the command line:
.. code-block:: text
python -m pytest [...]
This is almost equivalent to invoking the command line script ``pytest [...]``
directly, except that calling via ``python`` will also add the current directory to ``sys.path``.
.. _`pytest.main-usage`:
Calling pytest from Python code
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can invoke ``pytest`` from Python code directly:
.. code-block:: python
retcode = pytest.main()
this acts as if you would call "pytest" from the command line.
It will not raise :class:`SystemExit` but return the :ref:`exit code <exit-codes>` instead.
If you don't pass it any arguments, ``main`` reads the arguments from the command line arguments of the process (:data:`sys.argv`), which may be undesirable.
You can pass in options and arguments explicitly:
.. code-block:: python
retcode = pytest.main(["-x", "mytestdir"])
You can specify additional plugins to ``pytest.main``:
.. code-block:: python
# content of myinvoke.py
import sys
import pytest
class MyPlugin:
def pytest_sessionfinish(self):
print("*** test run reporting finishing")
if __name__ == "__main__":
sys.exit(pytest.main(["-qq"], plugins=[MyPlugin()]))
Running it will show that ``MyPlugin`` was added and its
hook was invoked:
.. code-block:: pytest
$ python myinvoke.py
*** test run reporting finishing
.. note::
Calling ``pytest.main()`` will result in importing your tests and any modules
that they import. Due to the caching mechanism of python's import system,
making subsequent calls to ``pytest.main()`` from the same process will not
reflect changes to those files between the calls. For this reason, making
multiple calls to ``pytest.main()`` from the same process (in order to re-run
tests, for example) is not recommended.