44Developer's tips for testing
55============================
66
7- Matplotlib has a testing infrastructure based on pytest _, making it easy to
8- write new tests. The tests are in :mod: `matplotlib.tests `, and customizations
9- to the pytest testing infrastructure are in :mod: `matplotlib.tests.conftest `
10- and :mod: `matplotlib.testing `. (There is other old testing cruft around, please
11- ignore it while we consolidate our testing to these locations.)
7+ Matplotlib's testing infrastructure depends on pytest _. The tests are in
8+ :file: `lib/matplotlib/tests `, and customizations to the pytest testing
9+ infrastructure are in :mod: `matplotlib.testing `.
1210
1311.. _pytest : http://doc.pytest.org/en/latest/
1412.. _mock : https://docs.python.org/dev/library/unittest.mock.html>
1513.. _Ghostscript : https://www.ghostscript.com/
1614.. _Inkscape : https://inkscape.org
1715.. _pytest-cov : https://pytest-cov.readthedocs.io/en/latest/
1816.. _pytest-pep8 : https://pypi.python.org/pypi/pytest-pep8
17+ .. _pytest-xdist : https://pypi.python.org/pypi/pytest-xdist
18+ .. _pytest-timeout : https://pypi.python.org/pypi/pytest-timeout
1919
2020Requirements
2121------------
@@ -31,6 +31,8 @@ Optionally you can install:
3131
3232 - pytest-cov _ to collect coverage information
3333 - pytest-pep8 _ to test coding standards
34+ - pytest-timeout _ to limit runtime in case of stuck tests
35+ - pytest-xdist _ to run tests in parallel
3436
3537
3638Building matplotlib for image comparison tests
@@ -69,11 +71,11 @@ commands, such as:
6971
7072======================== ===========
7173``--pep8 `` Perform pep8 checks (requires pytest-pep8 _)
72- ``--no- network `` Disable tests that require network access
74+ ``-m "not network" `` Disable tests that require network access
7375======================== ===========
7476
7577Additional arguments are passed on to pytest. See the pytest documentation for
76- supported arguments. Some of the more important ones are given here:
78+ ` supported arguments `_ . Some of the more important ones are given here:
7779
7880============================= ===========
7981``--verbose `` Be more verbose
@@ -84,26 +86,26 @@ supported arguments. Some of the more important ones are given here:
8486``--capture=no `` or ``-s `` Do not capture stdout
8587============================= ===========
8688
87- To run a single test from the command line, you can provide a dot-separated
88- path to the module, optionally followed by the function separated by two
89- colons, e.g., (this is assuming the test is installed )::
89+ To run a single test from the command line, you can provide a file path,
90+ optionally followed by the function separated by two colons, e.g., (tests do
91+ not need to be installed, but Matplotlib should be )::
9092
91- python tests.py matplotlib. tests. test_simplification::test_clipping
93+ py.test lib/ matplotlib/ tests/ test_simplification.py ::test_clipping
9294
93- or by passing a file path, optionally followed by the function separated by two
94- colons, e.g., (tests do not need to be installed, but Matplotlib should be) ::
95+ or, if tests are installed, a dot-separated path to the module, optionally
96+ followed by the function separated by two colons, such as ::
9597
96- python tests.py lib/ matplotlib/ tests/test_simplification.py ::test_clipping
98+ py.test --pyargs matplotlib. tests.test_simplification ::test_clipping
9799
98100If you want to run the full test suite, but want to save wall time try
99101running the tests in parallel::
100102
101- python tests.py --capture=no --verbose -n 5
103+ py.test --verbose -n 5
102104
103105Depending on your version of Python and pytest-xdist, you may need to set
104106``PYTHONHASHSEED `` to a fixed value when running in parallel::
105107
106- PYTHONHASHSEED=0 python tests.py --capture=no --verbose -n 5
108+ PYTHONHASHSEED=0 py.test --verbose -n 5
107109
108110An alternative implementation that does not look at command line arguments
109111and works from within Python is to run the tests from the Matplotlib library
@@ -112,16 +114,8 @@ function :func:`matplotlib.test`::
112114 import matplotlib
113115 matplotlib.test()
114116
115- .. hint ::
116-
117- To run the tests you need to install pytest and mock if using python 2.7::
118-
119- pip install pytest
120- pip install mock
121117
122-
123- .. _pytest-xdist : https://pypi.python.org/pypi/pytest-xdist
124- .. _pytest-timeout : https://pypi.python.org/pypi/pytest-timeout
118+ .. _supported arguments : http://doc.pytest.org/en/latest/usage.html
125119
126120
127121Writing a simple test
@@ -140,10 +134,10 @@ Pytest determines which functions are tests by searching for files whose names
140134begin with ``"test_" `` and then within those files for functions beginning with
141135``"test" `` or classes beginning with ``"Test" ``.
142136
143- Tests that have side effects that need to be cleaned up, such as created
144- figures using the pyplot interface or modified rc params, will be automatically
145- reset by the pytest fixture
146- :func: ` ~matplotlib.tests.conftest.mpl_test_settings ` .
137+ Some tests have internal side effects that need to be cleaned up after their
138+ execution (such as created figures or modified rc params). The pytest fixture
139+ :func: ` ~matplotlib.testing.conftest.mpl_test_settings ` will automatically clean
140+ these up; there is no need to do anything further .
147141
148142
149143Writing an image comparison test
0 commit comments