.. currentmodule:: qinfer
Unit and Documentation Testing
As described in :ref:`docs_devguide_snippets`, Sphinx integrates with
Python's :mod:`doctest` module to help ensure that both the documentation
and underlying library are correct. Doctests consist of short snippets
of code along with their expected output. A doctest passes if the
actual output of the snippet matches its expected output. For instance,
a doctest that
1 + 1 correctly produces
2 could be written as:
Below, we show an example of addition in practice: >>> 1 + 1 2 Later, we will consider more advanced operators.
The blank lines above and below the doctest separate it from the surrounding text, while the expected output appears immediately below the relevant code.
To run the doctests in the QInfer documentation using Linux or OS X:
$ cd doc/ $ make doctest
To run the doctests on Windows using PowerShell, use
PS > cd doc/ PS > .\make doctest
As with the unit tests, doctests are automatically run on pull requests, to help ensure the correctness of contributed documentation.
A doctest snippet may be annotated with one or more comments that change
the behavior of that test. The :mod:`doctest` documentation goes into far
more detail, but the two we will commonly need are
# doctest: +SKIP
# doctest: +ELLIPSIS. The former causes a test to be skipped entirely.
Skipping tests can be useful if the output of a doctest is random, for instance.
+ELLIPSIS, causes any ellipsis (
...) in the expected output
to act as a wild card. For instance, both of the following doctests would pass:
>>> print([1, 2, 3]) # doctest: +ELLIPSIS [1, ..., 3] >>> print([1, 2, 3, 4, 'foo', 3]) # doctest: +ELLIPSIS [1, ..., 3]
There are a few annoyances that come along with writing tests based on string-equivalence of outputs, in particular for a cross-platform and 2/3 compatible library. In particular:
NumPy, 2to3 and
class: Python 2 and 3 differ on whether the type of NumPy :class:`~numpy.ndarray` prints as
>>> print(type(np.array())) <type 'numpy.ndarray'> # Python 2 <class 'numpy.ndarray'> # Python 3
Thus, to write a doctest that checks if something is a NumPy array or not, it is preferred to use an isinstance check instead:
>>> isinstance(np.array(), np.ndarray) True
Though this sacrifices some on readability, it gains on portability and correctness.
int versus long in array shapes: The Windows and Linux versions of NumPy behave differently with respect to when a NumPy shape is represented as a tuple of int versus a tuple of long values. This can cause doctests to choke on spurious
>>> print(np.zeros((1, 10)).shape) (1, 10) # tuple of int (1L, 10L) # tuple of long
Since int and long respect
==, however, the same trick as above can help:
>>> np.zeros((1, 10)).shape == (1, 10) True