test_guidelines.rst

Test guidelines

Warning

Currently, the code is only partially tested via doctests. However, tests are very important to ensure that the code is working as expected. Without full test suites we cannot leave the alpha state of development.

Site contents

• Writing tests
  • Consistency checks at runtime
  • Unit tests
  • Integration tests
  • Regression tests
  • Performance tests
• Running tests
  • Doctests
  • Unit tests
  • Integration tests
  • Regression tests
  • Performance tests

Writing tests

Consistency checks at runtime

Implement consistency checks in your code that check at runtime whether the input and output of scripts, functions and other objects is sensible. See :ref:`optimized-mode-label` and :ref:`debug-mode-label`.

Unit tests

For functions that accept arrays of arbitrary dimensions, write unit tests for at least one-, two- and three-dimensional input arrays.

Integration tests

Regression tests

Performance tests

Running tests

Tests are automatically run as GitHub Actions when pushing changes to the upstream repository. However, sometimes it is also useful to run them locally.

Doctests

We use :mod:`doctest` to check whether the code examples given in the documentation work as expected. Sphinx already brings a convenient extension to run doctest on all our example codes. Simply do:

cd path/to/mdtools/docs/
make doctest

Note that you have to install the requirements in :file:`docs/requirements-docs.txt` to be able to run the above command (see :ref:`build-docs-label`).

Unit tests

We plan to use |pytest| to run unit tests. The tests can then be run via

cd path/to/mdtools/
python3 -m pytest .

Integration tests

Regression tests

Performance tests