For Developers: Test suite

This page explains how to run the test suites that are part of the repository.

At its simplest, the test suite can be invoked as part of the normal build process:

$ meson builddir
$ ninja -C builddir
$ sudo ninja -C builddir test

While the test suite is also integrated with the autotools make check functionality, using meson is strongly recommended.

C-based Test suite

The primary function of the C-based test suite is to add unit-tests to components of the code (e.g. helper functions). It is not an integration test suite, i.e. it does not verify the driver itself.

The driver code itself has a number of these unit tests throughout the code that are conditionally enabled for the test suite. Assuming a meson build directory of builddir, these tests can be run directly:

$ export LD_LIBRARY_PATH=$PWD/builddir
$ ./builddir/wacom-tests
- running test_get_scroll_delta           SUCCCESS
- running test_get_wheel_button           SUCCCESS
- running test_common_ref                 SUCCCESS
...

The test suite can be run through gdb.

Adding tests to the C test suite

As a general rule, to add a new test look at the existing tests and copy their behavior. Adding new test is done by adding a TEST_CASE(name) function in the core driver code (i.e. in src/, git grep for examples). The tests are discovered by the test suite automatically. The test suite does not currently support filtering of tests but this could be added in the future if required.

The test suite itself is very simple, see the wacom-test-suite.h and wacom-test-suite.c files for the source. For Developers: Automated Testing has more information on how it all fits together.

Python-based Test suite

The Python-based test suite is ideal for integration tests. It uses the GObject-based libgwacom helper library to expose the driver through Python's FFI. The test suite can be run directly via pytest. It requires permissions to create /dev/uinput devices and read from /dev/input/event nodes as well as some environment variables to be set up correctly. The wacom-test-env.sh script does this automatically:

$ sudo ./test/wacom-test-env.sh
root@localhost $ pytest -vv
test/test_wacom.py::test_proximity PASSED                      [ 10%]
test/test_wacom.py::test_relative_motion[NONE] PASSED          [ 20%]
test/test_wacom.py::test_relative_motion[CW] PASSED            [ 30%]
...  

The test suite employs pytest, see the pytest documentation for details. The most useful switch is -k to selectively run a test with the given substring (e.g. pytest -vv -k test_proximity).

On older distributions (e.g. Ubuntu 20.04) you must run pytest-3 instead of pytest. The pytest binary on those distributions may point to a Python 2.x-compatible version which throws syntax errors. See this PR.

Adding tests to the Python test suite

As a general rule, to add a new test look at the existing tests and copy their behavior.

Most tests work by setting up a Device (based on a recording in test/devices) and a Monitor (to collect events). Raw evdev events are written to the uinput device and driver events are collected by the Monitor (e.g. motion, proximity, ...). Then the tests assert that the event sequence received matches expectations. See the test/test_wacom.py file for existing tests and test/__init__.py for the Device and Monitor classes.

Non-exhaustive set of libs to install to run meson builddir and the tests

On Ubuntu:

sudo apt install libgirepository1.0-dev libevdev-dev cmake meson libglib2.0-dev python3-pip python-pytest
pip3 install libevdev pytest

pip packages should be installed as root because the tests are run as root.

I find it helpful to run pytest (as root) as follows pytest -s -vv --log-level=DEBUG