Skip to content

Commit

Permalink
doc: update test suite page for ninja and other recent changes
Browse files Browse the repository at this point in the history
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
(cherry picked from commit b15b666)
  • Loading branch information
whot committed Nov 14, 2017
1 parent 03cd377 commit e7be909
Showing 1 changed file with 69 additions and 34 deletions.
103 changes: 69 additions & 34 deletions doc/test-suite.dox
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
/**
@page test-suite libinput test suite

The libinput test suite is based on
[Check](http://check.sourceforge.net/doc/check_html/) and runs automatically
during `make check`. Check itself is wrapped into a libinput-specific test
suite called *litest*. Tests are found in `$srcdir/test/`, the main test
suite is `libinput-test-suite-runner`.

The test suite has a make-like job control enabled by the `-j` or `--jobs`
flag and will fork off as many parallel processes as given by this flag. The
default if unspecified is 8. When debugging a specific test case failure it
is recommended to employ test filtures (see @ref test-filtering) and disable
parallel tests. The test suite automatically disables parallel make when run
in gdb.
libinput ships with a number of tests all run automatically on `ninja test`.
The primary test suite is the `libinput-test-suite-runner`. When testing,
the `libinput-test-suite-runner` should always be invoked to check for
behavior changes.

The test suite runner uses
[Check](http://check.sourceforge.net/doc/check_html/) underneath the hood
but most of the functionality is abstracted into *litest* wrappers.

The test suite runner has a make-like job control enabled by the `-j` or
`--jobs` flag and will fork off as many parallel processes as given by this
flag. The default if unspecified is 8. When debugging a specific test case
failure it is recommended to employ test filtures (see @ref test-filtering)
and disable parallel tests. The test suite automatically disables parallel
make when run in gdb.

@section test-config X.Org config to avoid interference

Expand All @@ -28,35 +31,67 @@ with your desktop.

Most tests require the creation of uinput devices and access to the
resulting `/dev/input/eventX` nodes. Some tests require temporary udev rules.
<b>This usually requires the tests to be run as root</b>.
<b>This usually requires the tests to be run as root</b>. If not run as
root, the test suite runner will exit with status 77, interpreted as
"skipped" by ninja.

@section test-filtering Selective running of tests

litest's tests are grouped by test groups and devices. A test group is e.g.
"touchpad:tap" and incorporates all tapping-related tests for touchpads.
Each test function is (usually) run with one or more specific devices.
The `--list` commandline argument shows the list of suites and tests.
litest's tests are grouped into test groups, test names and devices. A test
group is e.g. "touchpad:tap" and incorporates all tapping-related tests for
touchpads. Each test function is (usually) run with one or more specific
devices. The `--list` commandline argument shows the list of suites and
tests. This is useful when trying to figure out if a specific test is
run for a device.

@code
$ ./test/libinput-test-suite-runner --list
device:wheel:
wheel only
blackwidow
device:invalid devices:
no device
device:group:
no device
logitech trackball
MS surface cover
mouse_roccat
wheel only
blackwidow
...
pointer:left-handed:
pointer_left_handed_during_click_multiple_buttons:
trackpoint
ms-surface-cover
mouse-wheelclickcount
mouse-wheelclickangle
low-dpi-mouse
mouse-roccat
mouse-wheel-tilt
mouse
logitech-trackball
cyborg-rat
magicmouse
pointer_left_handed_during_click:
trackpoint
ms-surface-cover
mouse-wheelclickcount
mouse-wheelclickangle
low-dpi-mouse
mouse-roccat
mouse-wheel-tilt
mouse
logitech-trackball
cyborg-rat
litest-magicmouse-device
pointer_left_handed:
trackpoint
ms-surface-cover
mouse-wheelclickcount
mouse-wheelclickangle
low-dpi-mouse
mouse-roccat
mouse-wheel-tilt
mouse
...
@endcode

In the above example, the "device:wheel" suite is run for the "wheel only" and
the "blackwidow" device. Both devices are automatically instantiated through
uinput by litest. The "no device" entry signals that litest does not
instantiate a uinput device for a specific test (though the test itself may
In the above example, the "pointer:left-handed" suite contains multiple
tests, e.g. "pointer_left_handed_during_click" (this is also the function
name of the test, making it easy to grep for). This particular test is run
for various devices including the trackpoint device and the magic mouse
device.

The "no device" entry signals that litest does not instantiate a uinput
device for a specific test (though the test itself may
instantiate one).

The `--filter-test` argument enables selective running of tests through
Expand Down Expand Up @@ -93,7 +128,7 @@ environment variable, if set, also enables verbose mode.

@code
$ ./test/libinput-test-suite-runner --verbose
$ LITEST_VERBOSE=1 make check
$ LITEST_VERBOSE=1 ninja test
@endcode

*/

0 comments on commit e7be909

Please sign in to comment.