Skip to content

Commit

Permalink
update documentation for developing
Browse files Browse the repository at this point in the history
* including new `ctest` as runner
* some tips which options should be used
  • Loading branch information
felixschurk committed May 9, 2024
1 parent 9df2550 commit 5c3a92e
Showing 1 changed file with 35 additions and 16 deletions.
51 changes: 35 additions & 16 deletions doc/devel/contrib/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Satisfy the Requirements:

* CMake 3.0 or later
* CMake 3.22 or later
* gcc 7.0 or later, clang 6.0 or later, or a compiler with full C++17 support
* libuuid (if not on macOS)
* Rust 1.64.0 or higher (hint: use https://rustup.rs/ instead of using your system's package manager)
Expand Down Expand Up @@ -49,28 +49,47 @@ cmake --build build-clang
```

## Run the Test Suite:
First switch to the test directory:

```
$ cd build/test
```
Then you can run all tests, showing details, with
```
$ make VERBOSE=1 test
```
Alternately, run the tests with the details hidden in `all.log`:
For running the test suite [ctest](https://cmake.org/cmake/help/latest/manual/ctest.1.html) is used.
Before one can run the test suite the `task_executable` must be built.
After that also the `build_tests` executable must be build, which can be done over:
```sh
cmake --build build --target build_tests
```
$ ./run_all
Again you may also use the `-j <number-of-jobs>` option for parallel builds.

Now one can invoke `ctest` to run the tests.
```sh
ctest --test-dir build
```
Either way, you can get a summary of any test failures with:
This would run all the test in serial and might take some time.

### Running tests in parallel
```sh
ctest --test-dir build -j <number-of-jobs>
```
$ ./problems

Further it is adviced to add the `--output-on-failure` option to `ctest`, to recieve a verbose output if a test is failing as well as the `--rerun-failed` flag, to invoke in subsequent runs only the failed ones.

### Running specific tests
For this case one can use the `-R <regex>` or `--tests-regex <regex>` option to run only the tests matching the regular expression.
Running only the `cpp` tests can then be achieved over
```sh
ctest --test-dir build -R cpp
```
You can run a single test suite, with source file `foo.test.cpp` or `foo.test.py`, with
or running the `variant_*` tests
```sh
ctest --test-dir build -R variant
```
$ make foo.test

### Repeating a test case
In order to find sporadic test failures the `--repeat` flag can be used.
```sh
ctest --test-dir build -R cpp --repeat-until-fail 10
```

There are more options to `ctest` such as `--progress`, allowing to have a less verbose output.
They can be found in the [ctest](https://cmake.org/cmake/help/latest/manual/ctest.1.html) man page.

Note that any development should be performed using a git clone, and the current development branch.
The source tarballs do not reflect HEAD, and do not contain the test suite.
Follow the [GitHub flow](https://docs.github.com/en/get-started/quickstart/github-flow) for creating a pull request.

0 comments on commit 5c3a92e

Please sign in to comment.