Skip to content

[O2-837] Add some usage info for ctest #2712

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sawenzel merged 2 commits into from
Dec 16, 2019
Merged

[O2-837] Add some usage info for ctest #2712

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9d1ce8f
[O2-837] Add some usage info for ctest
aphecetche Dec 15, 2019
c01eba8
Fixing the Codacy issues that can be easily fixed
aphecetche Dec 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 142 additions & 13 deletions doc/CMakeInstructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,42 @@
\\page refdocCMakeInstructions CMake Instructions

# CMake
<!-- vim-markdown-toc GFM -->

- [CMake and CTest tips for AliceO2](#cmake-and-ctest-tips-for-aliceo2)
- [CMake](#cmake)
- [Instructions for contributors (aka developers' documentation)](#instructions-for-contributors-aka-developers-documentation)
- [Typical CMakeLists.txt](#typical-cmakeliststxt)
- [Examples](#examples)
- [Ex1 Adding a basic library](#ex1-adding-a-basic-library)
- [Ex2 Adding a basic library with a Root dictionary](#ex2-adding-a-basic-library-with-a-root-dictionary)
- [Ex3 Adding an executable](#ex3-adding-an-executable)
- [Ex4 Adding a couple of tests](#ex4-adding-a-couple-of-tests)
- [Ex5 Adding a man page](#ex5-adding-a-man-page)
- [CTest](#ctest)
- [Selecting/excluding by test name (-R/-E)](#selectingexcluding-by-test-name--r-e)
- [Selecting/excluding by label (-L/-LE)](#selectingexcluding-by-label--l-le)
- [Speeding up ctest execution](#speeding-up-ctest-execution)

<!-- vim-markdown-toc -->

# CMake and CTest tips for AliceO2

## CMake

> Note that this document describe the [new CMake system](CMakeMigration.md) : the one based on buckets has been discontinued.

## Instructions for contributors (aka developers' documentation)
### Instructions for contributors (aka developers' documentation)

A sub-module `CMakeLists.txt` defines one or more _targets_.
A target generally corresponds to an actual build artifact like a (static or shared) library or an executable. Targets are the cornerstone of any modern cmake build system.

## Typical CMakeLists.txt
### Typical CMakeLists.txt

A typical module's `CMakeLists.txt` contains

- a call to [o2_add_library](../cmake/O2AddLibrary.cmake) to define a library (and its dependencies)
- call(s) to [o2_add_executable](../cmake/O2AddExecutable.cmake) to define one or more executables (and their dependencies)
- call(s) to [o2_add_test](../cmake/O2AddTest.cmake) to define one or more tests (and their dependencies)
- a call to [o2_add_library](../cmake/O2AddLibrary.cmake) to define a library (and its dependencies)
- call(s) to [o2_add_executable](../cmake/O2AddExecutable.cmake) to define one or more executables (and their dependencies)
- call(s) to [o2_add_test](../cmake/O2AddTest.cmake) to define one or more tests (and their dependencies)

Optionally it might contain a call to [o2_target_root_dictionary](../cmake/O2TargetRootDictionary.cmake) if the module's library requires a Root dictionary.

Expand All @@ -25,11 +46,11 @@ Note that despite the parameter name, the `PUBLIC_LINK_LIBRARIES` should refer t

Note also CMakeLists.txt should be considered as code and so the same care you put into writing code (e.g. do not repeat yourself, comments, etc...) should be applied to CMakeLists.txt. Also, like the rest of our code, we can take of the formatting using the [cmake-format](https://github.com/cheshirekow/cmake_format) tool (that tool is certainly not as robust as `clang-format` but it can get most of the job done easily).

## Examples
### Examples

The example outputs below are from a Mac, so the shared library extension is `dylib`. On Linux it would be `so`.

### [Ex1](../Examples/Ex1) Adding a basic library
#### [Ex1](../Examples/Ex1) Adding a basic library

Using the following source dir :

Expand Down Expand Up @@ -80,7 +101,7 @@ And upon install `cmake --build . -- install` (the lonely `--` is not a typo) th
├── lib
│   ├── libO2Ex1.dylib

### [Ex2](../Examples/Ex2) Adding a basic library with a Root dictionary
#### [Ex2](../Examples/Ex2) Adding a basic library with a Root dictionary

Using a slightly modified version of the previous example (the [A.h](../Examples/Ex2/include/Ex2/A.h) now uses ClassDef for instance), we'll now add a Root dictionary :

Expand Down Expand Up @@ -138,7 +159,7 @@ The include installation will be similar to Ex1 : the `LinkDef.h` file is _not_
│   ├── libO2Ex2.dylib
│   ├── libO2Ex2.rootmap

### [Ex3](../Examples/Ex3) Adding an executable
#### [Ex3](../Examples/Ex3) Adding an executable

Adding an executable to previous example :

Expand Down Expand Up @@ -201,7 +222,7 @@ Third, the created executable can be launched "as is" from the build tree, witho

That is because the [RPATH](https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/RPATH-handling) was correctly set by CMake for the build tree. It should be set correctly also when installing.

### [Ex4](../Examples/Ex4) Adding a couple of tests
#### [Ex4](../Examples/Ex4) Adding a couple of tests

Let's add two basic test2 to our previous example.

Expand Down Expand Up @@ -232,7 +253,7 @@ Those two test executables will be under `stage/bin` with a name starting with `
├── libO2Ex3.dylib
└── libO2Ex4.dylib

By default tests are not be installed, unless the `INSTALL` option is given to `o2_add_test`. So in the installation zone only the first test will be available, under the `tests` subdirectory. So the full installation of our 4 examples would give :
By default tests are _not_ installed, unless the `INSTALL` option is given to `o2_add_test`. So in the installation zone only the first test will be available, under the `tests` subdirectory. So the full installation of our 4 examples would give :

../install-Debug/
├── bin
Expand Down Expand Up @@ -311,7 +332,7 @@ Tests can also be _excluded_ based on label (`-LE`) or name (`-RE`).
dummy = 0.07 sec*proc (1 test)
fast = 0.07 sec*proc (1 test)

### [Ex5](../Examples/Ex5) Adding a man page
#### [Ex5](../Examples/Ex5) Adding a man page

If a module provides one or more executables, it might be of interest for the users of those executables to have access to a man page for them. Ex5 illustates that use case.

Expand All @@ -328,3 +349,111 @@ The [man page](ManPages.md) is created using :
o2_target_man_page([targetName] NAME ex5 SECTION 7)

where `NAME xx` refers to a file `doc/xx.[SECTION].in`, and the actual `targetName` can be found from the base target name (ex5 in that case) using the [o2_name_target](../cmake/O2NameTarget.cmake) function.

## CTest

In the build directory of O2, if you launch the `ctest` command, all the O2 tests will be ran, which is not always what you want/need, in particular during development.

As shown above in [Ex4](#ex4-adding-a-couple-of-tests), there are fortunately various ways to narrow the set of tests which are ran by `ctest`.

Note that in all the commands below, the `-N` option is used to show the list of test that would be selected, without running them. Remove that option to actually run the tests.

### Selecting/excluding by test name (-R/-E)

Probably the simplest is to select the test name, using the `-R` (regexp) option :

```shell
> ctest -N -R test/Contour
Test #269: Detectors/MUON/MCH/Contour/test/Contour.cxx
Test #270: Detectors/MUON/MCH/Contour/test/ContourCreator.cxx

> ctest -N -R "test/Contour\.(.)"
Test #269: Detectors/MUON/MCH/Contour/test/Contour.cxx

```

Instead of selecting the tests to run, you can invert the logic and exclude tests matching a regular expression using `-E` instead of `-R`.

### Selecting/excluding by label (-L/-LE)

Assuming the tests have been labelled, you can use `-L` (to include) or `-LE` (to exclude) tests based on their labels. The full list of defined labels can be shown using the `--print-labels` option.

```shell
> ctest -N -L mch
Test #268: Detectors/MUON/MCH/Contour/test/BBox.cxx
Test #269: Detectors/MUON/MCH/Contour/test/Contour.cxx
Test #270: Detectors/MUON/MCH/Contour/test/ContourCreator.cxx
Test #271: Detectors/MUON/MCH/Contour/test/Edge.cxx
Test #272: Detectors/MUON/MCH/Contour/test/Interval.cxx
Test #273: Detectors/MUON/MCH/Contour/test/Polygon.cxx
Test #274: Detectors/MUON/MCH/Contour/test/SegmentTree.cxx
Test #275: Detectors/MUON/MCH/Contour/test/Vertex.cxx
Test #276: Detectors/MUON/MCH/Simulation/macros/drawMCHGeometry.C
Test #277: Detectors/MUON/MCH/Simulation/macros/drawMCHGeometry.C_compiled
```

### Speeding up ctest execution

Finally, note that `ctest` can run tests in parallel (unless those that are explicitely marked as not safe to run in parallel) using the `-j n` option, where `n` is the max number of tests to run in parallel.

Serial execution :

```shell
> ctest -L mch -LE long -E Raw
Start 265: Detectors/MUON/MCH/Contour/test/BBox.cxx
1/8 Test #265: Detectors/MUON/MCH/Contour/test/BBox.cxx ............. Passed 0.04 sec
Start 266: Detectors/MUON/MCH/Contour/test/Contour.cxx
2/8 Test #266: Detectors/MUON/MCH/Contour/test/Contour.cxx .......... Passed 0.05 sec
Start 267: Detectors/MUON/MCH/Contour/test/ContourCreator.cxx
3/8 Test #267: Detectors/MUON/MCH/Contour/test/ContourCreator.cxx ... Passed 0.05 sec
Start 268: Detectors/MUON/MCH/Contour/test/Edge.cxx
4/8 Test #268: Detectors/MUON/MCH/Contour/test/Edge.cxx ............. Passed 0.05 sec
Start 269: Detectors/MUON/MCH/Contour/test/Interval.cxx
5/8 Test #269: Detectors/MUON/MCH/Contour/test/Interval.cxx ......... Passed 0.05 sec
Start 270: Detectors/MUON/MCH/Contour/test/Polygon.cxx
6/8 Test #270: Detectors/MUON/MCH/Contour/test/Polygon.cxx .......... Passed 0.05 sec
Start 271: Detectors/MUON/MCH/Contour/test/SegmentTree.cxx
7/8 Test #271: Detectors/MUON/MCH/Contour/test/SegmentTree.cxx ...... Passed 0.05 sec
Start 272: Detectors/MUON/MCH/Contour/test/Vertex.cxx
8/8 Test #272: Detectors/MUON/MCH/Contour/test/Vertex.cxx ........... Passed 0.05 sec

100% tests passed, 0 tests failed out of 8

Label Time Summary:
mch = 0.38 sec*proc (8 tests)
muon = 0.38 sec*proc (8 tests)

Total Test time (real) = 0.49 sec

```

Parallel execution (on a machine with 16 cores) :

```shell
> ctest -L mch -LE long -E Raw -j 16
Start 270: Detectors/MUON/MCH/Contour/test/Polygon.cxx
Start 271: Detectors/MUON/MCH/Contour/test/SegmentTree.cxx
Start 268: Detectors/MUON/MCH/Contour/test/Edge.cxx
Start 267: Detectors/MUON/MCH/Contour/test/ContourCreator.cxx
Start 269: Detectors/MUON/MCH/Contour/test/Interval.cxx
Start 266: Detectors/MUON/MCH/Contour/test/Contour.cxx
Start 265: Detectors/MUON/MCH/Contour/test/BBox.cxx
Start 272: Detectors/MUON/MCH/Contour/test/Vertex.cxx
1/8 Test #270: Detectors/MUON/MCH/Contour/test/Polygon.cxx .......... Passed 0.08 sec
2/8 Test #271: Detectors/MUON/MCH/Contour/test/SegmentTree.cxx ...... Passed 0.08 sec
3/8 Test #268: Detectors/MUON/MCH/Contour/test/Edge.cxx ............. Passed 0.08 sec
4/8 Test #269: Detectors/MUON/MCH/Contour/test/Interval.cxx ......... Passed 0.08 sec
5/8 Test #267: Detectors/MUON/MCH/Contour/test/ContourCreator.cxx ... Passed 0.08 sec
6/8 Test #266: Detectors/MUON/MCH/Contour/test/Contour.cxx .......... Passed 0.08 sec
7/8 Test #265: Detectors/MUON/MCH/Contour/test/BBox.cxx ............. Passed 0.08 sec
8/8 Test #272: Detectors/MUON/MCH/Contour/test/Vertex.cxx ........... Passed 0.08 sec

100% tests passed, 0 tests failed out of 8

Label Time Summary:
mch = 0.65 sec*proc (8 tests)
muon = 0.65 sec*proc (8 tests)

Total Test time (real) = 0.19 sec

```