Permalink
Browse files

[web] Finished testing section in devguide.

  • Loading branch information...
bilke committed Feb 26, 2018
1 parent 88ba0e0 commit 95f0f59123559a0512a065ef32b374f9adb4106f
@@ -0,0 +1,59 @@
+++
date = "2018-02-26T11:00:13+01:00"
title = "Jenkins"
author = "Lars Bilke"
weight = 1022

[menu]
[menu.devguide]
parent = "testing"
+++

## Introduction

[Jenkins](https://jenkins.io/) is a self-hosted powerful [Continuous Integration](../../development-workflows/continuous-integration) server. Our server is available at <https://jenkins.opengeosys.org>.

To configure the tasks of the CI system we mainly use [Jenkins Pipeline](https://jenkins.io/doc/pipeline) jobs which are defined in a [script inside the OGS source code](https://github.com/ufz/ogs/tree/master/Jenkinsfile). Scripting and versioning the configuration together with the source code is very powerful, e.g. if you introduce a new OGS CMake configuration in a pull request even the change of the CI jobs configuration can be part of the pull request.

## Jenkins Pipeline

In the [Jenkinsfile](https://github.com/ufz/ogs/tree/master/Jenkinsfile) you can see lots of `stage`-blocks. Each stage is a test sub-job running on different test nodes. Inside each stage there are `steps` doing things such as configuring the build with CMake and building specific targets:

```java
stage('Win') {
agent {label 'win && conan' } // Runs on a Windows node with Conan installed
environment {
MSVC_VERSION = '2017' // Sets Visual Studio version
}
steps {
script {
configure { // Runs CMake with options
cmakeOptions = '-DOGS_USE_CONAN=ON '
}
build { } // Builds the project
build { target="tests" } // Runs unit tests
build { target="ctest" } // Runs benchmark tests
}
}
}
```

Functions such as `configure` and `build` are defined in a separate [Pipeline library](https://github.com/ufz/jenkins-pipeline-library).

Each pipeline run is visualized as follows:

![](../Jenkins-Pipeline.png)

The status of all stages are shown. You can select one stage (e.g. *Win*) and you will see all steps inside that stage at the bottom list. You can expand the log output of a specific step by clicking the small arrow on the left.

On the *Artefacts*-tab in the top navigation bar are links to the compiled binaries and generated websites (*Doxygen* code documentation and the *Web* site containing also the benchmark documentation).

All of this is part of the modern Jenkins UI called *Blue Ocean*. You can always switch to the classic more comprehensive UI by clicking the arrow inside square button in the top navigation bar.

## Automatic testing

The master-branch of the the UFZ repository as well as all pull requests on that repo are automatically tested. See [this page](https://jenkins.opengeosys.org/blue/organizations/jenkins/ufz%2Fogs/pr) for all currently tested PRs:

![](../Jenkins-PR-Overview.png)

Automatic testing can be easily setup for all your branches on your clone of OGS. See [the Continuous Integration page](../../getting-started/continuous-integration) for the procedure.
@@ -0,0 +1,42 @@
+++
date = "2018-02-26T11:00:13+01:00"
title = "Executable Testing / Test Data"
author = "Lars Bilke"
weight = 1023

[menu]
[menu.devguide]
parent = "testing"
+++

## Introduction

An executable test can be run with several *wrappers*, e.g. with `valgrind`, `memcheck` or a simple `time`-measurement. Each wrapper run can then be validated (called *testers*), e.g. with file-`diff`s or -`grep`s. This can be configured in CMake:

```cmake
AddTest(
NAME GroundWaterFlowProcess
PATH ${ExternalData_SOURCE_ROOT}/Elliptic/quad_20x10_GroundWaterFlow
EXECUTABLE ogs # optional
EXECUTABLE_ARGS quad_20x10_GroundWaterFlow.prj
WRAPPER time # optional
TESTER diff # optional
DIFF_DATA quad_20x10_constMat0.mesh.vtu quad_20x10_left_right.gml # optional
)
```

Tests are then run with `make ctest` or for more verbose output with `ctest -VV` (you may also use other [ctest options](https://cmake.org/cmake/help/v3.4/manual/ctest.1.html)). If the checker has some errors they are displayed.

The functionality is very flexible and more wrappers and checker can be added later on. e.g. for running some statistics on output files and comparing them with statistics from reference files.

## Test Data

Test data is stored with git lfs in `Tests/Data`. Generated test output files should be found in `[build-dir]/Tests/Data`.

In the OGS-cli outputting to `[build-dir]/Tests/Data` is already handled (via the `-o` parameter). For other executables you have to implement this, e.g. a with parameter specifying the output directory.

In code `BaseLib::BuildInfo::data_path` (from `BuildInfo.h`) references the data source directory and `BaseLib::BuildInfo::data_binary_path` references the data output directory.

For adding new data files make sure you have [setup git lfs](../../getting-started/prerequisites) already. Then simply commit the new files as usual. For pushing you need to have [setup an account](TODO: Gitlab link) on our own GitLab server as the lfs files are stored there (due to bandwidth limitations on GitHub). When asked for GitLab credentials on pushing use your GitLab account name (should be the same as the GitHub account name) and your created GitLab personal access token ([see the GitLab Setup page](TODO: Gitlab link)).

Check this [in-depth tutorial](https://www.atlassian.com/git/tutorials/git-lfs) to learn more about git lfs.
@@ -0,0 +1,22 @@
+++
date = "2018-02-26T11:00:13+01:00"
title = "Unit Testing"
author = "Lars Bilke"
weight = 1021

[menu]
[menu.devguide]
parent = "testing"
+++

## Introdution

[Unit testing](http://en.wikipedia.org/wiki/Unit_testing) can be used for testing small pieces of functionality. We use [googletest](https://code.google.com/p/googletest/wiki/V1_6_Primer) as our test framework.

## Running Tests

Googletest is already included in the OGS source code. Running tests is as simple as building the `tests`-target.

## Writing Tests

All tests are located inside the *Tests*-directory in library-specific subfolders. To write a new test you simply create a new *.cpp*-file and use the googletest macros.

0 comments on commit 95f0f59

Please sign in to comment.