Unit_Testing

Unit Testing

We strongly encourage you to always unit test your code. When you are developing within the gunns & ms-utils repositories, or developing your own links with the intent of eventually including them in a GUNNS release, it helps us a lot if the code is well-tested. For us to include your updates they must meet the coding & testing standards established by the existing baseline. Updates submitted to us for inclusion in a GUNNS release should meet the requirements listed below.

We use CppUnit as our unit testing automation tool, Valgrind and AddressSanitizer (ASan) as our memory error checkers, and LCOV as our code coverage tool. Each source folder in the gunns & ms-utils repositories has a test/ subfolder containing the unit test classes and makefiles to run them. Example: gunns/core/test/.

The tests are set up to work in the GUNNS environment, so you need to run: source bin/cshrc from your gunns/ folder first, or use an equivalent setup.

Running Individual Test Folders

To run an individual test folder’s tests with Valgrind+LCOV, within the test/ folder do:

$ make clean; make cleanlib; make

This produces the CppUnit pass/fail results & summary, a Valgrind memory checker error report, and an LCOV code coverage report.

To run an individual test folder’s tests with AddressSanitizer, within the test/ folder do:

$ make clean; make cleanlib; make ASAN=‘1’

This produces the CppUnit pass/fail results & summary, and memory errors found by ASan are output to std::err and logged in the outputs/ folder in files asan.log*.

The make cleanlib step is only necessary if you have code changes to the actual GUNNS code (not unit test code), or if you are switching between Valgrind & ASan testing. If you are re-running a test without having made any changes to the GUNNS code, or are not switching between Valgrind/ASan, then you can skip this step.

Running All Unit Tests

You can run the unit tests with Valgrind & LCOV for all gunns and relevant ms-utils folders in one go by running this script from the gunns/ folder:

$ test/make_all_ut.sh

This outputs a results summary to gunns/make_all_ut.out. For each test folder this indicates an overall OK result, or failures to build, test failures, or memory errors. View the unit-tests-valgrind.log file in individual test folders for more detailed results.

You can run the unit tests with AddressSanitizer for all gunns and relevant ms-utils folders in one go by running this script from the gunns/ folder:

$ test/make_all_ut_asan.sh

This outputs a results summary to gunns/make_all_ut.out. For each test folder this indicates an overall OK result, or failures to build, test failures, or memory errors. View the outputs/asan.log* files in individual test folders for more detailed ASan error descriptions.

Requirements

Before any release, we strive to meet these standards with our unit test results. We allow some exceptions for cases where there hasn’t been time to keep the unit tests up-to-date with the code and the failures are well understood and have minimal impact.

• Zero test failures.
• 100% test line coverage, and yellow or green functions & branches coverage
• Zero Valgrind errors.
• Zero AddressSanitizer errors.