Skip to content

Unit testing

Jump to bottom
tobiasvanelk edited this page May 3, 2021 · 3 revisions

This document is a summary of the Boost.Test user manual and is intended to serve as a guide for us to write our unit tests. This document will change as our understanding of the Boost Unit Testing Framework (UTF) improves.

Usage

We'll use the example of adding a new plugin to Scroom. (Hopefully this structure is easily adaptable to testing other components.)

First we setup the directory structure:

plugins/
├─ .../
├─ exampleplugin/
│  ├─ src/
│  │  ├─ main.cc
│  │  ├─ exampleplugin.cc
│  │  ├─ exampleplugin.hh
│  ├─ test/
│  │  ├─ main.cc
│  │  ├─ exampleplugin_tests.cc
│  │  ├─ exampleplugin_tests.hh
│  ├─ CMakeLists.txt

The plugin is located in the exampleplugin directory, which contains a CMakeLists.txt file with a CMake subproject.

CMakeLists.txt:

# Here we define the target for the plugin itself
add_library(exampleplugin SHARED)
# Add source files to the target
target_sources(exampleplugin
  PRIVATE src/main.cc
          src/exampleplugin.cc
          src/exampleplugin.hh)
# Link any libraries the plugin needs
# The project_options and project_warnings libraries
# set compile-time options and warnings for the project
# and should probably be included with every target
target_link_libraries(exampleplugin
  PRIVATE project_options
          project_warnings)
# We could link more libraries here, but this example
# doesn't rely on any others

# [Add any other commands here]
# This is also where the plugins call the 'install'
# command, but that has less to do with the unit testing


if(ENABLE_BOOST_TEST) # Variable set from outside
  # Add an *executable* for the tests
  add_executable(exampleplugin_tests)
  # Link up the sources
  target_sources(exampleplugin_tests
    PRIVATE test/main.cc
            test/exampleplugin_tests.cc)
  target_include_directories(exampleplugin_tests PRIVATE src)
  # Link up the related libraries
  target_link_libraries(exampleplugin_tests
    # The boosttesthelper library includes the Boost testing framework
    PRIVATE boosttesthelper
          project_options
          project_warnings
          # Include the target we want to test
          exampleplugin)

  # Add the target as a new test case
  add_test(NAME exampleplugin_tests COMMAND exampleplugin_tests)
endif()

Next we'll add a function to illustrate the test with. (We also add the function to the associated header file.)

src/exampleplugin.cc:

int add(int a, int b) { return a + b; }

Next we define the test module that will contain our test suites and test cases. These can span multiple files, so we define the module once in main.cc:

test/main.cc

// Define the test module BEFORE the #include!
// The name can contain spaces
#define BOOST_TEST_MODULE ExamplePlugin Test Module
// Indicate we're using the dynamically linked library
#define BOOST_TEST_DYN_LINK
#include <boost/test/unit_test.hpp>

Now we can define our test cases:

test/exampleplugin_tests.cc:

// We have to include the unit_test header here too
#include <boost/test/unit_test.hpp>
// Include the plugin's header
#include "exampleplugin.hh"

// This name cannot contain spaces
BOOST_AUTO_TEST_SUITE(ExamplePlugin_Tests)

  BOOST_AUTO_TEST_CASE(test_addition)
  {
    BOOST_CHECK_EQUAL(add(1, 2), 3);
    BOOST_CHECK_EQUAL(add(-1, 2), 1);
    BOOST_CHECK_EQUAL(add(1, -2), -1);
    BOOST_CHECK_EQUAL(add(1, 0), 1);
  }

BOOST_AUTO_TEST_SUITE_END()

To run the tests in CLion, add a new Boost.Test configuration in the Run/Debug Configuration window and add the exampleplugin_tests as its target.

Declaring test cases

A test case contains one or more assertions and is intended to test one aspect of some unit. Test cases can be used with or without parameters.

The most common method of declaring tests is automatically through the BOOST_AUTO_TEST_CASE(test_name) macro:

#define BOOST_TEST_MODULE example
#include <boost/test/included/unit_test.hpp>

BOOST_AUTO_TEST_CASE( free_test_function )
/* Compare with void free_test_function() */
{
  BOOST_TEST( true /* test assertion */ );
}

The macro will create and register the test case with the name free_test_function.

Test cases can be declared using datasets. See the manual for more details.

In order to test template components (so components that use generic types), you can use template test cases. See the manual for more details.

Decorators

Decorators can be used to update various attributes of automatically registered test units. You can specify a list of decorators as the second argument to BOOST_AUTO_TEST_CASE.

Example:

#define BOOST_TEST_MODULE decorator_01
#include <boost/test/included/unit_test.hpp>
namespace utf = boost::unit_test;

BOOST_AUTO_TEST_CASE(test_case1, * utf::label("trivial"))
{
  BOOST_TEST(true);
}

BOOST_AUTO_TEST_CASE(test_case2,
  * utf::label("trivial")
  * utf::label("cmp")
  * utf::description("testing equality of ones"))
{
  BOOST_TEST(1 == 1);
}

Output:

> decorator_01 --run_test=@trivial
Running 2 test cases...

*** No errors detected


> decorator_01 --run_test=@cmp
Running 1 test case...

*** No errors detected

Each decorator is preceded by an asterisk *. You can see the first test case is marked with the label "trivial" and the second test case with the labels "trivial" and "cmp". (Labels can be used for filtering.) The second test case also carries a description decorator.

Fixtures

Fixtures are a way to share setup and cleanup code between different test units. (See the manual.)

Test suites and the test tree

Test cases are leaves in the test tree which is a hierarchy of test suites. The root of this tree is known as the master test suite. It is common for all test cases to reside in the master test suite, so this summary will omit this functionality.

One thing to note is that the master test suite can be used to access command line parameters passed to the test module. You can access these with the following:

boost::unit_test::framework::master_test_suite().argc
boost::unit_test::framework::master_test_suite().argv

API summary for test declarations

https://www.boost.org/doc/libs/1_76_0/libs/test/doc/html/boost_test/tests_organization/summary_of_the_api_for_declaring.html

Writing unit tests

A test case consists of a number of operations interwoven with assertions. The BOOST_TEST macro can be used to test statements in general, but currently in Scroom, the preference appears to lie with using explicit assertion macros, of the form BOOST_<level>. See the reference for a list of all assertions. The <level> refers to the severity level:

  • REQUIRE - When an assertion of this level fails, the entire test case is aborted and flagged as failed. Use this to test requirements that need to be met for the rest of the test case to continue.
  • CHECK - The most common assertion level. When an assertion of this level fails, the test case is flagged as failed, but its execution does continue.
  • WARN - An assertion of this level does not mark a test case as failed when it is false, but will log a warning message. Use this to indicate less important aspects, such as performance, portability, usability, etc.

Example usage:

#define BOOST_TEST_MODULE example
#include <boost/test/included/unit_test.hpp>

BOOST_AUTO_TEST_CASE( test )
{
  int i = 2;
  int j = 1;
  BOOST_REQUIRE_EQUAL( i, j );
}

Clone this wiki locally