Improve code coverage

[nahueespinosa]

Description

In the latest version of beluga (fb8294d at the time of this post), the code coverage report shows:

Summary: Line Rate = 59% (861 / 1467), Branch Rate = 16% (915 / 5548)

We should start working towards improving these metrics for existing code. In particular, beluga_amcl does not have any tests for the ROS 2 node implementation.

Definition of done

• Line rate coverage greater than 90%.
• Branch rate coverage greater than 90%.

Additional considerations

Reference: https://testing.googleblog.com/2020/08/code-coverage-best-practices.html

Focusing on getting the number as close as possible to 100% leads to a false sense of security. It could also be wasteful, burning machine cycles and creating technical debt from low-value tests that now need to be maintained. Bad code being pushed to production due to missing tests could happen either because (a) your tests did not cover a specific path of code, a test gap that is easy to identify with code coverage analysis, or (b) because your tests did not cover a specific edge case in an area that did have code coverage, which is difficult or impossible to catch with code coverage analysis. Code coverage does not guarantee that the covered lines or branches have been tested correctly, it just guarantees that they have been executed by a test. Be mindful of copy/pasting tests just for the sake of increasing coverage, or adding tests with little actual value, to comply with the number.

[glpuga]

Do we know which parts are the most under-tested? different code sections may have different reasons for that.

JIC templates may also add subtle distortions in coverage measurement, an so do inline functions:

• https://stackoverflow.com/questions/9666800/getting-useful-gcov-results-for-header-only-libraries
• https://stackoverflow.com/questions/55058715/how-to-get-correct-code-coverage-for-member-functions-in-header-files
• https://stackoverflow.com/questions/39611631/how-to-suppress-inlining-of-templates-with-gcov

[nahueespinosa]

Do we know which parts are the most under-tested?

For beluga, we get:

  Reading tracefile /__w/beluga/beluga/ros_ws/build/beluga/coverage.info
  Summary coverage rate:
    lines......: 82.8% (731 of 883 lines)
    functions..: 85.4% (169 of 198 functions)
    branches...: 21.5% (782 of 3632 branches)

And for beluga_amcl:

  Reading tracefile /__w/beluga/beluga/ros_ws/build/beluga_amcl/coverage.info
  Summary coverage rate:
    lines......: 23.1% (130 of 563 lines)
    functions..: 34.1% (15 of 44 functions)
    branches...: 6.9% (133 of 1916 branches)

So, most of the untested code is in beluga_amcl.

JIC templates may also add subtle distortions in coverage measurement, an so do inline functions:

That's useful to know, thanks!

[olmerg]

We can make an integration test like is make in nav2, which will be similar to the beluga example, but with a node which validate that

• set of the initial position
• publication of estimated pose
• broadcast publication
• particles publication
• likelihood_field_pub_

[ivanpauno]

Created a new ticket for system level tests: #127