Add support for generating coverage statistics for tests #1712
Conversation
|
Is this only for local usage or is this supposed to be used by nightlies/ci? |
|
@m-kuhn either I guess, but ideally it would be included in Travis builds and published automatically to a dashboard (orfeo? Coveralls?). From what I read this setup should be safe to use with clang builds too. It works really well locally, and I think it will be valuable for directing testing efforts. Especially if it gets integrated with Travis so that we can see coverage of pull requests prior to merging... |
Build option ENABLE_COVERAGE must be set. There's also a secondary option GENERATE_COVERAGE_DOCS which requires lcov and will generate html docs of unit test coverage. To generate coverage, first enable the cmake options, then build QGIS. From the build directory, run "make qgis_coverage". You can also add command line arguments to "make qgis_coverage" which are forwarded on to ctest. This allows for skipping failing tests which prevent generation of coverage docs. This is done by adding the command line argument CMD_ARGS= to the "make qgis_coverage" command. Eg, make qgis_coverage CMD_ARGS="-E \"PalLabeling|LocalServer|AppStartup\" will skip tests which match the listed expressions.
|
@m-kuhn Here's a sample of the output: http://nyalldawson.net/qgis/coverage/ |
| # whether coverage tests should be performed | ||
| SET (ENABLE_COVERAGE FALSE CACHE BOOL "Perform coverage tests?") | ||
|
|
||
| # whether coverage tests should be performed |
There was a problem hiding this comment.
The comment should probably be adjusted?
|
It's all opt-in, so it shouldn't break anything. Good to merge I'd say. You included the cmake module for clang support, right? |
|
Manually merged in a435441 . I'll keep refining this in an attempt to get automated generation working with Travis. |
Build option ENABLE_COVERAGE must be set. There's also a secondary
option GENERATE_COVERAGE_DOCS which requires lcov and will generate
html docs of unit test coverage.
To generate coverage, first enable the cmake options, then build
QGIS. From the build directory, run "make qgis_coverage".
You can also add command line arguments to "make qgis_coverage" which
are forwarded on to ctest. This allows for skipping failing tests
which prevent generation of coverage docs. This is done by adding
the command line argument CMD_ARGS= to the "make qgis_coverage"
command. Eg,
make qgis_coverage CMD_ARGS="-E "PalLabeling|LocalServer|AppStartup"
will skip tests which match the listed expressions.