Summary
While working toward pushing ITK's line coverage past 90% (the CDash itk-coverage build currently reports 89.51%, ~760 covered lines short), I found a systemic source of silent coverage loss: test source files that are compiled into their module test drivers but are never invoked by any itk_add_test. They build, count as "present," but never run — so the code they would exercise shows as uncovered.
itkOrientImageFilter was the motivating example: 0% coverage on CDash despite two existing test files, purely because neither was registered with itk_add_test. A toolkit-wide scan found ~30 more.
This issue tracks registering (or otherwise dispositioning) each one. Verified history shows these are move/reorg oversights, not deliberate removals: none ever had an itk_add_test (git log -G = 0 historical invocations), none have GoogleTest replacements, and Nonunit/IntegratedTest/test/CMakeLists.txt even carried explicit # missing itk_add_test() call comments.
Detection method (reproducible)
For each Modules/**/test/CMakeLists.txt: strip #-comments, collect itkFoo.cxx source tokens that exist on disk, skip GoogleTest bodies (TEST( / itkGTest.h), require the file to define int <basename>(int …), then flag if <basename> never appears in the CMake except as <basename>.cxx.
Known blind spots (fix before re-running): tests whose names are built in a foreach/variable (e.g. itkDiscrete${loop_var}Curvature…) are not dormant but look it; and a commented-out itk_add_test means intentionally disabled, not missing.
Done — registered, build + run locally, all pass (15 tests)
- Filtering/ImageGrid:
itkOrientImageFilterTest (useImageDirection=0) + legacy itkOrientImageFilterTest2
- Core/Common:
itkObjectStoreTest, itkImageComputeOffsetAndIndexTest, itkImageRegionExplicitTest, itkMetaProgrammingLibraryTest, itkPromoteType
- Nonunit/IntegratedTest:
itkFilterImageAddTest, itkReleaseDataFilterTest (the two with # missing itk_add_test() comments)
- IO/JPEG2000:
itkJPEG2000ImageIOFactoryTest01, itkJPEG2000ImageIOTest00
- Segmentation/LevelSetsv4:
itkLevelSetEquationChanAndVeseInternalTermTest, itkLevelSetEquationRegionTermTest, itkLevelSetEquationTermBaseTest, itkLevelSetEquationTermContainerTest
Skipped — fail when run or need investigation (NOT just a registration)
- Core/QuadEdgeMesh/
itkQuadEdgeMeshBasicLayerTest — runtime failure: Erroneous GetDestination() on edge number 0. Was expecting 2 but got 0. Behavior drift since it last ran.
- Segmentation/LevelSetsv4/
itkLevelSetEquationChanAndVeseExternalTermTest — runs to "CV External term created" then returns EXIT_FAILURE (internal assertion/value issue).
- Nonunit/Review/
itkWarpJacobianDeterminantFilterTest — ITKReview not built in the default configuration; needs the module enabled to verify.
Needs fixtures / parameters before registering
- Filtering/LabelMap (3):
itkStatistics{KeepNObjects,Opening,Relabel}LabelMapFilterTest1 — need 6 args (input + feature image + output + reverseOrdering + a valid statistics-attribute enum + count) and no baseline exists (would need one published).
- IO/JPEG2000:
itkJPEG2000ImageIOTest01/02/04 — need .j2k inputs + args.
- IO/DCMTK/
itkDCMTKImageIOMultiFrameImageTest — needs a multi-frame DICOM fixture.
- Video/BridgeVXL/
itkVXLVideoIOTest — needs VXL + a video fixture.
- Core/ImageFunction / ImageGradient / Registration/Metricsv4 (4
*SpeedTest) — benchmarks; likely intentionally not run in CI (confirm before enabling).
Scanner false positives (already running / intentionally disabled — leave alone)
- Filtering/QuadEdgeMeshFiltering (4):
itkDiscrete{Gaussian,Maximum,Mean,Minimum}CurvatureQuadEdgeMeshFilterTest — already registered via foreach(loop_var ${CURV_TESTS}).
- Filtering/IsotropicWavelets (2):
itkZeroDCImageFilterTest, itkWaveletFrequencyFilterBankGeneratorDownsampleTest — itk_add_test blocks are commented out (deliberate disable).
- Bridge/VtkGlue/
runViewImage — commented-out interactive viewer (needs a display).
Summary
While working toward pushing ITK's line coverage past 90% (the CDash
itk-coveragebuild currently reports 89.51%, ~760 covered lines short), I found a systemic source of silent coverage loss: test source files that are compiled into their module test drivers but are never invoked by anyitk_add_test. They build, count as "present," but never run — so the code they would exercise shows as uncovered.itkOrientImageFilterwas the motivating example: 0% coverage on CDash despite two existing test files, purely because neither was registered withitk_add_test. A toolkit-wide scan found ~30 more.This issue tracks registering (or otherwise dispositioning) each one. Verified history shows these are move/reorg oversights, not deliberate removals: none ever had an
itk_add_test(git log -G= 0 historical invocations), none have GoogleTest replacements, andNonunit/IntegratedTest/test/CMakeLists.txteven carried explicit# missing itk_add_test() callcomments.Detection method (reproducible)
For each
Modules/**/test/CMakeLists.txt: strip#-comments, collectitkFoo.cxxsource tokens that exist on disk, skip GoogleTest bodies (TEST(/itkGTest.h), require the file to defineint <basename>(int …), then flag if<basename>never appears in the CMake except as<basename>.cxx.Known blind spots (fix before re-running): tests whose names are built in a
foreach/variable (e.g.itkDiscrete${loop_var}Curvature…) are not dormant but look it; and a commented-outitk_add_testmeans intentionally disabled, not missing.Done — registered, build + run locally, all pass (15 tests)
itkOrientImageFilterTest(useImageDirection=0) + legacyitkOrientImageFilterTest2itkObjectStoreTest,itkImageComputeOffsetAndIndexTest,itkImageRegionExplicitTest,itkMetaProgrammingLibraryTest,itkPromoteTypeitkFilterImageAddTest,itkReleaseDataFilterTest(the two with# missing itk_add_test()comments)itkJPEG2000ImageIOFactoryTest01,itkJPEG2000ImageIOTest00itkLevelSetEquationChanAndVeseInternalTermTest,itkLevelSetEquationRegionTermTest,itkLevelSetEquationTermBaseTest,itkLevelSetEquationTermContainerTestSkipped — fail when run or need investigation (NOT just a registration)
itkQuadEdgeMeshBasicLayerTest— runtime failure:Erroneous GetDestination() on edge number 0. Was expecting 2 but got 0.Behavior drift since it last ran.itkLevelSetEquationChanAndVeseExternalTermTest— runs to "CV External term created" then returnsEXIT_FAILURE(internal assertion/value issue).itkWarpJacobianDeterminantFilterTest—ITKReviewnot built in the default configuration; needs the module enabled to verify.Needs fixtures / parameters before registering
itkStatistics{KeepNObjects,Opening,Relabel}LabelMapFilterTest1— need 6 args (input + feature image + output + reverseOrdering + a valid statistics-attribute enum + count) and no baseline exists (would need one published).itkJPEG2000ImageIOTest01/02/04— need.j2kinputs + args.itkDCMTKImageIOMultiFrameImageTest— needs a multi-frame DICOM fixture.itkVXLVideoIOTest— needs VXL + a video fixture.*SpeedTest) — benchmarks; likely intentionally not run in CI (confirm before enabling).Scanner false positives (already running / intentionally disabled — leave alone)
itkDiscrete{Gaussian,Maximum,Mean,Minimum}CurvatureQuadEdgeMeshFilterTest— already registered viaforeach(loop_var ${CURV_TESTS}).itkZeroDCImageFilterTest,itkWaveletFrequencyFilterBankGeneratorDownsampleTest—itk_add_testblocks are commented out (deliberate disable).runViewImage— commented-out interactive viewer (needs a display).