ENH: Add StructuralSimilarityImageFilter for SSIM image quality

[hjmjohnson]

Closes #6030. Supersedes #6031 (which had compile errors and was based on an off-by-default uniform window rather than the canonical Wang Gaussian).

Summary

N-dimensional, multi-threaded ITK filter for the Structural Similarity Index Measure (Wang et al., IEEE TIP 2004). Two inputs in, a per-pixel SSIM map out, and a scalar mean SSIM available via GetMeanSSIM() after Update().

Resources consulted

The implementation and tests follow the canonical SSIM formulation. Sources used while building this PR:

• Wang, Bovik, Sheikh, Simoncelli, "Image Quality Assessment: From Error Visibility to Structural Similarity," IEEE TIP 13(4), 2004 — original paper, formulas, default constants.
• Wang et al. SSIM MATLAB reference (utlive/ssim/ssim_index.m) — mu = filter2(window, ...), sigma_xy = filter2(window, x*y) - mu_x*mu_y, the simplified-product fast path with K1=0.01, K2=0.03, L=255 defaults.
• scikit-image v0.25 _structural_similarity.py — recommended configuration (gaussian_weights=True, sigma=1.5, use_sample_covariance=False), the crop by (win_size-1)/2 border before averaging the SSIM map, the cov_norm distinction between sample and population covariance.
• Wang, Simoncelli, Bovik, "Multi-Scale Structural Similarity for Image Quality Assessment," Asilomar 2003 — multi-scale extension whose API surface (the ScaleWeights array) is reserved here for a follow-up PR.
• Wikipedia: Structural similarity index measure — formulas, $\alpha\beta\gamma$ exponents, MS-SSIM defaults.

Algorithm

For two images $x$ and $y$, local statistics are computed by convolving with a discrete Gaussian (default $\sigma=1.5$, $11\times 11$):

$$\mu_x = G_\sigma * x,\quad \mu_y = G_\sigma * y$$

$$\sigma_x^2 = G_\sigma * x^2 - \mu_x^2,\quad \sigma_y^2 = G_\sigma * y^2 - \mu_y^2,\quad \sigma_{xy} = G_\sigma * (xy) - \mu_x\mu_y$$

The three SSIM components (with $C_1 = (K_1 L)^2$, $C_2 = (K_2 L)^2$, $C_3 = C_2/2$):

$$l(x,y) = \frac{2\mu_x\mu_y + C_1}{\mu_x^2 + \mu_y^2 + C_1},\quad c(x,y) = \frac{2\sigma_x\sigma_y + C_2}{\sigma_x^2 + \sigma_y^2 + C_2},\quad s(x,y) = \frac{\sigma_{xy} + C_3}{\sigma_x\sigma_y + C_3}$$

Combined: $\text{SSIM}(x,y) = l^\alpha c^\beta s^\gamma$. The default $\alpha=\beta=\gamma=1$ takes the simplified product fast path that matches Wang's reference SSIM exactly:

$$\text{SSIM}(x,y) = \frac{(2\mu_x\mu_y + C_1)(2\sigma_{xy} + C_2)}{(\mu_x^2 + \mu_y^2 + C_1)(\sigma_x^2 + \sigma_y^2 + C_2)}$$

Architecture

Composite ImageToImageFilter:

1. Internal sub-pipeline of 5 DiscreteGaussianImageFilter passes (for $\mu_x$, $\mu_y$, $\mu_{xx}$, $\mu_{yy}$, $\mu_{xy}$), reusing ITK's well-optimized multi-threaded smoothing.
2. A parallelized per-pixel combination via MultiThreaderBase::ParallelizeImageRegion reads from the five smoothed buffers and writes the SSIM map.
3. Mean SSIM is accumulated only over the cropped interior region (excluding the half-Gaussian-kernel border), matching scikit-image and the MATLAB reference.

API (covers all of #6030)

Parameter	Default	Notes
GaussianSigma	1.5	Wang canonical
MaximumKernelWidth	11	gives 11×11 with σ=1.5
K1 / K2	0.01 / 0.03	Wang defaults
DynamicRange $L$	NumericTraits-derived	1.0 for float/double, 255 for uchar, 65535 for ushort, etc.
LuminanceExponent $\alpha$	1.0
ContrastExponent $\beta$	1.0
StructureExponent $\gamma$	1.0
ScaleWeights	{1.0}	Single-scale. Length > 1 reserved for MS-SSIM (currently throws not-yet-implemented).

The filter is N-dimensional (tested 2D, 3D, 4D), templated over input/output image types, and exception-safe for invalid configurations.

Test strategy (30 GTest assertions)

Reference values were generated against skimage.metrics.structural_similarity with gaussian_weights=True, sigma=1.5, use_sample_covariance=False, data_range=255, win_size=11 (canonical Wang configuration).

Tests are split into seven categories that decouple correctness checks from sensitivity to discrete-Gaussian implementation differences between ITK's GaussianOperator and scipy's sampled Gaussian:

1. Mathematical identities (kernel-independent, tolerance 1e-9)

   • SSIM(x, x) = 1 exactly for constant, random, and gradient images
   • Symmetry: SSIM(a,b) == SSIM(b,a)

2. Closed-form analytic checks for constant inputs (tolerance 1e-9)

   • For constant inputs all variances and the covariance vanish, so $\text{SSIM}(a,b) = (2ab + C_1)/(a^2 + b^2 + C_1)$
   • Verified at (100, 150) → 0.9230923 and the textbook (0, 255) → 0.0000999900
   • Per-pixel verification across the entire output map for constant inputs (every map element matches the closed form)

3. Input validation (exception tests)

   • Mismatched input sizes, missing inputs, non-positive sigma, non-positive dynamic range, empty ScaleWeights, multi-element ScaleWeights (MS-SSIM not yet implemented)

4. Qualitative properties

   • Result range bounded in $[-1, 1]$
   • Strict monotonic decay of mean SSIM as additive Gaussian noise grows ($\sigma = 2 \to 8 \to 24$)
   • Strong anti-correlation for negated images (SSIM(x, 255-x) < -0.5)

5. scikit-image cross-checks (tolerance 5e-3)

   • Gradient + 30 luminance shift → 0.9676912545
   • Gradient × 0.5 contrast change → 0.7550069937
   • The 5e-3 tolerance absorbs minor discretization differences between ITK's GaussianOperator and scipy's sampled Gaussian (the two libraries do not produce bit-identical Gaussian kernels)

6. Code-path equivalence

   • The simplified-product fast path ($\alpha=\beta=\gamma=1$) and the general $l^\alpha c^\beta s^\gamma$ path (forced via a tiny exponent perturbation) agree to 1e-6 on the same inputs

7. Multi-dimensional and pixel-type coverage

   • 3D and 4D variants of the identity and constant-input tests
   • Default DynamicRange correct for unsigned char, unsigned short, float
   • unsigned char identical-image identity test

Results

Local build with GCC 13.3 / Ninja / Release on Ubuntu 24.04, 48 cores:

$ cmake --build build-ssim -j48 --target ITKImageCompareGTestDriver
[4/4] Linking CXX executable bin/ITKImageCompareGTestDriver

$ ./bin/ITKImageCompareGTestDriver --gtest_filter='StructuralSimilarityImageFilter.*'
[==========] 30 tests from 1 test suite ran. (60 ms total)
[  PASSED  ] 30 tests.

$ ctest -R 'StructuralSimilarity' --output-on-failure
100% tests passed, 0 tests failed out of 30
Total Test time (real) =   0.30 sec

ITKImageCompareHeaderTest1 (the auto-generated header self-test) and the existing ITKImageCompareTestDriver also link cleanly with the new module dependencies.

pre-commit (gersemi, clang-format, kw-pre-commit, etc.) reports all checks Passed on every touched file.

Module dependency changes

Modules/Filtering/ImageCompare/itk-module.cmake adds:

• ITKSmoothing as a COMPILE_DEPENDS (for DiscreteGaussianImageFilter)
• ITKGoogleTest as a TEST_DEPENDS (for the new GTest driver)

Files

Modules/Filtering/ImageCompare/include/itkStructuralSimilarityImageFilter.h    | 295 ++
Modules/Filtering/ImageCompare/include/itkStructuralSimilarityImageFilter.hxx  | 386 ++
Modules/Filtering/ImageCompare/test/itkStructuralSimilarityImageFilterGTest.cxx| 674 ++
Modules/Filtering/ImageCompare/wrapping/itkStructuralSimilarityImageFilter.wrap|   3 ++
Modules/Filtering/ImageCompare/itk-module.cmake                                |   2 +
Modules/Filtering/ImageCompare/test/CMakeLists.txt                             |   4 +
6 files changed, 1364 insertions(+)

Test plan

• All 30 SSIM GTests pass locally (ctest -R StructuralSimilarity)
• ITKImageCompareHeaderTest1 (header self-test) compiles and links
• ITKImageCompareTestDriver (full classic test driver) compiles and links
• pre-commit run clean on all touched files (gersemi, clang-format, kw-pre-commit)
• CI: Linux, Windows, macOS, ARM, Pixi
• CI: Python wrapping (itkStructuralSimilarityImageFilter.wrap provided for WRAP_ITK_REAL, 2D)

Future work (out of scope here)

• Multi-scale SSIM (MS-SSIM, Wang 2003): the ScaleWeights API surface is in place; implementation requires per-scale Gaussian downsampling and the per-component product across scales. A length > 1 array currently throws a clear not-yet-implemented exception.

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR adds a new StructuralSimilarityImageFilter implementing the Wang et al. 2004 SSIM metric as an N-dimensional, multi-threaded composite ITK filter. The implementation is thorough: five internal DiscreteGaussianImageFilter passes feed a parallelized per-pixel combination stage, with mean SSIM averaged over the cropped interior region matching the scikit-image/MATLAB reference. All 30 GTests pass locally. Two minor correctness edge-cases are worth noting before merge.

Confidence Score: 5/5

Safe to merge; all findings are P2 style/edge-case suggestions that do not affect the default use case.

The algorithm is correct for the default configuration (σ=1.5, 11×11 kernel, K1=0.01, K2=0.03, L from NumericTraits). The two flagged gaps — border-crop formula for even kernel widths, and no guard against K1/K2=0 NaN — are P2 edge cases that do not affect any of the 30 passing tests or the documented API. The test coverage is comprehensive and the implementation closely follows Wang 2004 and scikit-image.

itkStructuralSimilarityImageFilter.hxx lines 106–122 (K1/K2 validation) and line 240 (border-crop formula).

Important Files Changed

Filename	Overview
Modules/Filtering/ImageCompare/include/itkStructuralSimilarityImageFilter.h	Public API and class declaration; well-structured with standard ITK macros. Documentation note: header comment still says "BeforeGenerate" though validation is in VerifyPreconditions (flagged in previous thread).
Modules/Filtering/ImageCompare/include/itkStructuralSimilarityImageFilter.hxx	Core implementation; simplified and general SSIM paths are correct. Two edge-case gaps: K1/K2=0 produces silent NaN for constant/zero-mean inputs, and the border-crop formula diverges from scikit-image for even MaximumKernelWidth values.
Modules/Filtering/ImageCompare/test/itkStructuralSimilarityImageFilterGTest.cxx	30 GTest cases covering identities, analytic constant-image checks, input validation, qualitative properties, scikit-image cross-checks, code-path equivalence, and multi-dimensional/pixel-type coverage. Well-designed and comprehensive.
Modules/Filtering/ImageCompare/itk-module.cmake	Adds ITKSmoothing as COMPILE_DEPENDS (correct for template-only dependency) and ITKGoogleTest as TEST_DEPENDS. No issues.
Modules/Filtering/ImageCompare/test/CMakeLists.txt	Adds creategoogletestdriver for the new GTest file; GTest auto-discovery handles CTest registration, confirmed by author's ctest run showing 30 tests.
Modules/Filtering/ImageCompare/wrapping/itkStructuralSimilarityImageFilter.wrap	Wraps for WRAP_ITK_REAL (float/double) in 2D, consistent with SimilarityIndexImageFilter's wrapping pattern. Appropriate starting point.

_{Reviews (3): Last reviewed commit: "ENH: Add StructuralSimilarityImageFilter..." | Re-trigger Greptile}

[blowekamp]

I believe algorithms should go into remote modules first.

It may be useful to create new skill or AI tools in the Remove module template to assist with creating new modules.

I have used SSI before, I am unsure if I implemented it also with jus using composition of filters with SimpleITK or only use the scikit-image version.

[hjmjohnson]

I believe algorithms should go into remote modules first.

It may be useful to create new skill or AI tools in the Remove module template to assist with creating new modules.

I have used SSI before, I am unsure if I implemented it also with jus using composition of filters with SimpleITK or only use the scikit-image version.

@blowekamp I was thinking both of these thoughts too (1 from tempate remote module generator & RemoteModule for new filters), this too, but I think we should have a "FutureITK" module for putting proposed additions with a set of criteria for enabling/disabling elements of the entire module. Maintaining many modules is a major pain point, so minimimizing the number of modules is important.

ENABLE_SSIM_METRIC=ON|OFF
ENABLE_COOL_FILTER=ON|OFF

Hans

[blowekamp]

I believe algorithms should go into remote modules first.
It may be useful to create new skill or AI tools in the Remove module template to assist with creating new modules.
I have used SSI before, I am unsure if I implemented it also with jus using composition of filters with SimpleITK or only use the scikit-image version.

@blowekamp I was thinking both of these thoughts too (1 from tempate remote module generator & RemoteModule for new filters), this too, but I think we should have a "FutureITK" module for putting proposed additions with a set of criteria for enabling/disabling elements of the entire module. Maintaining many modules is a major pain point, so minimimizing the number of modules is important.

ENABLE_SSIM_METRIC=ON|OFF ENABLE_COOL_FILTER=ON|OFF

Hans

Yes, the remote module situation is not easy. Hopefully with AI agents and tools/skills the process can easier now.

What you are describing is similar to the "Review" module. Which is one of those optional modules that is not on for testing so it breaks. We should probably have this module enable for testing but not by default for users.

Minimizing the number of options is also important to. A header only filter, with a test is no addition to users who do not include the header. It's only an addition to testing, so then if it's not tested it won't be working.

[dzenanz]

Maintaining many modules is a major pain point, so minimimizing the number of modules is important.

I was also thinking about grouping the content of current remote modules into a much smaller set. Most remote modules have just a few filters in them, so combining them would be technically easy. What do you think about this? The main question is how to group them.

[hjmjohnson]

Maintaining many modules is a major pain point, so minimimizing the number of modules is important.

I was also thinking about grouping the content of current remote modules into a much smaller set. Most remote modules have just a few filters in them, so combining them would be technically easy. What do you think about this? The main question is how to group them.

I would take into consideration the information in *.remote.cmake.

#-- # Grading Level Criteria Report
#-- EVALUATION DATE: 2020-03-24
#-- EVALUATORS: [Dženan Zukić, Davis Vigneault]
#--
#-- ## Compliance level 5 star (AKA ITK main modules, or remote modules that could become core modules)
#--   - [ ] Widespread community dependance
#--   - [X] Above 90% code coverage
#--   - [ ] CI dashboards and testing monitored rigorously
#--   - [X] Key API features are exposed in wrapping interface
#--   - [ ] All requirements of Levels 4,3,2,1
#--
#-- ## Compliance Level 4 star (Very high-quality code, perhaps small community dependance)
#--   - [X] Meets all ITK code style standards
#--   - [X] No external requirements beyond those needed by ITK proper
#--   - [ ] Builds and passes tests on all supported platforms within 1 month of each core tagged release
#--            - [ ] Windows Shared Library Build with Visual Studio
#--            - [ ] Mac with clang compiler
#--            - [ ] Linux with gcc compiler
#--   - [X] Active developer community dedicated to maintaining code-base
#--   - [X] 75% code coverage demonstrated for testing suite
#--   - [X] Continuous integration testing performed
#--   - [X] All requirements of Levels 3,2,1

Anything in Level 2 or below should perhaps be combined into a single AlphaCodeSpecialtyModules Remote module.

Anything in Level 3 is BetaCodeSpecialtyModules modules

Anything in Level 4 is SpecialtyModules

[dzenanz]

We are still in 6.0 beta phase. We can do this now. But we should discuss this more widely. Hans, do you want to start a new forum topic?

[hjmjohnson]

@dzenanz I've got a lot going on today. Will try to address this weekend.

[hjmjohnson]

The lone failing check (ITK.Linux on Azure DevOps build 15505) appears to be a pipeline-side flake rather than a real build/test issue:

• ✅ CDash reports "All builds completed successfully" for SHA 46fabd3525 — meaning the same Linux ITK build that Azure ran successfully completed, ran its tests, and submitted the green results to CDash.
• ✅ Every other Linux build passes:
  • Pixi-Cxx (ubuntu-22.04) ✓
  • ARMBUILD-Ubuntu-24.04-arm ✓
  • ARMBUILD-x86_64-rosetta ✓ (macOS but same toolchain class)
• ✅ Every other Azure pipeline passes:
  • ITK.Windows ✓
  • ITK.macOS ✓
  • ITK.Linux.Python ✓
  • ITK.macOS.Python ✓
• ✅ The 30 new SSIM GTests run and pass on ARMBUILD-Python (verified in the run log: tests FindPython3.cmake unavailable before CMake 3.12 #1700–Add lambda function command #1735 all Passed).

The pattern (lone Azure pipeline failing, CDash green, all duplicate-coverage Linux builds green) almost always means the failure is in a post-CTest infrastructure step in the Azure pipeline (cache save, artifact upload, agent cleanup), not in the build itself. Re-triggering.

[hjmjohnson]

/azp run ITK.Linux

[blowekamp]

Some comment on pipeline management details.

The implementation this feeling is going to be rather memory intensive with the number of image converted and processed to real type.

Is this a 2D only implementation?

[hjmjohnson]

Updated diagnosis: the ITK.Linux failure was a real one in the legacy-removed Linux build (the Azure pipeline runs both a normal and an ITK_LEGACY_REMOVE=ON configuration; the normal one was green on CDash, hence the misleading "all builds completed successfully" CDash summary, but the legacy-removed one promoted a deprecation warning into an error).

The root cause:

itkStructuralSimilarityImageFilter.hxx:342:87:
warning: 'itk::ImageConstIterator<TImage>::IndexType
itk::ImageConstIterator<TImage>::GetIndex() const' is deprecated:
Please use ComputeIndex() instead, or use an iterator with index,
like ImageIteratorWithIndex! [-Wdeprecated-declarations]

  342 |   if (subInterior.GetNumberOfPixels() > 0 && subInterior.IsInside(outIt.GetIndex()))

The output iterator was a plain ImageRegionIterator, and the per-pixel loop called outIt.GetIndex() to test interior-region membership for the mean accumulator. GetIndex() on the index-less iterator is wrapped in #ifndef ITK_FUTURE_LEGACY_REMOVE (Modules/Core/Common/include/itkImageConstIterator.h:306-317) and emits a deprecation warning under ITK_LEGACY_REMOVE. CI promotes that warning to an error.

Fix (commit f4316869e7): switch the output iterator to ImageRegionIteratorWithIndex, which tracks the index natively and exposes a non-deprecated GetIndex(). One-line change: replace the using OutputIteratorType = ImageRegionIterator<OutputImageType> typedef and add the corresponding header include.

Local verification:

$ cmake -B build-ssim-legacy -S . -G Ninja \
       -DITK_LEGACY_REMOVE=ON -DITK_LEGACY_SILENT=OFF ...
$ cmake --build build-ssim-legacy -j 4 \
       --target ITKImageCompareHeaderTest1 ITKImageCompareGTestDriver
[1/4] Building CXX object .../ITKImageCompareHeaderTest1.dir/test/ITKImageCompareHeaderTest1.cxx.o
[2/4] Linking CXX executable bin/ITKImageCompareHeaderTest1
[3/4] Building CXX object .../itkStructuralSimilarityImageFilterGTest.cxx.o
[4/4] Linking CXX executable bin/ITKImageCompareGTestDriver

No deprecation warnings, clean build.

$ ./bin/ITKImageCompareGTestDriver --gtest_filter='StructuralSimilarityImageFilter.*'
[==========] 30 tests from 1 test suite ran. (333 ms total)
[  PASSED  ] 30 tests.

All 30 SSIM GTests still pass under the legacy-removed build with the new iterator.

[hjmjohnson]

@greptileai review this draft before I make it official