ENH: Standardize 2D Image Handling#1590
Conversation
Code ReviewWhat this PR doesMulti-purpose refactor of how SIMPLNX handles 2D/1D
Critical Issues
Other Issues
Test Coverage
Suggested merge order
Overall this is a worthwhile direction (compile-time dispatch over runtime branches is cleaner than the previous hand-rolled corner/edge enumeration), but the correctness of the |
- Patch bug in 2D Identify Sample Handling - Optimize Identify Sample Fill Hole Algorithm - Update Neighbors Docs
Locks in the contract for the 2D/1D dimensionality dispatch added in this branch. The tests exercise non-square dims so a regression of the wrong-stride bug on EmptyX/Y/Z 2D cases would fail here without any pipeline plumbing. Covers for every ImageDimensionState: - VoxelNeighbors<T>::k_FaceNeighborCount - initializeFaceNeighborInternalIdx<T>() ordering - initializeFaceNeighborOffsets<T>(dims) with non-square dims - computeValidFaceNeighbors<T>(x,y,z,dims) at corners, edges, interior - computeFaceSurfaceAreas<T>(spacing) precision and ordering - ProcessCorners/Edges/Faces<T>(fn, dims) visitation pattern Also adds the missing <vector> include to NeighborUtilities.hpp so it is standalone-compilable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…IdentifySample The pre-existing 2D tests for both filters used 5x5x1 layouts, which have dims[0] == dims[1] and therefore mask any wrong row-stride in the initializeFaceNeighborOffsets Empty2D dispatches. Add three new cases per filter, one for each EmptyX/Y/Z 2D dimensionality, using a 3x2 (and 3x4 for IdentifySample) layout with hand-computed expected outputs. In ComputeFeatureNeighbors the buggy stride would miss one of the three boundary faces and yield 2 * area instead of the expected 3 * area; in IdentifySample the buggy stride would either merge the two connected components or run off the end of the mask buffer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 2D-handling refactor removed the implicit empty-dim spacing coercion and replaced it with an explicit error on non-positive spacing. These tests capture the new contract so any future regression is caught by the core geometry test suite rather than by a pipeline-level integration test. Covers: - 3D with valid positive spacing computes volume = prod(spacing) - 2D 'piece of paper' example from Geometry.rst, with the spacing=1.0 override pattern the docs recommend for flat-surface analyses - non-positive spacing on any axis returns error -1530 - 1D image with two empty axes produces element sizes of the expected length Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
f97183e to
5f778d6
Compare
…x bool-mask bulk I/O Three logically related changes that finish reconciling the rebased branch with Nathan Young's PR BlueQuartzSoftware#1590 (ENH: Standardize 2D Image Handling) and fix one resulting OOC perf cliff: 1. Wholesale port of PR BlueQuartzSoftware#1590's two algorithm rewrites into the renamed in-core dispatch variants: - ComputeFeatureNeighborsDirect.cpp gets Nathan's templated ComputeFeatureNeighborsFunctor<ImageDimensionStateT> and ProcessVoxels dispatcher in place of the OOC-commit-era custom in-core logic. - IdentifySampleBFS.cpp gets Nathan's templated IdentifySampleFunctor plus the corresponding ProcessVoxels dispatch. The Scanline OOC variant of ComputeFeatureNeighbors is updated to reference the namespaced VoxelNeighbors<Image3D>:: constants while preserving its Z-slice rolling-window bulk-I/O structure. 2. Reapply PR BlueQuartzSoftware#1590's constexpr/const cleanups across the algorithm files where the rebase took --theirs (the OOC commit version) at the 2aa00ee conflict and dropped Nathan's small adjustments: SimplnxCore: ComputeBoundaryCellsDirect, ErodeDilateBadData, ErodeDilateCoordinationNumber, ErodeDilateMask, ReplaceElementAttributesWithNeighborValues, RequireMinimumSizeFeatures OrientationAnalysis: BadDataNeighborOrientationCheckWorklist, NeighborOrientationCorrelation The pattern is uniform: promote the inlined `6` neighbor-array sizes to use VoxelNeighbors<Image3D>::k_FaceNeighborCount via a local k_NumFaceNeighbors alias, make neighborVoxelIndexOffsets const, make faceNeighborInternalIdx constexpr, make isValidFaceNeighbor const where it is not mutated, drop the now-unused DataGroup.hpp include, and const-ify NeighborOrientationCorrelation's orientationOps. ComputeFeatureNeighborsFilter.md picks up Nathan's all-dimension note about user-set spacing for shared surface area calculation. 3. Fix a per-element OOC fallback in BadDataNeighborOrientationCheckScanline that was triggered whenever the input mask was a BoolArray rather than a UInt8Array. The previous code routed bool masks through maskCompare->isTrue / maskCompare->setValue per voxel per Z-slice, causing chunk thrashing under chunked OOC storage. The Small_IN100 pipeline test (a 189x201x117 volume with a bool mask produced by MultiThresholdObjects) ran in 4.7 s on simplnx-Rel but 3+ minutes on simplnx-ooc-Rel. AbstractDataStore<bool> already exposes copyIntoBuffer/copyFromBuffer just like AbstractDataStore<uint8>; the comment claiming otherwise was stale. Resolve a typed AbstractDataStore<bool>* alongside the existing uint8 store pointer and route both load and write-back through bulk I/O, with a small per-slice std::unique_ptr<bool[]> scratch buffer bridging between the algorithm's uint8 slice buffers and the bool data store's typed bulk API. With this change Small_IN100 OOC drops to 4.6 s (~1.6x in-core, in line with normal OOC overhead). Tests updated: - IdentifySampleTest.cpp adopts Nathan's PR BlueQuartzSoftware#1590 hand-built 2D Empty Z/Y/X Non-Square regression tests plus the parameterized identify_sample_v2 exemplar test and the SIMPL Backwards Compatibility test, all wrapped with the OOC dual-path pattern (ForceOocAlgorithmGuard + GENERATE(from_range(k_ForceOocTestValues))). The pre-existing 200x200x200 large-scale OOC validation test is retained. Verified: simplnx-Rel and simplnx-ooc-Rel preset builds both clean. All 43 affected-filter tests pass on simplnx-Rel; all 86 affected-filter tests pass on simplnx-ooc-Rel (regex covering ComputeFeatureNeighbors, IdentifySample, BadDataNeighborOrientation, ComputeBoundaryCells, ErodeDilate*, NeighborOrientationCorrelation, ReplaceElementAttributesWithNeighborValues, RequireMinimumSizeFeatures).
Batch D covers c-axis misalignments and neighbor-correlation filters. All reports remain DRAFT pending developer review of the tentative Algorithm Relationship and Oracle classifications. * ComputeFeatureNeighborCAxisMisalignmentsFilter — DIVISOR BUG REPLICATED VERBATIM from sibling ComputeFeatureNeighborMisorientationsFilter: hexNeighborListSize is reassigned on line 111 of the algorithm, clobbering the hexNeighborListSize-- decrement on line 150 and producing wrong per-feature AvgCAxisMisalignments whenever any non-hex neighbor exists. Production-relevant because EBSD_Hexagonal_Data_Analysis.d3dpipeline ships with find_avg_misals: true. Existing test exemplar is hex-only so it cannot trigger the bug. PR #1467 was OEM-reviewed but preserved the bug because the review covered parameters, not divisor logic. * ComputeFeatureNeighborsFilter — PR #1569 fixed an explicit "Major bug in calculation of Shared Surface Area List (Present in 6.5)"; the shared 6_6_stats_test_v2.tar.gz SSA values are confirmed bad-from-legacy and are intentionally skipped in the Legacy:SmallIn100 test. PR #1590 fixed an in-house row-stride bug in the EmptyZ/EmptyY/EmptyX dispatchers that had been masked by all 5x5x1 fixtures having dims[0] == dims[1]. 32 hand-derived inline test cases now cover 0D/1D/2D/3D x empty-axis x optional flag combinations. * BadDataNeighborOrientationCheckFilter — PR #1499 (REV) is the model V&V case in the audit: explicit iteration-guard bug fix paired with 28 algorithmic test cases (~1700 lines) using hand-derived expectedMask arrays as a de facto Class-1 analytical oracle. PR #1590 made the 6-face neighbor logic 2D-aware via NeighborUtilities. * NeighborOrientationCorrelationFilter — currentLevel > Level strict inequality means Level=6 is a no-op (surprising semantics); only one end-to-end algorithmic test exists; PR #1472 introduced a float-to-double widening that may drift vs legacy. Cross-cutting: - The divisor-bug pattern is now confirmed as a TWO-FILTER copy- paste pattern, not isolated. Screen remaining "average across neighbors" filters for the same shape (NeighborList::size() reassigned inside the inner loop, clobbering an earlier --). - Two AUDIT-CONFIRMED legacy 6.5 defects (PR #1569 SSA, PR #1499 iteration guard) — first instances where legacy is provably wrong rather than SIMPLNX merely diverging. - The model V&V pattern is hand-derived expected* arrays inline in test source. Filters with one happy-path exemplar test are measurably weaker. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
…x bool-mask bulk I/O Three logically related changes that finish reconciling the rebased branch with Nathan Young's PR BlueQuartzSoftware#1590 (ENH: Standardize 2D Image Handling) and fix one resulting OOC perf cliff: 1. Wholesale port of PR BlueQuartzSoftware#1590's two algorithm rewrites into the renamed in-core dispatch variants: - ComputeFeatureNeighborsDirect.cpp gets Nathan's templated ComputeFeatureNeighborsFunctor<ImageDimensionStateT> and ProcessVoxels dispatcher in place of the OOC-commit-era custom in-core logic. - IdentifySampleBFS.cpp gets Nathan's templated IdentifySampleFunctor plus the corresponding ProcessVoxels dispatch. The Scanline OOC variant of ComputeFeatureNeighbors is updated to reference the namespaced VoxelNeighbors<Image3D>:: constants while preserving its Z-slice rolling-window bulk-I/O structure. 2. Reapply PR BlueQuartzSoftware#1590's constexpr/const cleanups across the algorithm files where the rebase took --theirs (the OOC commit version) at the 2aa00ee conflict and dropped Nathan's small adjustments: SimplnxCore: ComputeBoundaryCellsDirect, ErodeDilateBadData, ErodeDilateCoordinationNumber, ErodeDilateMask, ReplaceElementAttributesWithNeighborValues, RequireMinimumSizeFeatures OrientationAnalysis: BadDataNeighborOrientationCheckWorklist, NeighborOrientationCorrelation The pattern is uniform: promote the inlined `6` neighbor-array sizes to use VoxelNeighbors<Image3D>::k_FaceNeighborCount via a local k_NumFaceNeighbors alias, make neighborVoxelIndexOffsets const, make faceNeighborInternalIdx constexpr, make isValidFaceNeighbor const where it is not mutated, drop the now-unused DataGroup.hpp include, and const-ify NeighborOrientationCorrelation's orientationOps. ComputeFeatureNeighborsFilter.md picks up Nathan's all-dimension note about user-set spacing for shared surface area calculation. 3. Fix a per-element OOC fallback in BadDataNeighborOrientationCheckScanline that was triggered whenever the input mask was a BoolArray rather than a UInt8Array. The previous code routed bool masks through maskCompare->isTrue / maskCompare->setValue per voxel per Z-slice, causing chunk thrashing under chunked OOC storage. The Small_IN100 pipeline test (a 189x201x117 volume with a bool mask produced by MultiThresholdObjects) ran in 4.7 s on simplnx-Rel but 3+ minutes on simplnx-ooc-Rel. AbstractDataStore<bool> already exposes copyIntoBuffer/copyFromBuffer just like AbstractDataStore<uint8>; the comment claiming otherwise was stale. Resolve a typed AbstractDataStore<bool>* alongside the existing uint8 store pointer and route both load and write-back through bulk I/O, with a small per-slice std::unique_ptr<bool[]> scratch buffer bridging between the algorithm's uint8 slice buffers and the bool data store's typed bulk API. With this change Small_IN100 OOC drops to 4.6 s (~1.6x in-core, in line with normal OOC overhead). Tests updated: - IdentifySampleTest.cpp adopts Nathan's PR BlueQuartzSoftware#1590 hand-built 2D Empty Z/Y/X Non-Square regression tests plus the parameterized identify_sample_v2 exemplar test and the SIMPL Backwards Compatibility test, all wrapped with the OOC dual-path pattern (ForceOocAlgorithmGuard + GENERATE(from_range(k_ForceOocTestValues))). The pre-existing 200x200x200 large-scale OOC validation test is retained. Verified: simplnx-Rel and simplnx-ooc-Rel preset builds both clean. All 43 affected-filter tests pass on simplnx-Rel; all 86 affected-filter tests pass on simplnx-ooc-Rel (regex covering ComputeFeatureNeighbors, IdentifySample, BadDataNeighborOrientation, ComputeBoundaryCells, ErodeDilate*, NeighborOrientationCorrelation, ReplaceElementAttributesWithNeighborValues, RequireMinimumSizeFeatures).
…x bool-mask bulk I/O Three logically related changes that finish reconciling the rebased branch with Nathan Young's PR BlueQuartzSoftware#1590 (ENH: Standardize 2D Image Handling) and fix one resulting OOC perf cliff: 1. Wholesale port of PR BlueQuartzSoftware#1590's two algorithm rewrites into the renamed in-core dispatch variants: - ComputeFeatureNeighborsDirect.cpp gets Nathan's templated ComputeFeatureNeighborsFunctor<ImageDimensionStateT> and ProcessVoxels dispatcher in place of the OOC-commit-era custom in-core logic. - IdentifySampleBFS.cpp gets Nathan's templated IdentifySampleFunctor plus the corresponding ProcessVoxels dispatch. The Scanline OOC variant of ComputeFeatureNeighbors is updated to reference the namespaced VoxelNeighbors<Image3D>:: constants while preserving its Z-slice rolling-window bulk-I/O structure. 2. Reapply PR BlueQuartzSoftware#1590's constexpr/const cleanups across the algorithm files where the rebase took --theirs (the OOC commit version) at the 2aa00ee conflict and dropped Nathan's small adjustments: SimplnxCore: ComputeBoundaryCellsDirect, ErodeDilateBadData, ErodeDilateCoordinationNumber, ErodeDilateMask, ReplaceElementAttributesWithNeighborValues, RequireMinimumSizeFeatures OrientationAnalysis: BadDataNeighborOrientationCheckWorklist, NeighborOrientationCorrelation The pattern is uniform: promote the inlined `6` neighbor-array sizes to use VoxelNeighbors<Image3D>::k_FaceNeighborCount via a local k_NumFaceNeighbors alias, make neighborVoxelIndexOffsets const, make faceNeighborInternalIdx constexpr, make isValidFaceNeighbor const where it is not mutated, drop the now-unused DataGroup.hpp include, and const-ify NeighborOrientationCorrelation's orientationOps. ComputeFeatureNeighborsFilter.md picks up Nathan's all-dimension note about user-set spacing for shared surface area calculation. 3. Fix a per-element OOC fallback in BadDataNeighborOrientationCheckScanline that was triggered whenever the input mask was a BoolArray rather than a UInt8Array. The previous code routed bool masks through maskCompare->isTrue / maskCompare->setValue per voxel per Z-slice, causing chunk thrashing under chunked OOC storage. The Small_IN100 pipeline test (a 189x201x117 volume with a bool mask produced by MultiThresholdObjects) ran in 4.7 s on simplnx-Rel but 3+ minutes on simplnx-ooc-Rel. AbstractDataStore<bool> already exposes copyIntoBuffer/copyFromBuffer just like AbstractDataStore<uint8>; the comment claiming otherwise was stale. Resolve a typed AbstractDataStore<bool>* alongside the existing uint8 store pointer and route both load and write-back through bulk I/O, with a small per-slice std::unique_ptr<bool[]> scratch buffer bridging between the algorithm's uint8 slice buffers and the bool data store's typed bulk API. With this change Small_IN100 OOC drops to 4.6 s (~1.6x in-core, in line with normal OOC overhead). Tests updated: - IdentifySampleTest.cpp adopts Nathan's PR BlueQuartzSoftware#1590 hand-built 2D Empty Z/Y/X Non-Square regression tests plus the parameterized identify_sample_v2 exemplar test and the SIMPL Backwards Compatibility test, all wrapped with the OOC dual-path pattern (ForceOocAlgorithmGuard + GENERATE(from_range(k_ForceOocTestValues))). The pre-existing 200x200x200 large-scale OOC validation test is retained. Verified: simplnx-Rel and simplnx-ooc-Rel preset builds both clean. All 43 affected-filter tests pass on simplnx-Rel; all 86 affected-filter tests pass on simplnx-ooc-Rel (regex covering ComputeFeatureNeighbors, IdentifySample, BadDataNeighborOrientation, ComputeBoundaryCells, ErodeDilate*, NeighborOrientationCorrelation, ReplaceElementAttributesWithNeighborValues, RequireMinimumSizeFeatures).
Naming Conventions
Naming of variables should descriptive where needed. Loop Control Variables can use
iif warranted. Most of these conventions are enforced through the clang-tidy and clang-format configuration files. See the filesimplnx/docs/Code_Style_Guide.mdfor a more in depth explanation.Filter Checklist
The help file
simplnx/docs/Porting_Filters.mdhas documentation to help you port or write new filters. At the top is a nice checklist of items that should be noted when porting a filter.Unit Testing
The idea of unit testing is to test the filter for proper execution and error handling. How many variations on a unit test each filter needs is entirely dependent on what the filter is doing. Generally, the variations can fall into a few categories:
Code Cleanup