ENH: Update to EbsdLib 2.0.0 API#1472
Merged
imikejackson merged 1 commit intoNov 24, 2025
Merged
Conversation
imikejackson
force-pushed
the
topic/ebsd_lib_rewrite
branch
10 times, most recently
from
62bf806 to
11dfa22
Compare
imikejackson
force-pushed
the
topic/ebsd_lib_rewrite
branch
from
11dfa22 to
031b31d
Compare
JDuffeyBQ
approved these changes
Nov 24, 2025
imikejackson
added a commit
that referenced
this pull request
Apr 28, 2026
Summary entry-point for the 22-filter Tier 1 retroactive audit. Provides: * At-a-glance metrics (filter counts, bug counts, oracle-class distributions) * Master table by batch with one row per filter, linking to each per-filter report (tentative Algorithm Relationship, tentative Oracle class, material PR count, bug flag) * Suspected-real-bugs table — 8 distinct bugs across 5 filters, with severity and production-relevance flags. Calls out the ComputeFeatureNeighborMisorientations / ComputeFeatureNeighbor- CAxisMisalignments divisor pattern as confirmed copy-paste. * Seven cross-cutting findings drawn from the audit (PR #1438 hidden-deviation hotspot, PR #1472 heterogeneity, the model V&V pattern of inline hand-derived expected arrays, the circular- oracle pattern across 5 filters, two audit-confirmed legacy 6.5 defects, shared-archive contagion via 6_6_stats_test_v2.tar.gz, AI-generated rewrite as a V&V red flag). * Next steps — bug triage, skill design, SimplnxReview-plugin reports, policy amendments. All per-filter reports remain DRAFT; this index is also DRAFT and will be updated as developers confirm or correct the tentative classifications. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
imikejackson
added a commit
that referenced
this pull request
May 4, 2026
Batch B covers rotations, orientation conversion, IPF coloring, and threshold logic. All reports remain DRAFT pending developer review of the tentative Algorithm Relationship and Oracle classifications. * RotateSampleRefFrameFilter (SimplnxCore) — flags the stale "only 90/180 deg verified" doc warning (45 deg tests were added in PR #1465); both v2 and v3 test archives still listed in CMakeLists; rotation math lives in shared ImageRotationUtilities * RotateEulerRefFrameFilter (OrientationAnalysis) — PR #1472 silently widened the per-voxel arithmetic from float to double precision (OrientationF -> *DType, Matrix3fR -> Matrix3dR); strongest deviation candidate vs legacy SIMPL * ComputeIPFColorsFilter (OrientationAnalysis) — PR #1472 (EbsdLib 2.0.0 bump) flagged as prime RGB-drift suspect; coverage is cubic-high + [001] only; existing diagnostic block in test asserts nothing * ConvertOrientationsFilter (OrientationAnalysis) — PR #1472 was a wholesale rewrite (~+387/-1190 net) of the dispatch and math layer, not a rename; PR #1438 hides a real algorithmic fix (getPositiveOrientation + zero-init); no exemplar archive; Stereographic excluded from cross-product; no round-trip test * MultiThresholdObjectsFilter (SimplnxCore) — suspected real bug: std::reverse is used in the replaceInput && inverse branch where boolean NOT appears to be intended; non-linear extraction story (#1301 stub -> #1521 not_used -> #1544 populated); dual-source SIMPL port consolidating MultiThresholdObjects v1 + v2 Cross-cutting: PR #1472 (EbsdLib 2.0.0) is materially significant for any filter that delegates math to EbsdLib and should be promoted out of the pruned list for those filters going forward. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
imikejackson
added a commit
that referenced
this pull request
May 4, 2026
Batch C covers orientation/c-axis math filters in OrientationAnalysis. All reports remain DRAFT pending developer review of the tentative Algorithm Relationship and Oracle classifications. * ComputeAvgOrientationsFilter — PR #1577 added vMF and Watson EM-based averaging methods; the filter now hosts three distinct algorithms under one UUID. v2 exemplars for the new methods are likely self-referential (added in the same PR). PR #1472 downgraded to pruned after scoped diff inspection (API rename only for this filter). * ComputeAvgCAxesFilter — PR #1438 silently changed three things: float-to-double accumulation, antipodal-flip behavior via counter reorder, and error-code renumbering. Exemplar 7_2_AvgCAxis.tar.gz is likely circular (regenerated post-fix). Mixed-phase warning path and counter==0 rescue path uncovered. * ComputeCAxisLocationsFilter — PR #1472 silently rewrote the qu-to-om path through a different EbsdLib API (Quaternion::toOrientationMatrix vs explicit qu2om), proposed as Deviation D4 pending comparison. PR #1582 cancel path returns success while leaving output partially zero-filled (D3 internal correctness issue). * ComputeFeatureReferenceCAxisMisorientationsFilter — PR #1438 has an explicit REV: review tag and StdDev double-precision fix that per the PR description brings output into agreement with DREAM3D > 6.5.172, but the change is not regression-locked by any new test in the same PR. * ComputeFeatureNeighborMisorientationsFilter — suspected port-fidelity bug: tempMisoList = featureNeighborList.size() is reassigned inside the inner j-loop, so the divisor reflects only the last neighbor's match state instead of a true matched-phase count. The ComputeAvgMisors=true test is an [.][UNIMPLEMENTED] stub, so the entire average-misorientation control flow has zero CI coverage. Cross-cutting: PR #1438 is the largest hidden-deviation hotspot surfaced in the audit so far — its "ENH" label is misleading; the PR was actually a correctness sweep with multiple silent algorithmic shifts. Multiple exemplar archives in this batch appear to be circular (regenerated from post-fix SIMPLNX output), so existing tests cannot prove legacy conformance even where they appear to. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
imikejackson
added a commit
that referenced
this pull request
May 4, 2026
Batch D covers c-axis misalignments and neighbor-correlation filters. All reports remain DRAFT pending developer review of the tentative Algorithm Relationship and Oracle classifications. * ComputeFeatureNeighborCAxisMisalignmentsFilter — DIVISOR BUG REPLICATED VERBATIM from sibling ComputeFeatureNeighborMisorientationsFilter: hexNeighborListSize is reassigned on line 111 of the algorithm, clobbering the hexNeighborListSize-- decrement on line 150 and producing wrong per-feature AvgCAxisMisalignments whenever any non-hex neighbor exists. Production-relevant because EBSD_Hexagonal_Data_Analysis.d3dpipeline ships with find_avg_misals: true. Existing test exemplar is hex-only so it cannot trigger the bug. PR #1467 was OEM-reviewed but preserved the bug because the review covered parameters, not divisor logic. * ComputeFeatureNeighborsFilter — PR #1569 fixed an explicit "Major bug in calculation of Shared Surface Area List (Present in 6.5)"; the shared 6_6_stats_test_v2.tar.gz SSA values are confirmed bad-from-legacy and are intentionally skipped in the Legacy:SmallIn100 test. PR #1590 fixed an in-house row-stride bug in the EmptyZ/EmptyY/EmptyX dispatchers that had been masked by all 5x5x1 fixtures having dims[0] == dims[1]. 32 hand-derived inline test cases now cover 0D/1D/2D/3D x empty-axis x optional flag combinations. * BadDataNeighborOrientationCheckFilter — PR #1499 (REV) is the model V&V case in the audit: explicit iteration-guard bug fix paired with 28 algorithmic test cases (~1700 lines) using hand-derived expectedMask arrays as a de facto Class-1 analytical oracle. PR #1590 made the 6-face neighbor logic 2D-aware via NeighborUtilities. * NeighborOrientationCorrelationFilter — currentLevel > Level strict inequality means Level=6 is a no-op (surprising semantics); only one end-to-end algorithmic test exists; PR #1472 introduced a float-to-double widening that may drift vs legacy. Cross-cutting: - The divisor-bug pattern is now confirmed as a TWO-FILTER copy- paste pattern, not isolated. Screen remaining "average across neighbors" filters for the same shape (NeighborList::size() reassigned inside the inner loop, clobbering an earlier --). - Two AUDIT-CONFIRMED legacy 6.5 defects (PR #1569 SSA, PR #1499 iteration guard) — first instances where legacy is provably wrong rather than SIMPLNX merely diverging. - The model V&V pattern is hand-derived expected* arrays inline in test source. Filters with one happy-path exemplar test are measurably weaker. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
imikejackson
added a commit
that referenced
this pull request
May 4, 2026
Batch E covers I/O and remaining microtexture / cleanup filters. This is the final batch of the 22-filter Tier 1 retroactive audit. All reports remain DRAFT pending developer review of the tentative Algorithm Relationship and Oracle classifications. * FillBadDataFilter — Algorithm Relationship: REWRITE. PR #1515 was an AI-generated four-phase chunk-aware CCL+UnionFind rewrite (+1139/-194 lines) that has never been formally reviewed; the only legacy-conformance claim is in the PR description. Two real bugs surfaced: (1) static_cast<int32>(size) < threshold at line 476 wraps for components > 2.1B voxels (D4); (2) potential infinite loop in Phase 4 — while(count != 0) has no no-progress termination (D3). storeAsNewPhase assigns the same phase index to all large defect regions (D1). Archive 6_5_fill_bad_data.tar.gz violates the project's own naming guidance — the data is post-#1515 NX-generated, not legacy. * ReadAngDataFilter — PR #1462 deliberately replaced the legacy 6.6 oracle with an NX-self-generated exemplar; the V&V oracle effectively shifted to Class 5 regression / golden-master. PR #1588 added 6.4 + 6.5 conversion JSONs but no TEST_CASE to exercise them. Phase index < 1 is silently clamped to 1, losing EDAX bad-phase signal (D3). PR #1472 correctly downgraded after diff inspection (pure namespace renames). * ITKImageWriterFilter — PR #1555 silently broke OOC support via dynamic_cast<DataStore<T>> with no preflight gate. WriteAs2DStack has an off-by-one loop bound that appears to be dead code. Test only verifies output-file existence and count, never pixel content. No Algorithms/ extraction — all logic in the filter .cpp anonymous namespace. * WritePoleFigureFilter — PR #1587 is unusually material: 391 lines of layout/rendering moved into EbsdLib 2.4.0 and the exemplar archive bumped v4 → v5. Tests provide no signal about pre-#1587 vs post-#1587 numerical drift. Two compounding EbsdLib bumps (#1472 → 2.0.0 and #1587 → 2.4.0) make the legacy comparison especially important. * ReplaceElementAttributesWithNeighborValuesFilter — Two real bugs: (1) bestNeighbor vector allocated once before while(keepGoing) and never reset between iterations, so a voxel replaced in iteration N can be silently re-replaced in N+1 from a stale entry (D1); (2) float32 best is used regardless of templated type T, truncating int64/uint64/ float64 values (D2). Existing test loads the same .dream3d file as both input and exemplar — may not actually exercise replacement. Significant unmerged work exists on remote branches joey/ooc-filter-optimizations and joey/worktree- ReplaceElementAttrsWithNeighborValuesValidation. * ComputeGroupingDensityFilter — surprise: despite being brand-new in the audit window (PR #1548), this is a Port, not a New filter (legacy DREAM3D FindGroupingDensity renamed per the Find* → Compute* convention), so legacy comparison applies. Test tag bug: 7 of 8 TEST_CASEs tagged [SimplnxReview] instead of [SimplnxCore], suggesting the filter was prototyped in SimplnxReview before being moved. Existing tests have hand-derived expected constants — Class 1 oracle gold standard. Cross-cutting Batch E findings: - Five new real-bug candidates surfaced (FillBadData int32 overflow, FillBadData potential infinite loop, ITKImageWriter OOC break, ReplaceElementAttributes bestNeighbor stale-state, ReplaceElementAttributes float32-truncation). Add to the audit bug log alongside the prior 3 (MultiThresholdObjects std::reverse + two-filter divisor pattern). - AI-generated rewrite (PR #1515) without formal review is a V&V red flag that the policy doc should explicitly call out as requiring extra scrutiny (algorithm review + independent oracle, not just "passes existing tests"). - Multiple filters have deliberately replaced their legacy oracle with an NX-self-generated exemplar (FillBadData 6_5_fill_bad_data.tar.gz, ReadAng read_ang_test.tar.gz, WritePoleFigure v5 archive, AvgCAxes 7_2_AvgCAxis.tar.gz), each shifting the de-facto oracle to Class 5 regression / golden-master. The policy should explicitly require an Oracle Provenance ReadMe entry for any such archive. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
imikejackson
added a commit
that referenced
this pull request
May 4, 2026
Summary entry-point for the 22-filter Tier 1 retroactive audit. Provides: * At-a-glance metrics (filter counts, bug counts, oracle-class distributions) * Master table by batch with one row per filter, linking to each per-filter report (tentative Algorithm Relationship, tentative Oracle class, material PR count, bug flag) * Suspected-real-bugs table — 8 distinct bugs across 5 filters, with severity and production-relevance flags. Calls out the ComputeFeatureNeighborMisorientations / ComputeFeatureNeighbor- CAxisMisalignments divisor pattern as confirmed copy-paste. * Seven cross-cutting findings drawn from the audit (PR #1438 hidden-deviation hotspot, PR #1472 heterogeneity, the model V&V pattern of inline hand-derived expected arrays, the circular- oracle pattern across 5 filters, two audit-confirmed legacy 6.5 defects, shared-archive contagion via 6_6_stats_test_v2.tar.gz, AI-generated rewrite as a V&V red flag). * Next steps — bug triage, skill design, SimplnxReview-plugin reports, policy amendments. All per-filter reports remain DRAFT; this index is also DRAFT and will be updated as developers confirm or correct the tentative classifications. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Will need to revert the VCPKG changes once the simplnx-registry is updated