systems comparison engine

Comparison engine

Active contributors: Douwe de Vries

Purpose

The comparison engine aligns rows by selected keys, applies normalization, classifies row outcomes, and generates summaries. It supports exact key matching and opt-in flexible key matching.

Directory layout

src/comparison/
├── engine.rs
├── rows.rs
├── value_compare.rs
├── mapping.rs
└── mod.rs

Key abstractions

Name	File	Description
ComparisonPlan	src/comparison/engine.rs	Precomputed key splits and flexible candidates used for cap validation and execution.
FlexibleKeyLimits	src/comparison/engine.rs	Candidate and comparison caps for flexible matching.
KeyedRows	src/comparison/rows.rs	Group of row indexes sharing a normalized key.
FlexibleKeyMatch	src/comparison/rows.rs	Match classes ranked for flexible candidate selection.
find_differences	src/comparison/value_compare.rs	Compares selected values and reports column-level differences.
suggest_mappings_with_data	src/comparison/mapping.rs	Suggests header and value-profile-based mappings.

How it works

graph TD
    Request[CompareRequest] --> Config[src/backend/validation.rs]
    Config --> Plan[ComparisonPlan::build]
    Plan --> Split[split rows by normalized key]
    Split --> Mode{Flexible keys?}
    Mode -->|no| Exact[compare exact key groups]
    Mode -->|yes| Flexible[classify and select flexible candidates]
    Exact --> Results[RowComparisonResult]
    Flexible --> Results
    Results --> Summary[generate_summary]

Loading

Exact matching requires the same number of row-key columns on both sides. Flexible matching can use different counts and candidate classes, but caps large candidate sets to protect local runtime memory and CPU.

Integration points

src/backend/workflow.rs calls ComparisonPlan::build through run_comparison. The UI controls normalization in frontend/src/components/mapping-config/NormalizationPanel.tsx, and request validation converts frontend selections into ComparisonConfig in src/backend/validation.rs.

Entry points for modification

For row classification changes, edit src/comparison/engine.rs and add cases to tests/comparison_engine_integration.rs. For value normalization, edit src/comparison/value_compare.rs and tests around cleanup settings. For auto mapping heuristics, edit src/comparison/mapping.rs and frontend auto-pair tests when UI selection is affected.

Key source files

File	Purpose
src/comparison/engine.rs	Comparison planning, execution, summary generation.
src/comparison/rows.rs	Key extraction, flexible key matching, wildcard token logic.
src/comparison/value_compare.rs	Normalization and value difference detection.
src/comparison/mapping.rs	Header/value-based mapping suggestion.
tests/comparison_engine_integration.rs	Main comparison regression suite.