features comparison rules

Comparison rules

Active contributors: Douwe de Vries

Purpose

Comparison rules let users tune how row keys and values are normalized before matching. Rules include missing values, flexible row-key matching, case, whitespace, numeric equivalence, decimal rounding, and date normalization.

Directory layout

frontend/src/config/normalization.ts
frontend/src/components/mapping-config/NormalizationPanel.tsx
src/data/types.rs
src/comparison/value_compare.rs
src/comparison/rows.rs

Key abstractions

Name	File	Description
ComparisonNormalizationConfig	src/data/types.rs, frontend/src/types/api.ts	Shared normalization config shape.
INITIAL_NORMALIZATION_CONFIG	frontend/src/config/normalization.ts	Frontend defaults matching backend serde defaults.
NormalizationPanel	frontend/src/components/mapping-config/NormalizationPanel.tsx	User-facing rule controls.
normalize_value	src/comparison/value_compare.rs	Backend normalization pipeline for comparisons.
classify_flexible_key_match	src/comparison/rows.rs	Flexible key candidate classification.

How it works

The backend applies null-token logic first, then date, decimal, numeric, and case normalization where enabled. Key normalization returns None for nullish keys, which creates ignored rows rather than false matches.

graph TD
    Value[Raw value] --> Trim[optional trim]
    Trim --> Null[empty/null token check]
    Null --> Date[optional date normalization]
    Date --> Number[optional numeric equivalence/rounding]
    Number --> Case[optional lower-case]
    Case --> Compare[compare normalized values]

Loading

Flexible row-key matching is separate from value normalization. It classifies candidate key pairs in src/comparison/rows.rs and applies caps in src/comparison/engine.rs before execution.

Integration points

The UI sends normalization inside CompareRequest from frontend/src/hooks/useComparisonWorkflow.reducer.ts. Backend validation in src/backend/validation.rs allows unequal key-column counts only when flexible_key_matching is enabled.

Entry points for modification

Add a rule to ComparisonNormalizationConfig in Rust and TypeScript, update defaults in src/data/types.rs and frontend/src/config/normalization.ts, add controls in NormalizationPanel, and add backend plus frontend tests.

Key source files

File	Purpose
src/data/types.rs	Normalization config structs and defaults.
src/comparison/value_compare.rs	Value normalization and difference detection.
src/comparison/rows.rs	Flexible key matching helpers.
frontend/src/config/normalization.ts	Frontend default config.
frontend/src/components/mapping-config/NormalizationPanel.tsx	User controls.