Skip to content

v1.10.0 - 5 complexity indicators + 3 new rules

Choose a tag to compare

@benzsevern benzsevern released this 08 May 23:11
50fc932

Highlights

  • 5 complexity indicators in core/indicators.py: column priors (identity + corruption scoring), sparse-match estimation, full-population matchkey hits, cross-blocking overlap, corruption score. Cheap two run eagerly; expensive three run lazily via IndicatorContext memoization.
  • IndicatorContext threads indicators through the policy/rule chain. RefitPolicy.propose gains optional ctx kwarg; controller introspects custom-policy signatures via inspect.signature for backward compat.
  • 3 new rules: rule_corruption_normalize adds normalize when blocking column has corruption + identity prior; rule_cross_blocking_disagreement proposes multi-pass when overlap is low; rule_sparse_match_expand lowers threshold + side-channel signal when sample is sparse.
  • GOLDENMATCH_AUTOCONFIG_INDICATOR_BUDGET=fast env var gates the two expensive indicators for users who prefer v1.9 wall-clock.

Benchmarks (zero-config, no LLM)

Dataset v1.9.0 v1.10.0
DBLP-ACM 0.9641 0.9641
Febrl3 0.9443 0.9443
NCVR 0.9719 0.9719
DQbench composite 62.87 66.91
T1 F1 89.3% 88.9%
T2 F1 58.7% 69.0% (+10.3pp)
T3 F1 53.8% 53.8%

Composite gain came from T2 recovery via rule_corruption_normalize and rule_cross_blocking_disagreement.

Honest framing

Shipped at fallback target (≥65), not the spec's primary target (≥70). T2 carried the gain. T3 still at 53.8% — needs richer matchkey design (queued for v1.11). T1 slight regression (-0.4pp) within measurement noise.

Breaking changes

None. Adds optional fields with default-None for v1.9 cache compat. RefitPolicy.propose gains optional ctx kwarg (custom 3-arg policies still work via signature introspection).

Full diff: v1.9.0...v1.10.0