Skip to content

v1.2.3

Choose a tag to compare

@benzsevern benzsevern released this 31 Mar 16:14

Bug Fixes

  • Fix OOM/segfault on wide datasets: Auto-configure now caps columns at 40, fuzzy fields at 5, and skips exact matchkeys on >10K rows to prevent O(N²) self-joins
  • Fix _GEO_PATTERNS regex: city no longer matches inside "Capacity" — uses negative lookbehind instead of word boundaries to preserve city_desc/county_code matching
  • Fix golden record schema mismatch: pl.DataFrame(golden_rows) now uses schema_overrides to prevent mixed-type inference across cluster rows
  • Fix BlockingConfig.keys required when unnecessary: keys now defaults to [] with model_validator enforcing per-strategy requirements
  • Cast to Utf8 in run_dedupe_df: Prevents schema mismatch from mixed-type columns (e.g. birth_year as i64 vs str)
  • Guard auto_select against empty keys: Prevents crash when blocking config has no keys

Testing

  • Verified on UK GIAS schools register (52,288 rows, 135 columns)
  • Zero-config dedupe_df() completes in 175s with 3,854 clusters
  • 1,181 tests passing