Skip to content

Releases

v1.0.0: High-Performance Entity Resolution Pipeline (V2 Architecture)

Choose a tag to compare

@anubhavaanand anubhavaanand released this 28 Sep 09:46

Summary of v1.0.0 Release

Initial production-ready release of EntityResolve-ML, an end-to-end Machine Learning pipeline for large-scale Business Entity Resolution & Record Linkage across 14M+ noisy records.

Key Features:

  • V2 High-Recall Blocking Engine: Domain-aware multi-key inverted indexing (country prefixes, company name prefixes, address street numbers, and token sets) with generic block capping.
  • 1,000x Speedup: Replaced O(N*M) single-threaded TF-IDF CSR matrix slicing with C++ RapidFuzz re-ranking, boosting throughput to 3,190+ entities/sec and slashing full run time from 120+ hours down to ~20 minutes.
  • 90.15% Ground-Truth Candidate Recall: Significant improvement over baseline (85.36%) while retaining top-50 candidate compactness.
  • Dual-Tree GBDT Ensembling: XGBoost + LightGBM classifiers optimized for precision-oriented F0.5 metrics.
  • Memory-Bounded Architecture: Guaranteed < 3GB peak RAM footprint via generator streaming and threaded joblib parallelization.
  • Automated CI & Validation: GitHub Actions workflow running tests across Python 3.10 and 3.11 with structural graph validation.