-
-
Notifications
You must be signed in to change notification settings - Fork 13
PPRL
Match records across organizations without sharing raw data. GoldenMatch uses bloom filter encoding with HMAC salting and supports both trusted third party (TTP) and secure multi-party computation (SMC) protocols.
Party A data -> Bloom filter encoding -> Encoded vectors
Party B data -> Bloom filter encoding -> Encoded vectors
|
Dice/Jaccard similarity
|
Matched pairs
- Each field value is converted to character n-grams (e.g., bigrams)
- N-grams are hashed with multiple hash functions into a bloom filter bit array
- HMAC salting ensures the same value produces different encodings with different keys
- Encoded vectors are compared using Dice or Jaccard similarity
- Pairs above the threshold are matched
Basic bloom filter encoding. Suitable for internal use across trusted departments.
result = gm.pprl_link("a.csv", "b.csv", security_level="standard")HMAC salting with per-field keys. Prevents frequency analysis attacks.
result = gm.pprl_link("a.csv", "b.csv", security_level="high")HMAC salting + balanced padding. Padding equalizes bloom filter density to prevent inference from bit population counts.
result = gm.pprl_link("a.csv", "b.csv", security_level="paranoid")pprl_auto_config profiles your data and selects optimal fields, bloom filter parameters, and threshold.
import goldenmatch as gm
config = gm.pprl_auto_config(df)
print(config.recommended_fields) # ['first_name', 'last_name', 'zip_code', 'birth_year']
print(config.recommended_config) # PPRLConfig with optimal parametersAuto-config heuristics:
- Penalizes near-unique fields (IDs) -- they leak information
- Penalizes long fields (>15 chars) -- more bits needed
- Penalizes high-null fields -- reduce match quality
- Limits to 4 fields (beats 6 in benchmarks)
- Minimum threshold 0.85
CLI:
goldenmatch pprl auto-config data.csvfrom goldenmatch.pprl.protocol import run_pprl, compute_bloom_filters, PartyData, LinkageResult
# Compute bloom filters manually
bf_a = compute_bloom_filters(df_a, fields, config)
bf_b = compute_bloom_filters(df_b, fields, config)
# Run matching
result: LinkageResult = run_pprl(df_a, df_b, config)
print(result.clusters)
print(result.match_count)
print(result.total_comparisons)Vectorized similarity uses numpy matrix multiply (mat_a @ mat_b.T) for bloom filter Dice -- 13x faster than per-pair Python loops.
| Strategy | Precision | Recall | F1 | Privacy |
|---|---|---|---|---|
| Normal fuzzy (baseline) | 56.5% | 74.6% | 64.3% | None |
| PPRL manual tuning | 98.2% | 82.6% | 89.8% | Per-field HMAC |
| PPRL auto-config | 99.7% | 86.1% | 92.4% | Per-field HMAC |
| PPRL paranoid | 98.9% | 76.0% | 86.0% | HMAC + balanced |
| Strategy | Precision | Recall | F1 |
|---|---|---|---|
| PPRL auto-config | 64.0% | 93.8% | 76.1% |
Auto-configuration beats manual tuning on both datasets. Zero-config PPRL profiles your data and picks optimal parameters automatically.
⚡ GoldenMatch — Entity resolution toolkit | PyPI | GitHub | Open in Colab | MIT License
🟡 Golden Suite (Monorepo)
Suite Packages
- GoldenCheck · data quality
- GoldenFlow · transforms
- GoldenPipe · orchestrator
- InferMap · schema mapping
Getting Started
- Installation
- Quick Start
- Auto-Config Controller · enhanced through v1.12
- Configuration
- Verification · new in v1.5
- CLI Reference
Core Concepts
AI Integration
Advanced
- PPRL
- Domain Packs
- Streaming / CDC
- Database Integration
- GPU & Vertex AI
- REST API
- Interactive TUI
- Web UI · new in v1.7
- Evaluation
Reference
pip install goldenmatch
npm install goldenmatch