Skip to content

v1.2.5 — Open Dataset Profiler: drift comparison, column mapping & reference baselines

Choose a tag to compare

@yakew7 yakew7 released this 13 Jul 07:34

Open Dataset Profiler — a major capability wave

This release rounds out the Open Dataset Profiler with six enhancements. All of
them run through one shared spec (faircode/SPEC.md), so the faircode CLI and
the in-browser profiler produce identical numbers.

⚖ Compare two datasets for representation drift (#60)

faircode compare train.csv prod.csv — or drop an A/B pair in the web profiler —
to see how each demographic dimension shifted. Reports the Population Stability
Index (PSI), Total Variation Distance, per-group share shifts, and
appeared/disappeared groups, flagging significant drift and score drops.

🗺 Manual column mapping (#62)

When auto-detection misses an oddly-named column (gndr, patient_region_code),
override it: faircode profile data.csv --map gndr=sex, or editable per-column
dropdowns on the web that re-run the audit instantly.

📊 Reference-population baseline (#56)

Score a dataset against an external population (e.g. Census age×sex), not just
internal balance: faircode profile data.csv --reference census.csv or a web
upload. Surfaces per-group expected-vs-actual deltas and flags
under-representation relative to who the model will actually serve.

✕ Choosable intersection pair (#58)

Cross any two demographic columns, not just the first two detected:
faircode profile data.csv --cross race,age or two dropdowns in the web UI.

🔗 Chi-squared proxy hints (#61)

faircode profile data.csv --proxy-hints flags strongly-associated column pairs
— a "this may be a proxy for a protected attribute" signal — with p-values and
Cramér's V. Opt-in via the optional scipy extra; informational, never scored.

🎚 Tunable thresholds (#63)

--min-share, --intersection-floor, --imbalance-flag, and --missing-flag
let auditors set their own bars without editing source.

Install: pip install -e ".[proxy]" for the chi-squared extra.
Full changelog: see CHANGELOG.md → [1.2.5].