Repository navigation
v2.0.1 — Reliable leakage signals in model-diagnostician
Patch release refining the model-diagnostician agent after an end-to-end validation run on the Telco Customer Churn dataset.
Changed
model-diagnosticianleakage-recheck signals reordered by reliability:- Anomalous score jump (AUC/R² spikes when a feature is added) — primary suspicion.
- Feature-vs-label consistency check (>95% ⇒ the feature is a relabeling/downstream of the target).
- Importance via gain/SHAP, not split-count.
- Documented that a binary leakage feature can be buried under continuous features in default split-count importance, making split-count unreliable for leakage detection — the gain/SHAP view (plus the score-jump and consistency signals) is what reliably exposes it.
- Updated both the portable role-prompt (
references/agents/model-diagnostician.md) and the plugin wrapper (agents/model-diagnostician.md).
Full changelog: see CHANGELOG.md