Skip to content

v2.0.1 — Reliable leakage signals in model-diagnostician

Choose a tag to compare

@jiachengwang-punch jiachengwang-punch released this 30 May 02:46
· 3 commits to main since this release

Patch release refining the model-diagnostician agent after an end-to-end validation run on the Telco Customer Churn dataset.

Changed

  • model-diagnostician leakage-recheck signals reordered by reliability:
    1. Anomalous score jump (AUC/R² spikes when a feature is added) — primary suspicion.
    2. Feature-vs-label consistency check (>95% ⇒ the feature is a relabeling/downstream of the target).
    3. Importance via gain/SHAP, not split-count.
  • Documented that a binary leakage feature can be buried under continuous features in default split-count importance, making split-count unreliable for leakage detection — the gain/SHAP view (plus the score-jump and consistency signals) is what reliably exposes it.
  • Updated both the portable role-prompt (references/agents/model-diagnostician.md) and the plugin wrapper (agents/model-diagnostician.md).

Full changelog: see CHANGELOG.md