Skip to content

v0.1.0 — The leakage audit

Choose a tag to compare

View all tags
@lfariabr lfariabr released this 12 Jul 21:41
v0.1.0
7ab4100

Honest-metrics release. Auditing the same pipeline for MLN601 Assessment 2 surfaced 1,177 exact duplicate rows in the raw UCI files crossing the train/test split and inflating every published v0.0.1 metric.

Changed

  • Dedup before split in ml/train.py: 6,497 raw rows → 5,320 unique. Provenance (raw_rows, duplicates_removed) recorded in metrics.json and served at GET /model/info.
  • Grade model is now the assessment-approved balanced tree: DecisionTreeClassifier(gini, max_depth=5, min_samples_leaf=20, class_weight="balanced"). It trades some false alarms for catching 73% of genuinely low wines (was 59% at default weighting) — in a screening problem, the miss is the expensive error.
  • Honest re-trained metrics:
Metric v0.0.1 v0.1.0
Regression R² 0.50 0.41
Regression RMSE 0.61 0.66
Classification ROC-AUC 0.81 0.79
Classification sensitivity (low) — 0.73
Classification specificity (high) — 0.73

The models did not get worse — the evaluation got corrected: v0.0.1 was graded partly on rows it had already seen.

  • metrics.json and /model/info now include sensitivity, specificity, F1 and the full confusion matrix; the Streamlit model card and About page lead with them.
  • Tests re-pinned to the deduplicated numbers, plus a gate test mirroring the assessment approval criteria (AUC ≥ 0.75, sensitivity ≥ 0.70, specificity ≥ 0.70). 24 tests, ruff clean.

Live

Assets 2
Loading