Adaptive Drift Intelligence Pipeline

A churn-prediction pipeline that automatically detects, quantifies, and mitigates data drift between training and inference data before training a gradient-boosted model. Built for the NAISC × Singtel 2026 Adaptive Drift Intelligence Challenge (Team junyiding).

The core idea: when the test distribution has shifted away from the training distribution, a model that ignores the shift degrades badly. This pipeline measures the drift per feature, applies the right mitigation to each drifted feature, and adapts to the unlabeled test distribution — without any hand-coded, dataset-specific rules.

Results

Ablation on the challenge dataset (test AU-PRC), each tier adding to the previous:

Pipeline tier	Test AU-PRC
Baseline (raw features, no mitigation)	0.759
+ Drift mitigation + feature engineering	0.867
+ Iterative importance-aware mitigation	0.870
+ Self-training on the test distribution	0.871

Drift handling is responsible for the bulk of the lift: +0.11 AU-PRC over the raw baseline.

How it works

The pipeline (src/main.py) runs in five stages:

1. Ingestion & profiling — loads data with Polars, infers feature types (numerical / categorical / temporal), coerces and downcasts dtypes, and subsamples a stratified "decision sample" so drift analysis stays fast on large datasets. A dynamic row cap keeps end-to-end runtime within budget.
2. Drift detection (drift_detector.py) — per-feature statistical tests: PSI and Kolmogorov–Smirnov for numerical features, Jensen–Shannon divergence and Chi² for categoricals. Each feature is scored and labelled low / moderate / high drift.
3. Drift mitigation (mitigation.py) — chooses a mitigation per drifted feature: category normalization / abbreviation matching, quantile matching, rank transforms, or dropping features whose drift can't be repaired.
4. Feature engineering (feature_engineer.py) — derives features dynamically (no hardcoded column names), with correlation pruning and top-feature interaction terms.
5. Modelling & adaptation (model.py) — trains a LightGBM classifier, then:
   • Iterative mitigation — re-mitigates features that are both high-importance and still drifted, and drops low-importance high-drift features.
   • Self-training — pseudo-labels high-confidence test rows and augments the training set to adapt toward the (unlabeled) test distribution.

Everything is data-driven — the same code handles arbitrary tabular schemas.

Setup

pip install -r requirements.txt

Requires Python 3.10+.

Usage

python ./src/main.py --train_data_filepath <train.csv> --test_data_filepath <test.csv>

Useful flags:

• --ablation — capture AU-PRC at each pipeline tier and write ablation_results.csv
• --seed <int> — random seed for LightGBM (default 42)
• --recency-decay — weight training rows by recency (requires a temporal column)

Outputs

• Console: drift detection & mitigation summary, per-stage runtime, and baseline-vs-mitigated AU-PRC
• prediction.csv — CustomerID and churn probability_score
• model.joblib — the trained LightGBM model
• dashboard_artifacts.joblib — cached pipeline outputs for the dashboard

Dashboard

An interactive Streamlit dashboard visualizes drift, mitigation decisions, and model performance:

python -m streamlit run src/dashboard.py

Opens at http://localhost:8501. It reads dashboard_artifacts.joblib, so run the pipeline once first.

Project structure

src/
  main.py             Pipeline orchestration (five stages)
  profiler.py         Data loading, type inference, dtype downcasting
  drift_detector.py   Per-feature drift statistics (PSI, KS, JSD, Chi²)
  mitigation.py       Per-feature drift mitigation strategies
  feature_engineer.py Dynamic feature generation & interactions
  model.py            LightGBM training, evaluation, prediction
  dashboard.py        Streamlit drift/performance dashboard
  polars_compat.py    Polars/pandas interop helpers
  utils.py            Logging, timing, subsampling helpers

Notes

The competition dataset is not included in this repository. Point --train_data_filepath / --test_data_filepath at your own tabular CSVs (binary target column, optional temporal column) to run the pipeline.