Home Credit Default Risk

A quick project to predict whether someone will default on a loan, using the Home Credit Default Risk dataset from Kaggle. Everything lives in one notebook: HomeCreditRiskDefault.ipynb.

Why I made this

I wanted hands-on practice with a real, messy, multi-table tabular dataset instead of a toy one, dealing with class imbalance, joining/aggregating relational data, tuning a gradient boosting model, and actually calibrating its output probabilities instead of just chasing AUC. It's also a decent portfolio piece for showing an end-to-end ML workflow: cleaning, feature engineering, model selection, calibration, interpretability (SHAP).

What's actually going on here

The main application data (application_train.csv) has a TARGET column. 1 if the client defaulted, 0 if they didn't. On its own that table gets you a decent baseline, but the dataset also ships a bunch of supplementary tables (credit bureau history, past applications, monthly balances, etc.) that carry a lot of signal if you bother to join them in.

Right now the notebook only pulls in the bureau data, it aggregates bureau_balance up to the bureau level (how many months on record, how often payments were late), rolls that into bureau, and then aggregates again up to the customer level before merging it onto the main table. The other tables (credit_card_balance, installments_payments, previous_application, POS_CASH_balance) are loaded but not used yet, that's the obvious next step if you want to push the score higher.

From there it's a fairly standard pipeline:

• Median-impute the numeric features
• LightGBM classifier, tuned with RandomizedSearchCV (30 iterations, 3-fold CV, optimizing ROC-AUC)
• Wrap the best model in CalibratedClassifierCV so the predicted probabilities actually mean something, not just rank things correctly
• Evaluate on a held-out 20% test split
• SHAP summary plot to see what the model is actually keying off of
• Dump the final calibrated model to home_credit_calibrated.pkl with joblib

Results

Numbers from the latest run (246K training / 61.5K holdout applicants, 8.07% default rate):

• ROC-AUC: 0.745 best CV score during the hyperparameter search, 0.749 on the held-out test set
• Calibration: Brier score went from 0.199 (raw model) to 0.068 (after Platt scaling), a 66% improvement in how trustworthy the predicted probabilities actually are
• Top SHAP drivers: the three EXT_SOURCE_* external credit-bureau score fields dominate, together accounting for ~49% of total feature importance (EXT_SOURCE_2 and EXT_SOURCE_3 each ~18.6%, EXT_SOURCE_1 ~12.1%)

Heads up on the data

The TARGET is heavily imbalanced, only about 8% of applicants actually default. That's why the pipeline uses class_weight="balanced" in LightGBM, and it's also why you shouldn't trust accuracy as a metric here (predicting "no default" for everyone still gets you ~92% accuracy while being useless). Watch the ROC-AUC and the classification report for the minority class instead. If recall on class 1 looks bad at the default 0.5 threshold, that's expected, it's worth trying a lower threshold before assuming the model is broken.

Getting it running

You'll need the dataset CSVs sitting in a data/ folder next to the notebook (https://www.kaggle.com/c/home-credit-default-risk) dataset:

data/
  application_train.csv
  application_test.csv
  bureau.csv
  bureau_balance.csv
  credit_card_balance.csv
  installments_payments.csv
  previous_application.csv
  POS_CASH_balance.csv

Set up a virtual environment

Pick your OS below. These all assume you're starting from this project's folder.

Windows (PowerShell)

python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Linux / macOS (bash/zsh)

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

When you're done, deactivate exits the virtual environment on any OS.

Then just open HomeCreditRiskDefault.ipynb and run it top to bottom. Fair warning: the hyperparameter search step is the slow part, 30 iterations of a 3-fold CV grid on ~250k rows takes a while, so don't panic if it sits there for a bit.

Known rough edges

• The bureau-feature-engineering cell used to blow up with a MergeError if you re-ran it without restarting the kernel (it was overwriting bureau in place). That's fixed now, it merges into a separate bureau_enriched variable instead, so re-running is safe.
• The evaluation cell used to compute ROC-AUC on the thresholded 0/1 predictions instead of the raw probabilities, which silently gave a meaningless 0.5 every time. Also fixed, it uses the probabilities now.
• test.py in this folder is just a leftover scratch file, not part of the pipeline.