Explainable ML NIDS — UNSW-NB15 (PyCharm / Local)

Student: Muhammad Hasnain Haider
Supervisor: Michael Ajao-Olarinoye
Module: COM748 Masters Research Project

Local, error-free PyCharm project

• All 5 models (LR, DT, RF, XGBoost, MLP)
• Binary + multiclass evaluation
• Feature importance, permutation importance, SHAP
• Holistic ranking
• Full HTML / DOCX / Markdown report package

---

Project structure

UNSW_NB15_PyCharm/
├── main.py                          ← RUN THIS (full pipeline)
├── requirements.txt
├── README.md
├── config/
│   └── settings.py                  ← paths, seeds, hyperparameters
├── data/                            ← CSVs auto-downloaded here
├── src/ids_framework/
│   ├── data_loader.py               ← download, EDA, preprocess
│   ├── models.py                    ← train + evaluate 5 models
│   ├── explainability.py            ← FI, permutation, SHAP, ranking
│   ├── report.py                    ← HTML + DOCX + MD report
│   └── utils.py                     ← metrics, plots, helpers
├── scripts/
│   ├── run_experiments.py           ← experiments only
│   └── run_report.py                ← report only
├── results/
│   ├── figures/                     ← all PNG plots
│   ├── tables/                      ← all CSV leaderboards
│   └── models/                      ← .joblib / .keras
└── report_output/                   ← final reports

---

PyCharm setup (step by step)

1. Open the project

1. Open PyCharm
2. File → Open → select the UNSW_NB15_PyCharm folder
3. Trust the project if prompted

2. Create a virtual environment

1. File → Settings → Project → Python Interpreter
2. Click the gear → Add → New environment (Virtualenv)
3. Base interpreter: Python 3.10+ (3.11 recommended)
4. Location: UNSW_NB15_PyCharm/.venv (default is fine)
5. OK

3. Install dependencies

Open the PyCharm Terminal (bottom panel) and run:

pip install -r requirements.txt

Windows note: if tensorflow install is slow, that is normal.
Apple Silicon: use the official TensorFlow macOS wheel (pip install tensorflow-macos if needed).

4. Mark sources (optional but recommended)

• Right-click src → Mark Directory as → Sources Root
• Working directory for runs must be the project root (UNSW_NB15_PyCharm)

5. Run the full pipeline

Either:

• Open main.py → right-click → Run 'main'
• Or use the pre-made run config Full Pipeline (in .idea/runConfigurations/)

Or in Terminal:

python main.py

---

Commands

Command	What it does
python main.py	Full pipeline (data → models → XAI → report)
python main.py --skip-report	Experiments only
python main.py --report-only	Rebuild report from existing results/
python main.py --no-multiclass	Skip multiclass block
python main.py --no-explainability	Skip SHAP / FI / permutation
python main.py --show-plots	Also pop up matplotlib windows
python scripts/run_experiments.py	Same as --skip-report
python scripts/run_report.py	Same as --report-only

---

Dataset

On first run the script auto-downloads:

• data/UNSW_NB15_training-set.csv
• data/UNSW_NB15_testing-set.csv

If download fails (firewall / offline):

1. Get the CSVs from https://research.unsw.edu.au/projects/unsw-nb15-dataset
2. Place them into the data/ folder with the exact names above
3. Re-run python main.py

---

Expected runtime (laptop / desktop)

Stage	Approx. time
Download + EDA + preprocess	2–5 min
Binary 5 models	5–20 min
Multiclass 5 models	10–30 min
SHAP + permutation	5–20 min
Report generation	1–3 min
Total	~25–70 min

---

Outputs

results/
├── figures/     # cm_*, roc_*, fi_*, shap_*, holistic_ranking.png, ...
├── tables/      # binary_model_comparison.csv, multiclass_..., final_holistic_..., etc.
└── models/      # logistic_regression.joblib, xgboost.joblib, mlp_binary.keras, ...

report_output/
├── UNSW_NB15_Explainable_IDS_Full_Report.html
├── UNSW_NB15_Explainable_IDS_Full_Report.docx
├── UNSW_NB15_Explainable_IDS_Full_Report.md
├── findings.json
└── figure_catalogue.csv

UNSW_NB15_Full_Report_Bundle.zip   # report + experiment_results

PDF from HTML

Open the HTML file in Chrome/Edge → Ctrl+P / Cmd+P → Save as PDF
(enable Background graphics).

---

Mapping to dissertation proposal

Proposal objective	Module
Obtain / explore / preprocess UNSW-NB15	data_loader.py
LR, DT, RF, XGBoost, MLP + metrics + times	models.py
FI, permutation importance, SHAP	explainability.py
Compare accuracy × explainability × efficiency	explainability.holistic_ranking
Full write-up of every table/figure	report.py

---

Configuration knobs

Edit config/settings.py to change:

• USE_SMOTE (default False — class weights)
• Model hyperparameters (RF_PARAMS, XGB_*, MLP epochs)
• SHAP / permutation sample sizes
• SHOW_PLOTS

Do not change SEED = 42 if you want reproducibility with the Colab run.

---

Troubleshooting

Problem	Fix
ModuleNotFoundError: config	Set Working directory = project root in Run Configuration
ModuleNotFoundError: tensorflow	pip install tensorflow (or tensorflow-macos on Apple Silicon)
ModuleNotFoundError: shap	pip install shap
Dataset download fails	Manually put CSVs into data/
SHAP KernelExplainer slow	Normal for MLP; reduce SHAP_MLP_EXPLAIN in settings
Out of memory on RF/XGB	Close other apps; or lower n_estimators in settings
Plots not appearing	Default saves to disk only; use --show-plots if needed

---

Academic references (IEEE)

[1] N. Moustafa and J. Slay, UNSW-NB15, MilCIS, 2015.
[2] J. R. Quinlan, Induction of Decision Trees, 1986.
[3] L. Breiman, Random Forests, 2001.
[4] T. Chen and C. Guestrin, XGBoost, KDD, 2016.
[5] S. M. Lundberg and S.-I. Lee, SHAP, NeurIPS, 2017.