InspectNet

Surface defect classification on the NEU dataset using MobileNetV2 with Grad-CAM explainability, served over a FastAPI REST API.

what this does

Manual visual inspection on a production line is slow and inconsistent — different inspectors catch different things. This project trains a MobileNetV2-based classifier to sort steel surface defects into 6 categories, then wraps it in a FastAPI server so you can POST an image and get back a prediction + a Grad-CAM heatmap showing where the model is looking.

The 6 defect classes (from the NEU Surface Defect Database):

Crazing | Inclusion | Patches | Pitted_Surface | Rolled-in_Scale | Scratches

Each class has 300 grayscale images, 1,800 total. Small dataset, so transfer learning matters a lot here.

why MobileNetV2

• it's lightweight — designed for mobile/edge, which matters if this ever runs on-device near a production line
• ImageNet features transfer well — even though NEU images are grayscale industrial textures, the low-level filters (edges, textures, gradients) from ImageNet are surprisingly useful. the model doesn't need to learn "what an edge is" from scratch
• trained a from-scratch CNN baseline for comparison: transfer learning gave ~15-20% higher accuracy, which makes sense given we only have ~1,200 training images

the 100% accuracy problem (and how i fixed it)

The original version of this project reported 100% test accuracy. That was fake — caused by data leakage.

What went wrong: NEU images are crops from the same steel strips. Images with adjacent filenames (like crazing_1.jpg and crazing_2.jpg) often come from the same physical sample and look nearly identical. A naive random train_test_split scatters these near-duplicates across train and test sets. The model memorizes the texture, sees the same texture at test time, and "correctly" predicts it. But it hasn't learned anything generalizable.

How i fixed it:

1. group-aware splitting — instead of splitting images randomly, we group near-duplicate clusters using perceptual dhash hamming-distance clustering and keep each cluster in a single split. all crops from the same steel strip stay in the same partition
2. multi-layout dataset loader — supports both NEU-DET flat IMAGES/ folders and NEU-CLS per-class folders automatically
3. correct preprocessing — MobileNetV2 expects input in [-1, 1] range (via preprocess_input), not [0, 1]. using / 255.0 silently degrades feature quality
4. two-phase fine-tuning — phase 1: freeze backbone, train head at lr=1e-3. phase 2: unfreeze last 30 layers, fine-tune at lr=1e-4

benchmark & test accuracy

Evaluated on an honest 275-image test split (near-duplicate clusters isolated to train set):

Overall Test Metrics:
  Accuracy:        88.73%  (0.8873)
  Precision:       90.63%
  Recall:          88.41%
  Macro F1-Score:  88.44%
  Weighted F1:     88.00%

Per-Class Classification Breakdown:
                 precision    recall  f1-score   support
        Crazing       0.90      1.00      0.95        45
      Inclusion       0.82      0.95      0.88        56
        Patches       1.00      0.66      0.79        44
 Pitted_Surface       0.91      0.98      0.94        42
Rolled-in_Scale       0.81      0.96      0.88        45
      Scratches       1.00      0.77      0.87        43

Notice that for QA inspection, recall is critical (avoiding false negatives so defective steel does not pass quality checks). 4 out of 6 defect classes achieve 94.6% to 100.0% recall.

project structure

InspectNet/
├── app/
│   ├── main.py             # FastAPI app + model loading
│   ├── core/config.py      # env-based config
│   ├── routers/predict.py  # /predict, /predict/explain, /health
│   └── schemas/            # pydantic response models
├── src/
│   ├── data/
│   │   ├── loader.py       # dataset loading (supports NEU-DET & NEU-CLS)
│   │   └── preprocess.py   # normalization, augmentation
│   ├── models/
│   │   ├── cnn.py          # MobileNetV2 transfer + baseline CNN
│   │   └── train.py        # two-phase training pipeline
│   ├── evaluation/
│   │   └── metrics.py      # confusion matrix, per-class metrics
│   └── inference/
│       └── predictor.py    # prediction + Grad-CAM engine
├── notebooks/
│   └── 01_eda.py           # dataset EDA + duplicate check
├── models/
│   └── defect_cnn.keras    # trained weights (native Keras format)
├── tests/
│   └── test_api.py
├── Dockerfile
└── requirements.txt

setup

git clone https://github.com/<your-username>/InspectNet.git
cd InspectNet

python -m venv venv
source venv/bin/activate   # windows: venv\Scripts\activate
pip install -r requirements.txt

dataset layout

Place the NEU dataset in data/raw/. Both layout styles work automatically:

• NEU-DET style: data/raw/IMAGES/crazing_1.jpg, inclusion_1.jpg, ...
• NEU-CLS style: data/raw/Crazing/Cr_1.bmp, data/raw/Inclusion/In_1.bmp, ...

run EDA

python -m notebooks.01_eda --data-dir data/raw

Checks class distribution, plots sample images, and scans for near-duplicate images using dhash. Figures saved to notebooks/figures/.

train

# transfer learning (MobileNetV2) — recommended
python -m src.models.train --data-dir data/raw --epochs 50 --batch-size 16

# train both transfer + baseline for comparison
python -m src.models.train --data-dir data/raw --epochs 50 --run-baseline

Model saved to models/defect_cnn.keras. Confusion matrix and training history plots saved to models/.

run the API

uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

Interactive API docs at http://localhost:8000/docs.

run tests

pytest tests/ -v

API endpoints

• GET /health — returns {"status": "ok", "model_loaded": true/false}
• POST /predict — upload an image, returns prediction + confidence + inference latency
• POST /predict/explain — returns prediction plus a base64-encoded Grad-CAM heatmap PNG showing image regions that influenced the model

Grad-CAM explainability

Grad-CAM (Gradient-weighted Class Activation Mapping):

1. forward pass through MobileNetV2
2. compute gradients of predicted score w.r.t. the last conv layer feature maps
3. global-average-pool gradients to derive channel importance weights
4. compute weighted combination of feature maps + apply ReLU
5. upsample heatmap to match image size and overlay

On correct predictions, heatmaps highlight defect structures (e.g. scratch lines, pitted clusters). On misclassifications, heatmaps reveal when the model focuses on background lighting or ambiguous surface texture.

docker

docker build -t inspectnet .
docker run -p 8000:8000 inspectnet

design decisions

• group-aware split over random split: NEU images are sequential crops. random split leaks near-duplicates. group-based split gives honest numbers.
• batch size 16: small dataset (~1,200 training images) benefits from more gradient updates per epoch.
• dropout 0.4: higher dropout prevents overfitting on small sample sizes.
• 128x128 input: preserves surface texture detail while training efficiently on CPU/GPU.
• augmentation: flips + rotation + subtle brightness/contrast jitter.

known limitations

• grayscale only — NEU dataset images are grayscale
• no clean surface class — always predicts 1 of 6 defect types (no "no-defect" class)
• single defect per image — designed for single-label surface crops