CNN From Scratch — MNIST Classifier

A complete, production-quality project demonstrating how to build a Convolutional Neural Network entirely from scratch in Python, utilizing only NumPy for tensor operations and Matplotlib for visualizations.

It trains on the MNIST dataset with a standard ConvNet architecture and manual backpropagation routines (no deep learning libraries like PyTorch or TensorFlow are used).

Architecture

The default implementation mirrors a classical LeNet-style architecture adapted for modern layers (using ReLU and Dropout):

Input: (N, 1, 28, 28)
↓ Convolution (3x3 Kernel, 32 Filters, Same Padding)
↓ ReLU Activation
↓ MaxPooling (2x2 Pool, Stride 2)
↓ Convolution (3x3 Kernel, 64 Filters, Same Padding)
↓ ReLU Activation
↓ MaxPooling (2x2 Pool, Stride 2)
↓ Flatten
↓ Dense (128 Units)
↓ ReLU Activation
↓ Dropout (p=0.3)
↓ Dense (10 Units)
↓ Softmax Activation

Mathematical Implementation Reference

All gradients are computed analytically.

• Dense Layer: $dW = X^T dZ$, $db = \sum dZ$, $dX = dZ W^T$
• Conv2D / MaxPool2D (im2col): A naive loop over pixels is too slow in Python. This project utilizes the im2col technique, reducing spatial convolution to a highly optimized matrix multiplication $O = W_{col} X_{col}$. Backward pass is achieved via col2im.
• Softmax + Categorical Cross-Entropy: Softmax logic applies numerically stable logs (subtracting the max logit). The combined gradient reduces cleanly down to $\frac{1}{N} (P - Y)$ where $P$ are predictions and $Y$ are one-hot encoded targets.

Installation

pip install -r requirements.txt

Usage

For standard execution (auto-downloads MNIST and trains for 10 epochs):

python main.py --epochs 10 --batch-size 64 --lr 0.001 --optimizer adam

Other available arguments:

python main.py --help
# Options:
# --train PATH          Path to training data CSV (if custom)
# --test PATH           Path to test data CSV (if custom)
# --lr-schedule STR     One of: constant, step, cosine (default: cosine)
# --optimizer STR       One of: sgd, momentum, adam (default: adam)
# --dropout FLOAT       Dropout rate
# --save-model          Save weights after training
# --load-model PATH     Load weights and skip training (eval only)
# --no-plots            Skip plot generation

Visualizations and Results

The training pipeline generates comprehensive visual output to monitor network health:

• loss_curve.png: Tracking Categorical Cross-entropy convergence.
• accuracy_curve.png: Tracking Top-1 Match across train and validation sets.
• lr_schedule.png: Plotting metric drop if Step or Cosine annealing is selected.
• confusion_matrix.png: Heatmap exposing misclassifications explicitly.

Following training, a dense evaluation map exposes accuracy, macro-precision, macro-recall, and an F1 score per digit.