Retail Demand Forecasting

Neural network forecasting for large-scale retail time series

A demand forecasting system built around a synthetic Tesco-scale retail dataset: 50 stores, 200 products, 3 years of daily sales. Five different neural forecasting architectures, a set of statistical baselines to compare them against, probabilistic and hierarchical forecasting, and a production-style backtesting and monitoring layer, not just a single notebook that trains one model and stops.

I built five architectures instead of one so I'd actually understand the trade-offs between them (LSTM vs. a fully parallelizable TCN vs. an attention-based Transformer vs. N-BEATS' pure basis-expansion approach) rather than picking whichever one a tutorial happened to use.

Architecture

flowchart LR
    A[UCI Online Retail II] --> C[Feature engineering]
    B[Synthetic Tesco-scale generator] --> C
    B --> Z[PySpark ETL]
    C --> D[Statistical baselines]
    C --> E[Neural network models]
    C --> F[Probabilistic forecasting]
    D --> G[Backtesting & evaluation]
    E --> G
    F --> G
    G --> H[Hierarchical reconciliation]
    G --> I[Django dashboard]
    H --> I
    G --> J[Monitoring & drift detection]

Loading

The five neural models

Model	Architecture	What it's good at
LSTM/GRU	Seq2seq with Bahdanau attention, teacher forcing	Long-range temporal dependencies
TCN	Dilated causal convolutions, residual connections	Parallelizable training, large receptive field
Transformer	TFT-inspired: variable selection, multi-head attention, gated residual networks	Interpretable feature importance
N-BEATS	Trend + seasonality + generic stacks, basis expansion	No manual feature engineering needed
DeepAR	Autoregressive LSTM with Gaussian output	Probabilistic prediction intervals

Statistical baselines

Seasonal naive, AutoARIMA, AutoETS, and Prophet (with UK holidays), so the neural models have something honest to beat rather than being compared only against each other.

Probabilistic and hierarchical forecasting

DeepAR outputs a Gaussian (mu, sigma) per step, evaluated with CRPS, interval coverage, and pinball loss instead of just a point forecast. Reconciliation supports bottom-up, top-down, and MinTrace (Ledoit-Wolf shrinkage), verified for coherency across the Product -> Category -> Department -> Store -> Region hierarchy.

Production pipeline

• Time-aware cross-validation with an expanding window and an explicit gap to prevent leakage
• A dedicated leakage-detection check, not just an assumption that the split is clean
• PSI-based drift monitoring and a retraining scheduler (time-based, drift-triggered, or performance-based)
• MLflow experiment tracking

Data

• UCI Online Retail II: real transaction data from a UK online retailer (UCI Machine Learning Repository).
• Synthetic Tesco-scale data: 50 stores across 5 UK regions, 200 products across 40 categories and 10 departments, 3 years of daily sales with weekly/yearly seasonality, UK holidays, promotions, trend, and noise, generated with a hierarchical structure so reconciliation has something real to reconcile.

Project structure

config/config.yaml         all hyperparameters and pipeline config
data/raw/, data/processed/ UCI download cache and generated parquet files
notebooks/                 8 notebooks, ingestion through production monitoring
src/
  data_generator.py         synthetic data generation
  feature_engineer.py        lag, rolling, calendar, holiday features
  baselines.py                ARIMA, ETS, Prophet
  lstm_model.py, tcn_model.py, transformer_model.py, nbeats_model.py, probabilistic.py
  hierarchical.py             MinTrace reconciliation
  backtesting.py               time-aware CV, leakage detection
  evaluator.py                  MAE, RMSE, MAPE, SMAPE, MASE, CRPS
  monitoring.py                 PSI drift detection, retraining scheduler
spark/data_pipeline.py     PySpark ETL at scale
webapp/                     Django dashboard (see below)
main.py                      CLI pipeline runner

Quick start

git clone https://github.com/Falli007/TimeSeriesForecasting.git
cd TimeSeriesForecasting

python -m venv venv
source venv/bin/activate      # venv\Scripts\activate on Windows
pip install -r requirements.txt

python prepare_data.py        # downloads UCI data, generates the synthetic dataset
python main.py --pipeline full

Run a single stage instead of the full pipeline with python main.py --pipeline data or --pipeline neural --device cuda.

Run the Spark ETL separately with python spark/data_pipeline.py.

Django dashboard

cd webapp
python manage.py runserver

Open http://localhost:8000, pick a store, product, and model, and generate a forecast. If no trained model or processed dataset is present yet, the dashboard falls back to a demo forecast generator so the UI is still fully explorable without running the training pipeline first.

Notebooks

The 8 notebooks under notebooks/ walk through the whole pipeline in order: data ingestion and EDA, feature engineering, statistical baselines, LSTM/GRU, TCN, Transformer, probabilistic and hierarchical forecasting, then evaluation and production monitoring.

Evaluation

Point forecasts are scored with MAE, RMSE, MAPE, SMAPE, and MASE (whether a model beats seasonal naive). Probabilistic forecasts are scored with CRPS, interval coverage, and pinball loss. main.py --pipeline evaluate builds a leaderboard across every model rather than reporting a single number in isolation, since the actual numbers depend on which run's synthetic data and random seed you're looking at.

GPU support

python main.py --pipeline neural --device cuda

CUDA is detected automatically if available; every neural model falls back to CPU otherwise.

Technology stack

Python 3.10+, PyTorch, statsforecast, Prophet, PySpark, Django, MLflow, Chart.js.

License

Portfolio and demonstration use.