Skip to content
BranchesTags

Repository files navigation

SmartContainer Risk Engine

AI/ML-based Container Shipment Risk Prediction System
Built for the Nirma University Hackathon

Live API GitHub

Problem Understanding

Containerized cargo forms the backbone of global trade. Customs authorities face a dual challenge: maximize inspection efficiency while minimizing disruption to legitimate trade flow.

Traditional rule-based screening fails because:

  • Static rules miss hidden irregularities that evolve with trade patterns
  • High false-positive rates generate unnecessary inspections and delays
  • Manual checks don't scale with modern trade volumes

This system addresses the hackathon challenge:

Design an AI/ML-based system that processes structured container shipment data, identifies anomalous patterns, predicts inspection risk, categorizes containers into risk levels, and provides basic explainability for each prediction.

Solution at a Glance

SmartContainer Risk Engine is an end-to-end machine learning system that identifies high-risk containerized shipments for customs authorities. It combines a LightGBM + XGBoost ensemble with Isolation Forest anomaly detection and rule-based signals to produce a composite Risk Score (0–100) for each declaration.
Training data 54,000 historical container declarations
Test data 8,481 real-time declarations
Critical detection rate ~0.9% flagged — precision-tuned (recall ≥ 0.93, minimizing false alarms)
Live API https://hp25-container-risk-engine.hf.space/api/health
GitHub https://github.com/HARRY5D/Container_risk_engine

Approach & Methodology

1. Problem Framing

The dataset has a severe class imbalance (critical shipments are rare by design). The primary objective is maximising recall on the Critical class while keeping false alarms low enough to be operationally practical. This led to choosing F2-Score as the primary optimisation metric (penalises false negatives twice as heavily as false positives).

2. Feature Engineering (16 raw → 26 model features)

Category Features
Weight anomaly Weight_Discrepancy, Weight_Discrepancy_Ratio, Log_Weight_Discrepancy
Value anomaly Value_Per_Kg, Log_Value, Value_x_Discrepancy
Temporal Dwell_Time_Hours, Log_Dwell, Dwell_Flag (> 72 h), Declaration_Hour
Trade identity Importer_ID_Freq, Exporter_ID_Freq, Importer_ID_OE, Exporter_ID_OE
Route Country_Pair_Freq, Origin_Country_OE, Destination_Port_OE
Commodity HS_Category (2-digit prefix), HS_Code_OE
Interaction Discrepancy_x_Dwell, Value_x_Discrepancy
Anomaly Anomaly_Score (Isolation Forest), Is_Anomaly flag

3. Anomaly Detection

An Isolation Forest (n_estimators=300, contamination=0.01) is trained on historical data to surface unsupervised anomalies independent of the supervised labels. Its normalised score becomes an input feature to the ensemble, so the model learns how much weight to give unsupervised signals per sample.

Patterns it captures:

  • Significant weight discrepancies (measured vs. declared)
  • Unusual value-to-weight relationships
  • Rare importer/exporter identities
  • Unusual dwell times for a given trade corridor

4. Class Balancing

Step Action
Undersampling 5:1 majority-to-minority ratio
Oversampling BorderlineSMOTE on minority class
Result ~1:1 balanced training set

5. Ensemble Model

Component Weight Rationale
LightGBM 60% Faster training, handles high-cardinality categoricals well
XGBoost 40% Strong generalisation, different error profile to LGB

Soft-vote probability fusion. Threshold tuned on validation PR curve to maximise F2.

6. Risk Score Formula

Risk_Score = 70% × (ML_Probability × 100)
           + 30% × Rule_Signal

Rule_Signal = weighted sum of:
  - Weight discrepancy ratio
  - Value anomaly flag
  - Dwell time flag (> 72 h)
  - Isolation Forest anomaly score

Score > 61 → Critical. Threshold is data-driven (optimised F2 on validation set, not a subjective constant).

7. Explainability

Each prediction includes a human-readable Explanation_Summary generated from the top contributing signals:

"Risk factors: high weight discrepancy of 350 kg (ratio=0.70);
 excessive dwell time (120 hours); rare importer with few historical records."

Model Performance

Evaluation Metrics (Validation Set — 10,800 samples)

Metric Value Notes
PR-AUC 0.9404 Primary metric — captures precision-recall trade-off under class imbalance
ROC-AUC 0.994 Near-perfect class separation
F2-Score 0.912 Optimisation target — penalises missed Critical containers 2×
Critical Precision 0.829 ~83% of flagged containers are genuinely critical
Critical Recall 0.936 Only 6.4% of actual Critical containers missed
Optimal Probability Threshold 0.672 Tuned on validation PR curve
Critical Risk Score Threshold 61 Data-driven, not subjective

Classification Report

Class Precision Recall F1
Critical 0.829 0.936 0.879
Low Risk 0.999 0.998 0.999

Confusion Matrix (Validation Set)

Predicted Low Risk Predicted Critical
Actual Low Risk 10,670 21
Actual Critical 7 102

7 critical containers missed out of 10,800 — a miss rate of 0.065%.

Ensemble Configuration

Component Weight Algorithm
LightGBM 60% Gradient Boosted Trees (primary)
XGBoost 40% Gradient Boosted Trees (secondary)

Top 10 Feature Importances (LightGBM splits)

Rank Feature Description
1 Weight_Discrepancy_Ratio (Measured − Declared) / Declared weight — ~850 splits
2 Log_Dwell log1p(Dwell_Time_Hours)
3 Dwell_Time_Hours Raw container dwell time
4 Discrepancy_x_Dwell Interaction: weight anomaly × dwell
5 Anomaly_Score Isolation Forest normalised score
6 Exporter_ID_OE Ordinal-encoded exporter identity
7 Value_x_Discrepancy Interaction: value anomaly × weight discrepancy
8 Exporter_ID_Freq Frequency of exporter in training data
9 Country_Pair_Freq Frequency of origin→destination country pair
10 Destination_Port_OE Ordinal-encoded destination port

Key Findings & Insights

Risk Distribution (Real-Time Test Set — 8,481 containers)

Risk Level Count Percentage
Critical ~76 0.9%
Low Risk ~8,405 99.1%

Top risk score recorded: 90.0 on the test set.

What Makes a Container Critical?

Analysis of the top Critical containers consistently shows:

  1. Weight discrepancy is the single strongest signal — containers with measured weight differing >20% from declared weight account for the majority of Critical flags
  2. Dwell time > 72 hours combined with weight discrepancy raises risk ~2× vs. either signal alone
  3. Rare exporters (few historical records) correlate strongly with high risk — low Exporter_ID_Freq is the 6th most important feature
  4. High-value, low-weight shipments (Value_Per_Kg outliers) with an anomaly score > 0.7 are almost universally flagged Critical
  5. Specific HS categories have elevated base risk — electronics (84xx) and chemicals (28-29xx) show higher Critical rates

Anomaly Detection vs. ML Agreement

The Isolation Forest flags ~1% of shipments as anomalies independently of the supervised labels. In ~78% of cases where both the anomaly detector and the ML model agree a container is suspicious, it is labelled Critical — confirming the value of combining unsupervised and supervised signals.

ML Pipeline

Raw CSV Input (16 columns)
       │
       ▼
┌──────────────────────────────────┐
│  1. Preprocessing                │  Date/time parsing, numeric coercion,
│                                  │  HS_Category extraction, null imputation
└──────────────────────────────────┘
       │
       ▼
┌──────────────────────────────────┐
│  2. Feature Engineering          │  26 features: weight ratios, log transforms,
│                                  │  temporal flags, frequency encodings,
│                                  │  composite keys, interaction terms
└──────────────────────────────────┘
       │
       ▼
┌──────────────────────────────────┐
│  3. Anomaly Detection            │  Isolation Forest (n=300, contamination=0.01)
│                                  │  → Anomaly_Score [0,1], Is_Anomaly flag
└──────────────────────────────────┘
       │
       ▼
┌──────────────────────────────────┐
│  4. Ensemble Inference           │  LGB (60%) + XGB (40%) soft-vote
│                                  │  RobustScaler + OrdinalEncoder applied
└──────────────────────────────────┘
       │
       ▼
┌──────────────────────────────────┐
│  5. Risk Score Computation       │  70% × ML probability
│                                  │  30% × rule signal (weight/value/dwell/anomaly)
│                                  │  → Score [0, 100]
└──────────────────────────────────┘
       │
       ▼
┌──────────────────────────────────┐
│  6. Classification + Explanation │  Score > 61 → Critical
│                                  │  Human-readable risk reason generated
└──────────────────────────────────┘

Dataset

Split Rows Source
Training 32,400 Historical Data.csv
Validation 10,800 Historical Data.csv
Holdout 10,800 Historical Data.csv
Test (Real-Time) 8,481 Real-Time Data.csv

Class balancing: 5:1 undersampling → BorderlineSMOTE → ~1:1 balanced training set

Output Files

1. Prediction CSV (predictions.csv)

Each row contains the required output fields per the problem specification:

Column Description
Container_ID Shipment identifier
Risk_Score Composite score 0–100
Risk_Level Critical / Low Risk
Explanation_Summary 1–2 line human-readable reason

2. Summary Dashboard (frontend/)

A full Next.js web application provides:

Metric Card Value shown
Total Containers Processed Count from uploaded dataset
Critical Containers Count + percentage
Average Risk Score Mean across all containers
High-Risk Alert Containers with score > 85

Visualisations: risk score histogram, Critical vs. Low Risk pie chart, top 10 highest-risk containers bar chart, dwell time vs. risk score scatter plot, full sortable/filterable/exportable table.

Hackathon Requirements Checklist

Must Have

Requirement Status Implementation
AI/ML risk assessment model (Risk Score + Risk Level) ✅ LGB + XGB ensemble, Risk Score 0–100, Critical / Low Risk
Anomaly detection for unusual trade patterns ✅ Isolation Forest — flags weight, value, dwell outliers
Explainability for each prediction ✅ Explanation_Summary field — auto-generated per container
Prediction output in CSV format ✅ predictions.csv with all 4 required columns
Source code with execution instructions ✅ Notebook + API + README

Nice to Have

Feature Status Implementation
Web-based dashboard for risk visualisation ✅ Next.js 14 + Recharts + TanStack Table — full dashboard
REST API ✅ Flask + gunicorn, /api/predict, /api/predict/batch, /api/health
Docker deployment-ready ✅ Dockerfile.hf for HF Spaces, docker-compose.yml for local
Modular structure ✅ Separate pipeline, API, and frontend layers
Advanced anomaly detection ✅ Isolation Forest + rule-based signals + interaction features
Ensemble approach ✅ LightGBM (60%) + XGBoost (40%) soft-vote
Customs workflow integration design ✅ REST API with CORS whitelist, batch endpoint, health check — drop-in for any customs portal

REST API

Base URL

https://hp25-container-risk-engine.hf.space

Endpoints

Method Endpoint Description
GET /api/health Service health + model metadata
POST /api/predict Single container risk prediction
POST /api/predict/batch Batch prediction (up to 500 containers)

Health Check

curl https://hp25-container-risk-engine.hf.space/api/health
{
  "status": "ok",
  "model": "SmartContainer Risk Engine",
  "features": 26,
  "critical_threshold": 61.0,
  "prob_threshold": 0.672
}

Single Prediction

curl -X POST https://hp25-container-risk-engine.hf.space/api/predict \
  -H "Content-Type: application/json" \
  -d '{
    "Container_ID": "C001",
    "Declaration_Date": "2024-03-01",
    "Declaration_Time": "02:30",
    "HS_Code": "8471300000",
    "Declared_Value": 150000,
    "Declared_Weight": 500,
    "Measured_Weight": 850,
    "Dwell_Time_Hours": 120,
    "Trade_Regime": "Import",
    "Origin_Country": "CN",
    "Destination_Country": "IN",
    "Destination_Port": "INMUN",
    "Shipping_Line": "COSCO",
    "Importer_ID": "IMP_NEW_001",
    "Exporter_ID": "EXP_NEW_001"
  }'
{
  "Container_ID": "C001",
  "Risk_Score": 87.42,
  "Risk_Level": "Critical",
  "Model_Probability": 0.9123,
  "Explanation_Summary": "Risk factors: high weight discrepancy of 350.0 kg (ratio=0.70); excessive dwell time (120 hours); rare importer with few historical records.",
  "Dwell_Time_Hours": 120.0,
  "Weight_Discrepancy_Ratio": 0.7,
  "Anomaly_Score": 0.82,
  "Risk_Flag_Count": 3,
  "status": "success"
}

Required Input Fields

Field Type Example
Container_ID string "C001"
Declaration_Date string (YYYY-MM-DD) "2024-03-01"
Declaration_Time string (HH:MM) "14:30"
Trade_Regime string "Import" / "Export" / "Transit"
Origin_Country string "CN"
Destination_Country string "IN"
Destination_Port string "INMUN"
HS_Code string "8471300000"
Importer_ID string "IMP_001"
Exporter_ID string "EXP_001"
Declared_Value number 150000
Declared_Weight number 500
Measured_Weight number 850
Shipping_Line string "COSCO"
Dwell_Time_Hours number 120

Error Responses

Code Cause
400 Invalid or empty JSON body
413 Request body > 5 MB
422 Missing required field or non-numeric value
500 Internal prediction error

Project Structure

Nirma_Hackathon/
├── SmartContainer_Risk_Engine.ipynb   # Full ML pipeline notebook
├── api.py                             # Flask REST API
├── auth.py                            # API key middleware (optional)
├── requirements.txt                   # Python dependencies
├── Dockerfile.hf                      # Docker image for HF Spaces
├── Historical Data.csv                # Training dataset (54,000 rows)
├── Real-Time Data.csv                 # Test dataset (8,481 rows)
├── predictions.csv                    # Model output on test set
├── risk_engine_pipeline.joblib        # Saved model bundle (generated by notebook)
├── hf_space/
│   └── README.md                      # HF Spaces card metadata
└── frontend/                          # Next.js 14 web application
    ├── src/
    │   ├── app/
    │   │   ├── dashboard/page.tsx     # Live dashboard with charts & table
    │   │   ├── predict/page.tsx       # Single container analysis
    │   │   └── batch/page.tsx         # CSV batch upload & results
    │   ├── components/
    │   │   ├── charts/                # Risk gauge, histogram, pie, scatter
    │   │   ├── tables/                # TanStack Table with sort/filter/export
    │   │   ├── forms/                 # 15-field container input form
    │   │   └── cards/                 # Animated metric cards
    │   ├── services/api.ts            # Axios client to Flask API
    │   ├── hooks/usePrediction.ts     # React Query mutations
    │   └── types/container.ts         # TypeScript type definitions
    ├── Dockerfile                     # 2-stage Next.js Docker build
    └── docker-compose.yml             # Frontend + API compose setup

Local Development

1. Generate the Model

Run all cells in the notebook top-to-bottom:

# Activate virtual environment
& d:\JAVA\CODE\.venv\Scripts\Activate.ps1

# Install dependencies
pip install -r requirements.txt

# Open notebook
jupyter notebook SmartContainer_Risk_Engine.ipynb

This generates risk_engine_pipeline.joblib.

2. Start the API

python api.py
# Listening on http://localhost:5000

3. Start the Frontend

cd frontend
npm install
npm run dev
# http://localhost:3000

4. Docker (Full Stack)

cd frontend
docker compose up --build
# Frontend: http://localhost:3000
# API:      http://localhost:5000

Frontend Features

Page Features
Dashboard 4 metric cards, alert banner (score > 85), risk histogram, pie chart, top risky containers bar, dwell vs risk scatter, full sortable/filterable table with CSV export
Predict 15-field form with validation, animated SVG risk gauge (0–100), risk badge, explanation text
Batch CSV drag-and-drop upload, PapaParse parsing, summary stats, charts, full results table with detail drawer

Tech stack: Next.js 14 · Tailwind CSS · Recharts · TanStack Table v8 · React Query v5 · Framer Motion · react-hook-form · Axios

Deployment

The API is deployed on Hugging Face Spaces (Docker runtime):

To redeploy after changes:

cd D:\JAVA\CODE\hf_space_deploy
# copy updated files
Copy-Item ..\Projects\Nirma_Hackathon\api.py .
git add .; git commit -m "update"
git push origin main

Evaluation Criteria Alignment

Criterion How we address it
Quality & effectiveness PR-AUC 0.9404, F2 0.912, only 7 misses on 10,800 validation samples
Practicality & real-world applicability Live REST API, Docker image, batch endpoint up to 500 containers, production gunicorn server
Feature engineering clarity 26 fully documented features with rationale; top feature (Weight_Discrepancy_Ratio) is intuitive and interpretable
Explainability Every prediction includes a human-readable Explanation_Summary describing the dominant risk factors
System robustness & simplicity Single .joblib model bundle, stateless API, no database required, runs from a cold start in seconds
Presentation quality Full-stack web dashboard, this README, inline notebook documentation

Team

Built at Nirma University Hackathon

Component Description
SmartContainer_Risk_Engine.ipynb Full ML pipeline — preprocessing, feature engineering, training, evaluation
api.py Production Flask REST API (gunicorn, CORS, validation, rate limiting)
frontend/ Next.js 14 web dashboard (predict, batch, live dashboard)
Dockerfile.hf HF Spaces deployment — live at https://hp25-container-risk-engine.hf.space

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages