RailMind AI

Intelligent Railway Safety & Security System

Turning existing CCTV infrastructure into proactive behavioural monitoring — no new cameras, no facial recognition.

This is a hackathon build for FAR AWAY 2026 by Team Accelerate. Everything described below as "implemented" has been verified by running it — backend boots, 57 backend tests pass, frontend builds, and the CV → LSTM → agent pipeline runs end-to-end on uploaded video.

Some capabilities described in early planning docs (edge/Jetson deployment, multi-camera re-identification, PA system integration, a mobile app) are not yet built — see What's Not Built Yet below. We'd rather you find out from this README than from the code.

• Overview
• What This Actually Does
• What's Not Built Yet
• Screenshots
• System Architecture
• AI Pipeline
• LSTM Behaviour Classifier
• Agentic Reasoning
• Tech Stack
• Quick Start
• Configuration
• Training the LSTM
• API Reference
• Testing
• Privacy & Ethics
• Project Structure
• Roadmap
• Team

RailMind AI processes existing CCTV video through a computer-vision pipeline (person detection → tracking → pose estimation → behavioural feature extraction) and feeds the result into a bidirectional LSTM that classifies 30-second behavioural windows into one of four categories: Normal, Suicide Risk, Pickpocketing, or Security Threat. A three-agent LangGraph pipeline takes that classification, computes a final risk score, and dispatches alerts to staff through a live dashboard — with no facial recognition and no biometric storage anywhere in the system.

Verified by running it, not just reading the code:

• Person detection & tracking — YOLOv8 detection + Ultralytics' BYTETracker for persistent IDs across frames.
• Pose estimation — YOLOv8-Pose extracts 17 COCO keypoints per tracked person.
• Behavioural feature extraction — 7 features per person per 30s window (edge proximity, loitering time, pacing count, movement speed, direction changes, following distance, crowd interactions).
• Trained LSTM classifier — a real, saved 2-layer bidirectional LSTM checkpoint (backend/lstm/saved_models/behavior_classifier.pt) with a fitted feature scaler, loaded and run at inference time.
• LangGraph multi-agent pipeline — a real StateGraph with Perception → Reasoning → Intervention nodes, compiled once and invoked per detection.
• Optional LLM-assisted reasoning — if an OpenAI or Anthropic API key is configured, the Reasoning Agent asks an LLM for a bounded risk-score adjustment (±10) and a reasoning summary; without a key, it falls back cleanly to the rule-based RiskScorer.
• Live dashboard — React 19 + TanStack Router/Query dashboard pulling real stats from the FastAPI backend (only camera thumbnail images are placeholders; incident counts, trends, and risk distributions are real).
• Video upload & playback pipeline — upload an .mp4, it runs through the full CV → LSTM → agent pipeline; not a canned demo loop.
• WebSocket alert delivery — a channel-based pub/sub ConnectionManager broadcasts alerts to all connected dashboard clients in real time.
• Escalation timers — unacknowledged alerts escalate after a configurable timeout (default 60s).
• Email alerts (SMTP) — implemented via smtplib, configurable via .env.
• SMS escalation (Twilio) — implemented in the escalation service, configurable via .env.
• 57 passing backend tests covering agents, CV pipeline, LSTM inference, risk scoring, alerts, incidents, escalation, and heatmaps.

Being direct about this so nobody — including us — overclaims it later:

2×2 grid of simultaneous CCTV feeds with AI-annotated bounding boxes, track IDs, and live risk labels overlaid directly on the video.

Aggregated analytics across all platforms — incident counts, 7-day trend, risk distribution, a platform heatmap, peak-risk-hour histogram, and a per-camera summary table.

Full alert triage view with filterable tabs by risk level, a sortable alert table, and a detail panel for acknowledging or resolving an incident.

┌─────────────── PROCESSING ───────────────┐
│ Video Source → OpenCV → YOLOv8 Detection │
│   → ByteTrack → YOLOv8-Pose               │
│   → Feature Extraction (7 features)       │
│   → BiLSTM Classifier (30s window)        │
└────────────────────┬───────────────────────┘
                      │ classification + confidence
┌─────────────────────▼──────────────────────┐
│ LangGraph Agent Pipeline                    │
│ Perception → Reasoning → Intervention       │
└────────────────────┬───────────────────────┘
                      │ risk score + action
┌─────────────────────▼──────────────────────┐
│ FastAPI + SQLite/PostgreSQL                 │
│ WebSocket broadcast → React Dashboard       │
│ Email (SMTP) / SMS (Twilio) escalation      │
└──────────────────────────────────────────────┘

Each tracked person generates a 7-dimensional feature vector per 30-second window:

feature_vector = [
    edge_proximity_seconds,   # Cumulative time within 0.5m of platform edge
    loitering_time,           # Stationary duration in a single spatial zone
    pacing_count,             # Back-and-forth movement cycles detected
    movement_speed,           # Average velocity (m/s)
    direction_changes,        # Heading reversals per minute
    following_distance,       # Sustained proximity to one other person
    crowd_interactions,       # Count of unique close-contact individuals
]

Input [30, 7] → BiLSTM (128 units) → Dropout (0.3)
              → BiLSTM (64 units)  → Dropout (0.3)
              → Dense (32, ReLU) → Dense (4, Softmax)

• Output classes: Normal, Suicide Risk, Pickpocketing, Security Threat
• Training data: synthetic sequences generated from rule-based behavioural definitions (see Training the LSTM)
• Saved artifacts: backend/lstm/saved_models/behavior_classifier.pt + a fitted StandardScaler pickle

Reported accuracy figures are measured against the synthetic held-out test set used for training. They are not yet validated against real, labelled incident footage — treat them as a development benchmark, not a production guarantee.

Three LangGraph nodes, compiled once into a StateGraph at startup:

If no LLM API key is set, the Reasoning Agent runs purely on rule-based scoring — the system does not require an LLM to function.

SQLite for development (used in this build); PostgreSQL + Redis are the intended production path but are not what this repo currently runs on by default.

• Python 3.11+
• Node.js 18+
• (Optional) an OpenAI or Anthropic API key for LLM-assisted reasoning

cd backend
pip install -r requirements.txt
cp .env.example .env   # edit values as needed
python run.py --mode server --port 8000

cd frontend
npm install
cp .env.example .env
npm run dev

The dashboard expects the backend at the URL configured in VITE_API_BASE_URL (defaults to http://localhost:8000/api).

DEBUG=True
DATABASE_URL=sqlite:///./railmind.db
SECRET_KEY=your-secret-key-here
RAILMIND_API_KEY=change-this-admin-api-key
LOG_LEVEL=INFO
CORS_ORIGINS=http://localhost:5173,http://localhost:3000

# LSTM behaviour label thresholds
BEHAVIOR_HIGH_SCORE_THRESHOLD=0.65
BEHAVIOR_ERRATIC_SCORE_THRESHOLD=0.4
BEHAVIOR_FOLLOWING_DISTANCE_METERS=1.2

# Email (SMTP) — optional
SMTP_HOST=smtp.example.com
SMTP_PORT=587
SMTP_USER=alert@example.com
SMTP_PASSWORD=your-smtp-password
ALERT_EMAIL_RECIPIENTS=security@example.com,ops@example.com

# SMS (Twilio) — optional
TWILIO_ACCOUNT_SID=your_twilio_account_sid
TWILIO_AUTH_TOKEN=your_twilio_auth_token
TWILIO_FROM_NUMBER=+1234567890
TWILIO_TO_NUMBERS=+19876543210,+10987654321

VITE_API_BASE_URL=http://localhost:8000/api
VITE_RAILMIND_API_KEY=change-this-admin-api-key
VITE_WS_URL=ws://localhost:8000

cd backend
python run.py --mode train-cli --model-type all --epochs 30 --batch-size 32

Or trigger training via the API while the server is running:

curl -X POST http://localhost:8000/api/training/trigger \
  -H "Content-Type: application/json" \
  -d '{"model_type": "all", "epochs": 30, "batch_size": 32}'

Training data is generated synthetically from rule-based behavioural definitions — see backend/app/lstm/train.py. Swapping in real labelled incident data is the top priority before any production claim about accuracy.

All routes are prefixed with /api unless noted. Selected endpoints:

Full route definitions live in backend/app/api/routes/.

cd backend
pytest                       # full suite — 57 tests, all passing as of this build
pytest tests/test_agents.py
pytest tests/test_lstm.py
pytest tests/test_risk_scoring.py
pytest tests/test_cv.py

Frontend:

cd frontend
npm run e2e   # Playwright E2E tests

• No facial recognition — no face-detection model is used anywhere in the pipeline.
• No biometric storage — track IDs are numeric, session-scoped integers, not linked to identity.
• Behaviour-only analysis — the feature vector contains only movement metrics.
• Human-in-the-loop — every alert requires staff acknowledgement; the system never triggers a physical intervention on its own.
• Operator accountability — alerts, acknowledgements, and resolutions are logged with timestamps.

This is an architectural property of the current code, not just a policy statement — there is no facial recognition or identity-linking code anywhere in this repository.

RailMind-AI/
├── assets/                  # Dashboard screenshots used in this README
├── backend/
│   ├── app/
│   │   ├── agents/        # LangGraph perception/reasoning/intervention nodes
│   │   ├── cv/             # Video processing, pose estimation
│   │   ├── features/       # Behavioural feature detectors
│   │   ├── lstm/            # Model definition, training, inference
│   │   ├── api/routes/      # FastAPI endpoints
│   │   ├── services/        # Risk scoring, alerts, escalation, notifications
│   │   ├── analytics/       # Dashboard metrics, heatmaps
│   │   ├── models/          # SQLAlchemy models
│   │   └── core/             # Config, database, WebSocket manager
│   ├── lstm/saved_models/    # Trained checkpoint + scaler
│   ├── tests/                  # Pytest suite
│   └── run.py
├── frontend/
│   ├── src/
│   │   ├── routes/            # Dashboard, alerts, live feed pages
│   │   ├── components/         # UI components
│   │   ├── lib/api/             # API client functions
│   │   └── hooks/                # useWebSocket, etc.
│   └── tests/e2e/                # Playwright tests
└── README.md

• Video upload + CV pipeline (YOLOv8 + ByteTrack + Pose)
• Bidirectional LSTM behaviour classifier with saved weights
• LangGraph Perception + Reasoning + Intervention agents
• SQLite storage with full schema
• React dashboard with live alerts and analytics
• WebSocket real-time alert broadcast
• Email (SMTP) and SMS (Twilio) escalation channels
• Configurable escalation timers
• Optional LLM-assisted reasoning (OpenAI/Anthropic)
• 57 passing backend tests

• Live RTSP camera validation (currently untested beyond file upload)
• PostgreSQL + Redis production migration
• LSTM retrained on real, labelled incident data
• Multi-camera person re-identification
• Edge deployment (Jetson Orin Nano / TensorRT)
• Automated PA system integration
• Mobile app for staff (React Native)
• True multi-tenant multi-station SaaS isolation
• Predictive analytics / crowd flow optimisation

Team Accelerate — FAR AWAY 2026 Hackathon