🛡️ Network Intrusion Detection System (IDS)

A production-grade hybrid Intrusion Detection System combining machine learning and stateful threat analytics for real-time network security monitoring.

---

✨ Features

• 🔴 Live Network Monitoring - Real-time packet capture and analysis with pyshark and tshark
• 📊 Offline File Analysis - Process .pcap, .pcapng, and .csv network traffic files
• 🤖 ML-Based Classification - RandomForest model achieving 94% accuracy on CICIDS2017 dataset
• 🚨 Advanced Threat Detection - Stateful, threshold-based detection for:
  • Port scans (SYN/FIN/NULL/XMAS patterns)
  • SYN/UDP floods
  • Brute-force attacks on service ports
  • Data exfiltration anomalies
  • DNS tunneling and tunnel frequency anomalies
  • HTTP attack signatures (SQLi/XSS/directory traversal)
  • Connection-rate and protocol anomalies
• 🌐 Interactive Web Dashboard - Real-time alerts and monitoring via Flask + Socket.IO
• 🤖 AI-Powered Explanations - Google Gemini integration for plain-language threat analysis
• 🔒 Security Hardened - Secure defaults for session cookies, CORS, and file uploads

---

🚀 Quick Start

Prerequisites

• Python 3.11+ (tested with 3.13)
• Wireshark and tshark command-line tool
• Windows users: Run terminal with administrator privileges for packet capture

Installation & Running

# 1. Clone the repository
git clone https://github.com/adityashm/network-ids.git
cd network-ids

# 2. Create and activate virtual environment
python -m venv .venv

# On Windows:
.\.venv\Scripts\Activate.ps1

# On Linux/Mac:
source .venv/bin/activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Configure environment
cp .env.example .env
# Edit .env with your settings (SECRET_KEY, NETWORK_INTERFACE, GEMINI_API_KEY, etc.)

# 5. Run the application
python "ids project/src/app.py"

# 6. Open browser and navigate to:
# http://127.0.0.1:5000

Health Check

curl http://127.0.0.1:5000/health

Architecture Overview

Web and Orchestration Layer

• File: ids project/src/app.py
• Responsibilities:
  • Flask + Socket.IO server
  • Health endpoint (/health)
  • File analysis endpoint (/analyze)
  • Live capture start/stop events
  • Alert standardization/sanitization
  • ML model loading and inference orchestration
  • Optional Gemini explanation endpoint over Socket.IO

Advanced Threat Engine

• File: ids project/src/enhanced_detection.py
• Implements stateful, threshold-based detection for:
  • Port scans (SYN/FIN/NULL/XMAS patterns)
  • SYN/UDP floods
  • Brute-force behavior on common service ports
  • Data exfiltration anomalies
  • DNS tunneling frequency anomalies
  • HTTP attack signatures (SQLi/XSS/traversal indicators)
  • Connection-rate and protocol anomalies

Feature Engineering

• File: ids project/src/extract_flows.py
• Converts packets into flow-level features aligned with CIC-IDS2017-style columns (FEATURE_COLUMNS).

Training Pipeline

• File: ids project/src/src_train.py
• Preprocessing:
  • Column cleanup
  • NaN/infinity normalization
  • Label normalization (BENIGN -> 0, non-benign -> 1)
• Model:
  • RandomForestClassifier(class_weight='balanced')
• Output model:
  • ids project/models/ids_model.joblib

Dataset Utilities

• File: ids project/src/combine_csv.py
• Combines multiple CIC CSV files into:
  • ids project/src/data/processed/MachineLearningCVE.csv

Frontend

• File: ids project/src/templates/index.html
• Provides:
  • Live monitoring controls
  • Status and alert feed
  • Offline upload analysis panel
  • Modal for AI explanations

Project Structure

IDS/
  README.md
  requirements.txt
  app.spec
  .env.example
  ids project/
    models/
      ids_model.joblib
    data/
      uploads/
    src/
      app.py                 # Canonical server entrypoint
      appc.py                # Legacy compatibility launcher
      enhanced_detection.py
      extract_flows.py
      src_train.py
      detect_live.py
      combine_csv.py
      templates/
        index.html
      data/
        raw/MachineLearningCSV/
        processed/MachineLearningCVE.csv

Requirements

• Python 3.11+ (tested in this workspace with 3.13)
• Wireshark tshark installed and accessible
• Windows users should run terminal/editor with privileges required for packet capture

Setup

1. Create and activate virtual environment:

python -m venv .venv
.\.venv\Scripts\Activate.ps1

2. Install dependencies:

pip install -r requirements.txt

3. Configure environment variables:

Copy-Item .env.example .env

Then edit .env and set at least:

• SECRET_KEY (strong random value)
• NETWORK_INTERFACE (for example Wi-Fi on Windows)
• TSHARK_PATH if auto-detection fails
• GEMINI_API_KEY if AI explanations are needed

Running the App

From workspace root:

python "ids project/src/app.py"

Default URL:

• http://127.0.0.1:5000

Health check:

• http://127.0.0.1:5000/health

Data and Model Workflow

1. Combine raw CSV files (optional if already combined):

python "ids project/src/combine_csv.py"

2. Train model:

python "ids project/src/src_train.py"

3. Run standalone live detector (optional):

python "ids project/src/detect_live.py"

4. Use web app upload flow (/analyze) for offline file analysis.

Security Notes

• Do not commit .env with real secrets.
• The app now rejects placeholder SECRET_KEY values and generates an in-memory fallback key with a warning.
• File uploads are extension allow-listed via ALLOWED_UPLOAD_EXTENSIONS.
• Session cookies are configured with secure defaults (HttpOnly, SameSite=Lax, Secure when not in debug).
• CORS origins are configurable via CORS_ALLOWED_ORIGINS and default to localhost only.

Packaging

• PyInstaller spec: app.spec
• Includes model and templates in the bundle.

Troubleshooting

• Model file not found:
  • Train first with python "ids project/src/src_train.py"
• tshark not found:
  • Install Wireshark and set TSHARK_PATH in .env
• Live capture fails:
  • Verify NETWORK_INTERFACE and terminal permissions
• No Gemini explanations:
  • Set GEMINI_API_KEY in .env