Explainable AI Intrusion Detection System (XAI-IDS)

This project implements an advanced Intrusion Detection System (IDS) that leverages a powerful hybrid CNN-LSTM deep learning model for accurate anomaly detection in network traffic. What sets XAI-IDS apart is its integration of SHAP (SHapley Additive exPlanations), providing clear and concise explanations for every detected threat.

Key Enhancement: live network monitoring capabilities, allowing the system to capture, analyze, and explain network traffic anomalies in real-time, making it suitable for proactive threat detection in dynamic network environments. It also addresses the challenges of class imbalance in network traffic datasets and the opacity of deep learning models.

Developed by: Sushminthiran S (211423104667) Guide: Mr. Sasikumar A.N M.E. Institution: Panimalar Engineering College

Features

• Hybrid CNN-LSTM Model: Utilizes a CNN-LSTM architecture for robust and accurate detection of network intrusions based on both spatial and temporal features.
• Class Imbalance Handling: Employs advanced techniques like SMOTE and Random Undersampling to improve the detection rates of rare attack types.
• Explainable AI (XAI) with SHAP: Integrates SHAP to provide clear, feature-based explanations for why a specific network flow was classified as an attack, enhancing transparency and trust.
• Streamlit Web Interface: Provides an intuitive and interactive dashboard for both offline simulation and live network monitoring.
• Live Network Monitoring (NEW!):
  • Real-time Packet Capture: Sniffs live network traffic using Scapy on a selected network interface.
  • On-the-fly Feature Extraction: Processes raw packets into statistical flow features (mimicking CIC-IDS format).
  • Real-time Inference: Applies the trained deep learning model to live traffic for immediate threat detection.
  • Dynamic Alerting: Displays detected intrusions and suspicious activities in a continuously updating log.
  • Live SHAP Explanations: Generates and displays SHAP explanations for real-time detected attacks, providing instant insights into the attack's root causes.
  • Configurable Packet Filters: Apply Berkeley Packet Filter (BPF) rules to selectively monitor specific traffic.
• Persistent Alert Logging: Automatically saves all detected alerts (including timestamp, attack type, confidence, and flow info) to a CSV file for post-analysis and auditing.

Project Structure

Explainable_AI_IDS_Project/
│
├── src/                      # Source code, data, and model
│   ├── app.py                # Main Streamlit application
│   ├── config.py             # Configuration settings
│   ├── utils.py              # Helper functions
│   ├── requirements.txt      # Python dependencies
│   ├── data/                 # CIC-IDS2017 dataset CSVs
│   └── saved_model/          # Trained Keras model (.h5)
│
├── docs/                     # Documentation and papers
│   ├── Project_Report.docx   # Full project report
│   ├── Final_Viva_PPT.pptx   # Presentation slides
│   ├── Journal_Paper.docx    # IEEE-style paper
│   ├── One_Page_Abstract.docx # Single-page abstract
│
├── demo_video/               # Demo video folder
│   ├── Project_Demo.mp4      # Recorded project demo
│
└── README.md                 # This file

Setup Instructions

1. Clone the Repository:

   git clone <your-repository-link>
   cd Explainable_AI_IDS_Project

2. Create a Python Virtual Environment:

   python -m venv .venv

3. Activate the Virtual Environment:

   • Windows (Command Prompt/PowerShell): .\.venv\Scripts\activate
   • macOS/Linux (Bash/Zsh): source .venv/bin/activate

4. Install Dependencies:

   pip install -r src/requirements.txt

   • Scapy Permissions (Linux/macOS): For live monitoring, Scapy requires elevated privileges to sniff network traffic. You might need to give your Python executable cap_net_raw and cap_net_admin capabilities.

     sudo setcap cap_net_raw,cap_net_admin=eip $(eval readlink -f $(which python))

     (Note: This command needs to be run once per Python installation if using the system Python, or for your virtual environment's Python executable.)

5. Install Npcap (for Live Monitoring on Windows):

   • Download and install Npcap from npcap.com.
   • Important during Npcap installation: Ensure the option "Install Npcap in WinPcap API-compatible Mode" is CHECKED for broader compatibility with Scapy. Administrator privileges are required for Npcap installation.

How to Run

1. Activate the Virtual Environment (if not already active).

2. Navigate to the src directory:

   cd src

3. Run the Streamlit Application:

   streamlit run app.py

   • The application will automatically open in your web browser.
   • Use the sidebar in the Streamlit UI to switch between "Offline Simulation" and "Live Monitoring".

4. For Live Monitoring (Crucial Steps):

   • You MUST launch your terminal or command prompt with Administrator privileges (Windows) or use sudo (macOS/Linux) when running the streamlit run app.py command to allow Scapy to access network interfaces.
     • Windows (Admin PowerShell/CMD): Right-click on PowerShell/CMD icon -> "Run as administrator". Then cd src and streamlit run app.py.
     • macOS/Linux: cd src then sudo streamlit run app.py (you will be prompted for your password).
   • In the Streamlit sidebar, select your active network interface (e.g., Wi-F1, eth0, en0).
   • (Optional) Configure a BPF packet filter to monitor specific traffic.
   • Click the "🟢 Start Monitoring" button.
   • Observe the "Live Alert Log" for real-time detected threats and the "Live SHAP Explanations" section for insights into why an attack was flagged.
   • Click the "🔴 Stop Monitoring" button to halt the capture process.

Notes

• The live monitoring feature uses Scapy and critically requires elevated permissions (Administrator/root) for proper network interface access.
• The SHAP explanation generation, especially the initial setup and for the first few explanations, can be computationally intensive and might introduce a slight delay.
• Ensure the file paths specified in config.py correctly point to your data and saved_model directories relative to where app.py is executed.
• The app.py directly embeds the scapy_worker logic using Python's multiprocessing module for efficient background packet sniffing.