A modular and extensible platform for network traffic collection, enrichment, and analysis, built on top of Arkime, Flask, and RabbitMQ.
This organization hosts all repositories related to the project’s pipeline, enrichment service, and observability dashboard.

The Network Flow Observability Platform is designed to capture, enrich, and visualize network traffic data for security and performance monitoring purposes.
It integrates online packet collection, offline packet collection, protocol classification, risk scoring, and session enrichment into one cohesive ecosystem.

The project is composed of three main modules, each developed as a standalone repository:

        ┌──────────────────────────────┐
        │      Network Traffic         │
        │     (Captured by Mitmproxy)  │
        └───────────────┬──────────────┘
                        │
                        ▼
        ┌──────────────────────────────┐
        │  Network Data Pipeline       │
        │  (Producer / Consumer / API) │
        ├──────────────────────────────┤
        │  - Converts .mitm → .pcap     │
        │  - Uses nDPI for classification│
        │  - Publishes via RabbitMQ      │
        │  - Flask API for enrichment    │
        └───────────────┬──────────────┘
                        │
                        ▼
        ┌──────────────────────────────┐
        │  Arkime Wise Integration     │
        │  (source.myService.js)       │
        │  - Queries Flask API         │
        │  - Adds enrichment fields    │
        │  - Exposes risk/category info│
        └───────────────┬──────────────┘
                        │
                        ▼
        ┌──────────────────────────────┐
        │ Observability Dashboard      │
        │ - Displays flow analytics    │
        │ - Advanced search / filters  │
        │ - Visual risk metrics        │
        └──────────────────────────────┘

✅ Traffic Capture & Collection

• Intercepts and logs network flows using Mitmproxy
• Converts captured data into .pcap format

✅ Data Enrichment

• Uses nDPI for protocol detection and application classification
• Flask API provides risk, category, and byte/packet metrics for each session

✅ Arkime Integration

• Custom Wise plugin (source.myService.js) adds enriched fields to Arkime sessions
• Enables real-time enrichment and display within Arkime’s Session Viewer

✅ Visualization & Analytics

• The dashboard visualizes collected data, allowing users to perform:
  • Advanced searches
  • Session-based filtering
  • Risk-level analysis
  • Protocol and application statistics

• Captures and processes traffic logs
• Provides Flask API for enrichment endpoints (/update-cache, /enrich)
• Implements Producer/Consumer pattern using RabbitMQ
• Converts .mitm to .pcap for Arkime ingestion

Key Technologies: Python, Flask, Mitmproxy, RabbitMQ, nDPI

• Custom Wise plugin for Arkime (Node.js)
• Fetches enrichment data from Flask API and injects it into Arkime sessions
• Adds multiple metadata fields: risk level, data ratio, byte counts, duration, etc.
• Easily extensible for new enrichment attributes

Key Technologies: Node.js, Arkime Wise API, Axios

• Web interface for users to visualize and query enriched session data
• Integrates directly with Arkime’s REST API
• Provides CSV export, search filtering, and live statistical insights

Key Technologies: Flask, HTML, JavaScript (DataTables.js), Nginx

1. Pipeline VM

   • Runs Flask API, Producer, and Consumer services.
   • Connects to RabbitMQ and captures traffic via Mitmproxy.

2. Arkime VM

   • Receives .pcap data through pcap-over-ip.
   • Loads the Wise plugin to fetch enrichment from the Flask API.

3. Dashboard VM

   • Fetches enriched sessions from Arkime REST API.
   • Presents data visualization and analytics interface.

This project is developed for educational and research purposes in the domain of network observability and cybersecurity analytics.
It must only be used for traffic capture and analysis in authorized environments with proper permissions.

For collaboration or inquiries:

Organization: ArkimeNetLab
Website: eniacgroup.ir
Email: info@eniacgroup.ir

If you find this project useful:

• Give it a ⭐ on GitHub
• Contribute by improving documentation, dashboards, or enrichment models
• Submit feature requests via issues in the relevant repositories