SIEM-X

An autonomous Security Orchestration, Automation, and Response (SOAR) engine that detects, correlates, and instantly neutralizes multi-stage network attacks.

SIEM-X ingests Suricata IDS alerts from Elasticsearch or static replay fixtures, maps them to an ordered MITRE ATT&CK kill chain, attributes the primary threat actor, forcefully terminates active reverse-shell sessions, and synthesizes grounded AI incident narratives into an interactive SOC report.

⚠️ Lab use only. SIEM-X executes real network containment actions — modifying firewall rules, forcefully destroying active TCP sockets (ss -K), flushing connection states (conntrack), and injecting live IDS block rules. Run only in an isolated lab environment. Always use --dry-run for initial evaluations and testing.

---

Interactive SOC Incident Dashboard & Workflow Showcase

SIEM-X transforms high-volume security telemetry into actionable, automated containment and clear threat intelligence.

1. Telemetry Ingestion & Elastic Security Detection

Monitors network perimeter traffic through Suricata IDS, streaming EVE JSON events into Elasticsearch to capture reconnaissance scans, exploit payloads, and reverse-shell callbacks.

---

2. ATT&CK Kill-Chain Mapping & Correlation

Deduplicates raw alerts by signature and endpoint tuples, mapping every event to its verified MITRE ATT&CK technique and phase using a longest-match keyword engine.

---

3. Autonomous Incident Response & Active Containment

Executes multi-layered active socket teardown (ss -K state established, conntrack -D, fuser -k), injects active TCP reset (REJECT --reject-with tcp-reset) and DROP firewall rules, and dynamically updates Suricata IPS rules.

---

4. Grounded AI Threat Intelligence & Profiling

Queries LLMs (via Groq API) strictly constrained to the deterministic MITRE ATT&CK mapping calculated by SIEM-X, producing a hallucination-free executive summary, attacker profile, severity assessment, and actionable remediation steps.

---

Architecture

flowchart TB
    %% Cyber SOC Color Palette
    classDef redNode fill:#2a1115,stroke:#f43f5e,stroke-width:2px,color:#fda4af;
    classDef blueNode fill:#0c1c2e,stroke:#38bdf8,stroke-width:2px,color:#bae6fd;
    classDef purpleNode fill:#1e1538,stroke:#a855f7,stroke-width:2px,color:#e9d5ff;
    classDef greenNode fill:#062316,stroke:#22c55e,stroke-width:2px,color:#86efac;
    classDef amberNode fill:#281806,stroke:#f59e0b,stroke-width:2px,color:#fde68a;
    classDef cyanNode fill:#072733,stroke:#06b6d4,stroke-width:2px,color:#a5f3fc;
    classDef clusterBox fill:#0a0e17,stroke:#1e293b,stroke-width:1.5px,color:#94a3b8;

    subgraph External["  External Adversary  "]
        A["Threat Actor<br><code>Recon · Exploitation · C2</code>"]:::redNode
    end

    subgraph Telemetry["  Telemetry & Sensor Stack  "]
        S["Suricata IDS<br><code>Protocol Inspection · EVE JSON</code>"]:::blueNode
        FB["Filebeat Shipper<br><code>Security Event Pipeline</code>"]:::blueNode
        ES[("Elasticsearch Cluster<br><code>.alerts-security Index</code>")]:::blueNode
    end

    subgraph Core["  SIEM-X Core Engine  "]
        SX["Correlation Engine<br><code>Deduplication · Longest-Match ATT&CK</code>"]:::purpleNode
        PB["Playbook Executor<br><code>YAML Playbook Handler</code>"]:::purpleNode
    end

    subgraph Containment["  Active Containment & IR  "]
        SK["Kernel Socket Teardown<br><code>ss -K · conntrack -D · fuser</code>"]:::greenNode
        FW["Firewall Enforcement<br><code>iptables REJECT (TCP-RST) & DROP</code>"]:::greenNode
        IPS["Live IPS Rule Reload<br><code>Dynamic auto_block.rules</code>"]:::greenNode
    end

    subgraph Synthesis["  AI Intelligence & Reporting  "]
        G["Groq LLM Synthesis<br><code>Grounded MITRE Prompting</code>"]:::amberNode
        R[("Interactive SOC Dashboard<br><code>siem_x_report.html</code>")]:::cyanNode
    end

    A -->|"Inbound Attacks & Outbound Shells"| S
    S -->|"EVE JSON Alerts"| FB
    FB -->|"Index Security Telemetry"| ES
    ES -->|"Live Alert Polling"| SX
    SX -.->|"Offline Fixtures (--replay)"| SX
    SX -->|"Confirmed Attack Signals"| PB
    PB -->|"Force Destroy Sockets"| SK
    PB -->|"Inject Firewall Rules"| FW
    PB -->|"Hot-Reload IPS Rules"| IPS
    FW -->|"Immediate TCP RST & Drop"| A
    SX -->|"Deterministic MITRE Mapping"| G
    G -->|"Structured SOC Narrative"| R
    SX -->|"Render Real-Time Telemetry"| R

    class External,Telemetry,Core,Containment,Synthesis clusterBox;

Loading

---

What it does

SIEM-X executes an 8-stage deterministic detection and automated response pipeline:

1. Ingest: Fetches raw alerts from live Elasticsearch (ElasticsearchAlertSource) or local JSON fixtures (--replay, via JSONFixtureAlertSource).
2. Deduplicate: Groups alerts by (rule_name, src_ip, dst_ip) tuples while aggregating occurrence counts, timestamp bounds, and severity levels.
3. MITRE ATT&CK Mapping: Maps each alert signature to a technique ID, technique name, and tactic using a longest-match keyword classification algorithm to eliminate generic misclassifications.
4. Kill Chain Reconstruction: Groups correlated alerts into ordered ATT&CK phases (Reconnaissance → Initial Access → Execution → Lateral Movement → Discovery → Command and Control).
5. Attacker Attribution: Calculates a weighted confidence score prioritizing critical-severity signals over background noise volume, and executes two-pass reverse-shell heuristic detection.
6. Automated Incident Response: Triggers YAML-defined playbooks to flush stale rules, forcefully destroy active kernel sockets (ss -K), clear connection tracking tables (conntrack -D), inject active TCP RST (REJECT) and DROP firewall rules, and dynamically reload Suricata block rules.
7. AI-Grounded Synthesis: Prompts an LLM (via Groq API) strictly constrained to the deterministic MITRE mapping computed by the engine, generating human-readable attack narratives without hallucinated technique IDs.
8. Interactive Reporting: Generates a self-contained HTML SOC dashboard featuring kill-chain tracks, severity distribution bars, MITRE matrix cards, and complete IR audit trails.

---

Comparison: SIEM-X vs. Typical Detection Scripts

Capability	Typical Lab Script	SIEM-X SOAR Engine
Data Ingestion	Static log parsing only	Live Elasticsearch query API + offline JSON fixture replay (--replay)
Detection Stack	Single standalone tool	Suricata IDS + Filebeat + Elasticsearch cluster integration
MITRE ATT&CK Mapping	Hardcoded tags or absent	Longest-match keyword precedence engine across ATT&CK tactics
Attacker Attribution	Naive highest-frequency IP	Weighted scoring (critical severity + reverse-shell heuristics over raw noise)
Active Containment	None (read-only alert output)	Multi-layered teardown: ss -K socket destroy, conntrack purge, active TCP reset injection, and Suricata dynamic rule reload
Playbook Engine	Hardcoded shell commands	Human-readable, extensible YAML playbooks (playbooks/*.yaml)
AI Integration	Ungrounded generic prompts	Deterministically grounded Groq LLM synthesis with anti-hallucination constraints
Reproducibility	Requires live lab setup	100% verifiable offline via bundled fixture files and automated test suites
Reporting	Raw CLI terminal logs	Interactive, self-contained HTML SOC dashboard

---

Quickstart (Replay Mode — No Infrastructure Required)

Replay mode evaluates the full correlation, attribution, containment logic (dry-run), and report generation pipeline offline without requiring Elasticsearch or Suricata.

1. Clone & Set Up Virtual Environment

git clone <repository-url>
cd SiemX

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

2. Configure Environment

cp .env.example .env
cp config.yaml.example config.yaml
# Edit .env to add your GROQ_API_KEY if you want live AI analysis.
# If omitted or left as placeholder, SIEM-X degrades gracefully with structured fallback notes.

3. Run Replay Scenarios

# Reverse-shell threat scenario (Full IR dry-run + containment report)
python3 -m siemx --replay tests/fixtures/reverse-shell.json --dry-run

# High-volume background noise alongside critical reverse shell
python3 -m siemx --replay tests/fixtures/noisy-reverse-shell.json --dry-run

# Multi-attacker reconnaissance and exploit scenario
python3 -m siemx --replay tests/fixtures/mixed-attacker.json --dry-run

# Reconnaissance-only scan scenario (clean monitoring state)
python3 -m siemx --replay tests/fixtures/scan-only.json --dry-run

4. View Incident Report

Open the generated report in any browser:

firefox siem_x_report.html
# or:
xdg-open siem_x_report.html

---

Configuration & Environment Setup

This project requires a .env file for credentials and a config.yaml file for environment-specific settings. Neither file is committed to version control — both are explicitly excluded in .gitignore. Copy the provided example templates and configure your values before running against live infrastructure.

cp .env.example .env
cp config.yaml.example config.yaml

Environment Variables (.env)

Variable	Requirement	Purpose
ES_USER	Required (Live mode)	Elasticsearch username for authentication
ES_PASS	Required (Live mode)	Elasticsearch password for authentication
VICTIM_USER	Required (Live mode)	Target host username for lab identity and authorization context
VICTIM_PASS	Required (Live mode)	Target host password for lab identity
GROQ_API_KEY	Optional	Groq API key for AI kill-chain synthesis (system degrades gracefully if omitted)
GROQ_MODEL	Optional	Groq model identifier (e.g. openai/gpt-oss-120b, overrides config.yaml)

Configuration File (config.yaml)

Key	Default	Description
victim_ip	"192.168.x.x"	Target host IPv4 address to monitor and protect
monitor_iface	"enp0s3"	Network interface monitored by IDS and response tooling
report_file	"siem_x_report.html"	Output file path for generated HTML incident report
log_file	"siem_x_response.log"	Output file path for execution and audit logging
suricata_auto_rules	"/var/lib/suricata/rules/auto_block.rules"	Target file path for dynamic Suricata IPS block rule injection
es_url	"https://localhost:9200"	Elasticsearch endpoint URL
groq_model	"openai/gpt-oss-120b"	Default AI model identifier used for analysis
playbooks_dir	"playbooks"	Directory containing YAML incident response playbooks
alert_fetch_size	500	Maximum number of raw alerts to fetch per query window
default_since	"1h"	Default time lookback window for live alert queries
es_verify_ssl	false	Enable/disable TLS certificate verification for Elasticsearch

---

Live-Mode Setup

For executing SIEM-X against a live attack simulation on an isolated network:

1. Topology Placement: Deploy SIEM-X on the gateway or inline monitoring node with access to the monitored network interface (monitor_iface).
2. IDS Pipeline: Ensure Suricata outputs EVE JSON events and Filebeat forwards them to Elasticsearch under .alerts-security.alerts-default.
3. Configure Secrets & Targets: Populate .env with live Elasticsearch credentials and config.yaml with your target host IP.
4. Preview with Dry-Run:

   sudo ./venv/bin/python -m siemx --since 30m --dry-run

5. Execute Active Containment:

   sudo ./venv/bin/python -m siemx --since 30m

CLI Options

• --replay PATH: Load and replay alerts from a local JSON fixture file.
• --since DURATION: Lookback time window for live Elasticsearch queries (e.g., 30m, 2h, 1d; overrides config.yaml).
• --dry-run: Evaluate detection, attribution, and playbook logic without executing subprocess commands (iptables, tcpkill, conntrack, sysctl, Suricata reload).
• --json-logs: Output structured JSON log lines for SIEM/collector ingestion.
• --log-level LEVEL: Set logging verbosity (DEBUG, INFO, WARNING, ERROR; default: INFO).

---

Project Structure

SiemX/
├── .env.example                     # Environment variables template
├── .gitignore                       # Excludes .env, config.yaml, caches, logs, and venvs
├── config.yaml.example              # Configuration settings template
├── LICENSE                          # MIT License file
├── playbooks/
│   └── reverse_shell.yaml           # YAML incident response playbook
├── pytest.ini                       # Pytest configuration and pathing
├── README.md                        # Project documentation
├── requirements.txt                 # Python dependencies
├── screenshots/                     # Dashboard and execution screenshots
├── siemX.py                         # Backward-compatible CLI entrypoint wrapper
├── siemx/
│   ├── __init__.py                  # Package exports
│   ├── __main__.py                  # python -m siemx entrypoint
│   ├── ai.py                        # Groq LLM integration & prompt grounding
│   ├── attribution.py               # Attacker attribution & reverse shell detection
│   ├── config.py                    # Config validation, secrets loading & fail-fast checks
│   ├── enrich.py                    # Deduplication, ATT&CK mapping & kill-chain grouping
│   ├── ingest.py                    # Alert ingestion dispatcher
│   ├── logging_setup.py             # Console and file logger formatting
│   ├── main.py                      # CLI parser & core pipeline orchestration
│   ├── playbooks.py                 # YAML playbook loader and execution engine
│   ├── report.py                    # HTML SOC report generation & styling
│   ├── response.py                  # Containment actions (iptables, tcpkill, conntrack)
│   ├── retry.py                     # Exponential backoff retry handler
│   └── sources/
│       ├── __init__.py              # AlertSource exports
│       ├── base.py                  # Abstract AlertSource base class
│       ├── csv.py                   # CSV alert source implementation
│       ├── elasticsearch.py         # Live Elasticsearch alert source
│       └── json_fixture.py          # Fixture replay source with dynamic timestamps
└── tests/
    ├── fixtures/
    │   ├── mixed-attacker.json      # Multi-attacker reconnaissance & exploit scenario
    │   ├── noisy-reverse-shell.json # High-volume background noise + reverse-shell scenario
    │   ├── reverse-shell.json       # Active reverse-shell threat scenario
    │   └── scan-only.json           # Reconnaissance scanning scenario
    ├── test_ai.py                   # Groq grounding & graceful degradation unit tests
    ├── test_attribution.py          # Attribution priority & reverse-shell detection tests
    ├── test_enrich.py               # MITRE keyword matching & phase grouping tests
    └── test_report.py               # Threat banner & status badge rendering tests

---

Design Notes & Known Limitations

• Inline vs. Passive Enforcement: Automated session termination (tcpkill, active TCP reset injection) and firewall blocking are fully effective when SIEM-X operates on an inline gateway or host bridging traffic between attacker and victim. In passive/mirrored tap configurations, detection and reporting operate normally, but packet reset injection may not reach endpoints.
• Aggressive Lab Containment Defaults: The default response playbook flushes existing iptables chains before applying targeted blocks to ensure reset packets flow unimpeded. This behavior is designed for dedicated cybersecurity research labs and should be customized for shared networks.
• AI Narrative Grounding: The LLM prompt is strictly constrained to reference only the deterministic technique IDs computed by the correlation engine (verified by automated tests). While this eliminates hallucinated technique IDs, the deterministic alert table remains the primary source of truth.
• Provider Model Availability: Model identifiers (e.g. openai/gpt-oss-120b) are managed by the upstream API provider. If a model identifier is deprecated by the provider, update GROQ_MODEL in .env or config.yaml.
• Query Time Window: Live queries default to a 1h window (configurable via --since or default_since). Replay mode processes all fixture events regardless of timestamp.
• SSL Certificate Verification: es_verify_ssl is disabled by default to accommodate self-signed lab certificates. Enable certificate validation in production environments.

---

Testing

Run the full automated test suite using pytest:

pytest -v

The test suite covers:

• ATT&CK Classification Accuracy: Longest-match keyword precedence against literal Suricata rule strings.
• Attacker Attribution Priority: Priority weighting of critical severity alerts over high-volume noise.
• AI Grounding Verification: Mocked API tests asserting deterministic MITRE tuples and strict grounding instructions are transmitted to the LLM.
• Report Rendering: Conditional rendering of threat banners and monitoring states.
• Fixture Replay Integrity: Dynamic timestamp recalculation and relative delta preservation.

---

License

This project is licensed under the MIT License — see the LICENSE file for details.