Skip to content

Vigil-SOC/vigil

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

39 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Vigil

Vigil is a community built AI-Native Security Operations Center built on three pillars: Agents for performing specific capabilities, Workflows for orchestrated multi-agent workflows, and Integrations for data integestion, tools and integrations to other open source projects. Actually the most important pillar is YOU - this is your project, please contribute via feedback, code, a repo star, memes on Discord or otherwise.


12 Specialized AI Agents

Every agent has access to 19 backend tools via Agent SDK and 100+ additional tools via MCP. Agents are the building blocks that Workflows orchestrate.

Agent Role Thinking Key Capability
Triage Rapid alert assessment Fast Severity scoring, false-positive filtering, escalation decisions
Investigator Root cause analysis Deep Evidence collection, timeline reconstruction, cross-source correlation
Threat Hunter Proactive hunting Deep Hypothesis-driven anomaly detection, pattern intelligence from 7,200+ rules
Correlator Multi-signal linking Deep Campaign identification, attack chain reconstruction, entity mapping
Responder Containment actions Fast NIST IR containment, blast radius assessment, confidence-scored approval requests
Reporter Documentation Balanced Executive summaries, technical reports, audience-tailored content
MITRE Analyst ATT&CK mapping Deep Technique identification, coverage analysis, gap prioritization, detection templates
Forensics Digital forensics Deep Artifact analysis, chain of custody, multi-domain examination
Threat Intel IOC enrichment Deep Actor attribution, campaign tracking, OSINT integration
Compliance Regulatory checks Balanced NIST, ISO, PCI-DSS, HIPAA, GDPR, SOC 2 assessment
Malware Analyst Malware examination Deep Static/dynamic analysis, family classification, C2 identification
Network Analyst Traffic analysis Deep Flow analysis, protocol anomalies, lateral movement detection

Workflows — One-Click Multi-Agent Workflows

Workflows are the operational core of Vigil. Each worfklow chains multiple specialized AI agents into an end-to-end playbook that executes with a single command. No manual hand-offs, no copy-pasting between tools — the agents coordinate automatically.

Workflow Agents What It Does
Incident Response Triage → Investigator → Responder → Reporter NIST IR framework: triage an alert, investigate root cause, contain the threat, produce an audit-ready report
Full Investigation Investigator → MITRE Analyst → Correlator → Responder → Reporter Deep-dive with ATT&CK mapping, cross-signal correlation, response planning, and comprehensive documentation
Threat Hunt Threat Hunter → Network Analyst → Malware Analyst → Threat Intel → Reporter Hypothesis-driven hunting across network, endpoint, and threat intel — with IOC enrichment and detection recommendations
Forensic Analysis Forensics → Malware Analyst → Network Analyst → Reporter Post-incident digital forensics with evidence preservation, chain-of-custody documentation suitable for legal proceedings

How it works: Say "Run incident response on finding f-20260215-abc123" and the system sequences four agents — triage scores the alert, investigator digs into root cause, responder submits containment actions with confidence-based approval, and reporter generates the final documentation.

Workflows are defined as WORKFLOW.md files in the workflows/ directory and are fully customizable. Create your own by defining the agent sequence, tools used, and phase-by-phase instructions.

workflows/
├── incident-response/WORKFLOW.md
├── full-investigation/WORKFLOW.md
├── threat-hunt/WORKFLOW.md
└── forensic-analysis/WORKFLOW.md

Integrations -

Vigil uses the Model Context Protocol to connect agents to your existing tools. These MCP servers give every agent real-time access to your SIEM, EDR, threat intel, sandbox, ticketing, and communication platforms — all through a unified interface.

Category Integrations Tools
SIEM Splunk Natural language → SPL, search by IP/host/user, index listing
EDR / XDR CrowdStrike Alert lookup, host isolation/unisolation, host status
Threat Intel VirusTotal, Shodan, AlienVault OTX, MISP Hash/IP/domain/URL reputation, host recon, pulse matching, IOC search
Sandbox Hybrid Analysis, Joe Sandbox, ANY.RUN File submission, report retrieval, IOC extraction
Timeline Timesketch Forensic timeline analysis, evidence export
Detection Engineering Security-Detections-MCP 7,200+ rules (Sigma, Splunk, Elastic, KQL), 71 tools, coverage analysis, gap identification
Ticketing Jira Issue creation, updates, search
Communication Slack Alerts, channel creation, file uploads
Data Pipeline Cribl Stream Log normalization, noise filtering, multi-destination routing
Core DeepTempo Findings, Approval, ATT&CK Layer, Tempo Flow Built-in SOC operations

Coming soon: AWS Security Hub, Azure Sentinel, GCP Security, Okta, Microsoft Defender, SentinelOne, Carbon Black, PagerDuty.

MCP servers live in mcp-servers/ and are configured via the Settings UI or mcp_config.json. Add a new integration by dropping an MCP server into the tools/ directory — or use the built-in Custom Integration Builder to generate one from API docs. If you build an integration that you find useful, chances are someone else will as well. Please contribute!


Quick Start

Prerequisites

  • Python 3.10+
  • Node.js 18+ (for frontend)
  • Docker Desktop (must be running — used for PostgreSQL)
  • Git (with submodule support)
  • Claude API key from console.anthropic.com (optional for initial testing)

Step 1: Clone & start the platform

git clone --recurse-submodules https://github.com/Vigil-Soc/vigil.git
cd vigil
./start_web.sh

This starts PostgreSQL, Redis, the FastAPI backend (port 6987), LLM worker, and React frontend (port 6988). The startup script handles everything: creates the Python virtual environment, installs dependencies, starts PostgreSQL, initializes the database with a default admin user, installs frontend packages, and launches both servers.

Default login: admin / admin123 (auth is bypassed when DEV_MODE=true, which is the default).

URL
Frontend http://localhost:6988
API http://localhost:6987
API Docs http://localhost:6987/docs
Manual install (if you need to set up step-by-step)
# Clone with submodules
git clone --recurse-submodules https://github.com/Vigil-Soc/vigil.git
cd vigil

# If you already cloned without --recurse-submodules:
git submodule update --init --recursive

# Environment (DEV_MODE enabled by default)
cp env.example .env
# Edit .env and add your ANTHROPIC_API_KEY (optional for testing)

# Backend setup
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Frontend setup
cd frontend
npm install
cd ..

Step 2: Add your API key

Edit .env and set:

ANTHROPIC_API_KEY=sk-ant-...

Restart after changing.

Step 3: Ingest data

You have several options:

Option A — Generate sample data (quickest way to test):

source venv/bin/activate
python scripts/generate_sample_data.py --count 50

Option B — Upload your own file (JSON, JSONL, CSV, or Parquet):

Go to http://localhost:6988 → Settings → Import and upload a file with security findings.

Option C — API upload:

curl -X POST http://localhost:6987/api/ingestion/upload \
  -F "file=@your_findings.json" \
  -F "data_type=finding" \
  -F "format=json"

A finding looks like:

{
  "finding_id": "f-001",
  "description": "Suspicious PowerShell activity detected on WORKSTATION-42",
  "severity": "critical",
  "status": "new",
  "data_source": "edr",
  "anomaly_score": 0.87,
  "mitre_predictions": {"T1059.001": 0.92},
  "entity_context": {"source_ip": "192.168.1.100", "hostname": "WORKSTATION-42"},
  "timestamp": "2026-03-21T15:30:00Z"
}

Step 4: Configure integrations (optional)

In the UI: Settings → Integrations, or set in .env:

Integration Env Vars
Splunk SPLUNK_URL, SPLUNK_USERNAME, SPLUNK_PASSWORD
CrowdStrike CROWDSTRIKE_CLIENT_ID, CROWDSTRIKE_CLIENT_SECRET
VirusTotal VIRUSTOTAL_API_KEY
Slack SLACK_BOT_TOKEN, DAEMON_SLACK_CHANNEL

Step 5: Run workflows on findings

In the UI:

  1. Go to Findings
  2. Select a finding
  3. Click Run Workflow → choose one:
    • Incident Response — triage → investigate → respond → report
    • Full Investigation — deep analysis with MITRE mapping
    • Threat Hunt — proactive hunting across data sources
    • Forensic Analysis — evidence gathering and chain of custody

Or use the chat interface — just ask Claude:

"Run incident response on finding f-001"
"Find similar findings and add them to a case"

Step 6: Run the autonomous daemon (optional)

./start_daemon.sh

This polls your connected sources (Splunk, CrowdStrike), auto-triages new findings, and responds based on confidence thresholds. It listens for webhooks on port 8081 and exposes metrics on port 9090.

Step 7: Shut down

./shutdown_all.sh              # Stop processes, keep Docker running
./shutdown_all.sh -d           # Stop everything including Docker
./shutdown_all.sh -d --full    # Stop + remove containers and volumes

Architecture

┌──────────────────────────────────────────────────────────────────┐
│                       Workflows Layer                             │
│  Incident Response │ Full Investigation │ Threat Hunt │ Forensics │
│              (Multi-agent workflow orchestration)                  │
└──────────────────────────────────────────────────────────────────┘
                               │
                               ▼
┌──────────────────────────────────────────────────────────────────┐
│                   12 Specialized AI Agents                        │
│  Triage │ Investigator │ Hunter │ Correlator │ Responder │ ...   │
└──────────────────────────────────────────────────────────────────┘
                │                              │
     Agent SDK (19 tools)              MCP (100+ tools)
                │                              │
                ▼                              ▼
┌──────────────────────────┐  ┌────────────────────────────────────┐
│     Backend Services     │  │          MCP Servers (30+)         │
│  Detections (7,200+)     │  │  Splunk │ CrowdStrike │ VirusTotal │
│  Case Management         │  │  Shodan │ Jira │ Slack │ Cribl    │
│  Approvals │ MITRE ATT&CK│  │  Timesketch │ MISP │ ANY.RUN      │
│  Similarity Search       │  │  Hybrid Analysis │ Joe Sandbox    │
└──────────────────────────┘  └────────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────────────────────┐
│                  Data Sources + PostgreSQL                         │
│  Logs │ Alerts │ Findings │ Embeddings │ Cases │ Detection Rules  │
└──────────────────────────────────────────────────────────────────┘

Additional Features

  • Chat-Driven Case Management — Build cases through natural language. Say "add this to case XYZ" and the system handles findings, activities, timelines, and MITRE tagging. Learn more
  • Detection Engineering — 7,200+ detection rules (Sigma, Splunk, Elastic, KQL) with coverage analysis, gap identification, and AI-assisted template generation. Learn more
  • Case Management — Full lifecycle tracking with PDF reports
  • Approval Workflow — Human-in-the-loop with confidence-based automation (auto-approve above 0.90, require review below 0.85)
  • AI Enrichment — Automatic threat analysis cached per finding
  • MITRE ATT&CK — Technique mapping and Navigator layer visualization

Project Structure

ai-opensoc/
├── workflows/         # WORKFLOW.md definitions (4 built-in)
├── mcp-servers/       # MCP server implementations (30+)
├── backend/           # FastAPI backend API + Agent SDK tools
├── frontend/          # React + MUI frontend
├── services/          # Business logic (workflows service, etc.)
├── daemon/            # Headless autonomous SOC service
├── tools/             # Additional tool implementations
├── database/          # PostgreSQL models and migrations
├── core/              # Config, rate limiting, exceptions
├── docker/            # Docker Compose setup
├── docs/              # Documentation
└── data/schemas/      # JSON validation schemas

Submodules

Vigil depends on two git submodules that must be initialized alongside the main repo (hence --recurse-submodules in the clone command). deeptempo-core provides the shared core library used by the backend and agents. mcp-servers contains the 30+ Model Context Protocol server implementations (Splunk, CrowdStrike, VirusTotal, Jira, Slack, etc.) that give every agent access to your existing security tooling. If you cloned without --recurse-submodules, run git submodule update --init --recursive to pull them in.

Example Usage

Run a Workflow

You: "Run incident response on finding f-20260215-abc123"
Claude: [triage] Severity: Critical — confirmed C2 beaconing from HOST-42
        [investigate] Root cause: phishing email → macro execution → Cobalt Strike beacon
        [respond] Submitted host isolation (confidence 0.96 — auto-approved)
        [report] Incident report generated with MITRE ATT&CK layer

Proactive Threat Hunt

You: "Hunt for C2 beaconing activity across all network findings"
Claude: [hunt] Hypothesis: periodic outbound connections to rare destinations
        [network] Found 3 hosts beaconing to 185.220.101.0/24 every 300s
        [malware] Cobalt Strike beacon — extracted 4 IOCs
        [intel] IP attributed to APT28 infrastructure (confidence 0.72)
        [report] Hunt report with 12 IOCs and 3 new detection recommendations

Chat-Driven Case Building

You: "Add this to case-20260121-abc123 and note it's part of the kill chain"
Claude: ✓ Added finding to case
        ✓ Logged activity: Part of lateral movement kill chain
        ✓ Tagged with T1021.001 (RDP)

You: "Find similar findings and add them all to this case"
Claude: ✓ Found 3 similar findings via embedding search
        ✓ Added f-002, f-003, f-004 to case
        ✓ Updated timeline with lateral movement progression

Documentation

Doc Contents
docs/AGENTS.md 12 SOC AI agents reference
docs/INTEGRATIONS.md MCP integrations — Splunk, CrowdStrike, VirusTotal, 28+ tools
docs/DETECTION_ENGINEERING.md Detection engineering with 7,200+ rules
docs/CHAT_CASE_MANAGEMENT.md Chat-driven case building guide
docs/CHAT_CASE_QUICK_REFERENCE.md Quick reference for chat commands
docs/CONFIGURATION.md Environment variables, secrets, deployment
docs/FEATURES.md Cases, approvals, enrichment
docs/API.md MCP tool contracts, data models
docs/README.md Architecture overview
docs/SPLUNK_TESTING_GUIDE.md Splunk test data and integration testing

Testing with Splunk & Claude

Click to expand Splunk testing instructions
# Generate 280 realistic security events
python3 scripts/generate_splunk_test_data.py

# Send directly to Splunk
python3 scripts/generate_splunk_test_data.py \
    --send-to-splunk \
    --hec-url https://your-splunk:8088/services/collector \
    --hec-token your-hec-token \
    --no-verify-ssl

# Test full integration (generate → create case → enrich with Claude)
python3 scripts/test_splunk_claude_integration.py \
    --generate-data \
    --create-case

Test data: 280 events (brute force, malware, C2 traffic, exfiltration, privilege escalation, lateral movement, recon) with full MITRE ATT&CK mappings and realistic IOCs.

See the Splunk Testing Guide for complete instructions.

Export PostgreSQL Data to Splunk

Click to expand export instructions
# Export everything to Splunk
python scripts/export_postgres_to_splunk.py \
    --hec-url https://your-splunk:8088/services/collector \
    --hec-token your-hec-token \
    --index deeptempo \
    --no-verify-ssl

# Save to file for review first
python scripts/export_postgres_to_splunk.py \
    --save-to-file postgres_export.json

Quick Start: See POSTGRES_TO_SPLUNK_QUICKSTART.md Full Guide: See docs/POSTGRES_TO_SPLUNK_EXPORT.md

Contributing

Contributions are welcome! Whether you're fixing bugs, adding new MCP integrations, improving agent prompts, or building new workflows or agents — we'd love your help and leadership.

Join the community: Connect with the DeepTempo community on Discord to discuss ideas, get help, and collaborate with other contributors.

To contribute:

  1. Fork the repo and create a feature branch
  2. Make your changes and test them
  3. Submit a pull request with a clear description

See QuickStart to get your local environment running.


License

Apache 2.0 — See LICENSE

References

About

Vigil - an ever improving 100% OpenSource AI system for security

Resources

License

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors