SigmaShield

A Sigma detection rule engine and multi-SIEM query converter for detection engineers.

  ____  _                       ____  _     _      _     _
 / ___|(_) __ _ _ __ ___   __ _/ ___|| |__ (_) ___| | __| |
 \___ \| |/ _` | '_ ` _ \ / _` \___ \| '_ \| |/ _ \ |/ _` |
  ___) | | (_| | | | | | | (_| |___) | | | | |  __/ | (_| |
 |____/|_|\__, |_| |_| |_|\__,_|____/|_| |_|_|\___|_|\__,_|
          |___/
         Sigma Detection Rule Engine & Multi-SIEM Converter

Author: Edward Marez
License: MIT
Version: 1.0.0

---

What Is Sigma?

Sigma is to log detection what Snort/Suricata rules are to network traffic, and what YARA is to malware files. It's a generic, open signature format for SIEM systems that lets you write detection rules once and deploy them anywhere.

Without Sigma, a detection engineer writes:

• One rule in Splunk SPL
• Another rule in Elasticsearch Lucene
• Another rule in KQL for Microsoft Sentinel
• Shell commands for incident response log hunting

All for the same threat. With Sigma, you write the detection logic once in YAML, and convert it to any SIEM automatically.

SigmaShield implements a full Sigma rule processing engine in pure Python — parser, converter, validator, and coverage reporter — along with 40 built-in detection rules covering the most common attack techniques.

---

Why Detection Engineering Matters

Detection engineering is the #1 skill gap in Security Operations Center (SOC) hiring. According to industry surveys:

• 75% of SOC teams struggle to write effective detection rules
• The average SOC has coverage for fewer than 30% of MITRE ATT&CK techniques
• Attackers dwell undetected for an average of 200+ days in environments with poor detection coverage
• Detection engineers earn $120k–$180k — one of the most in-demand security roles

Understanding Sigma, MITRE ATT&CK, and SIEM query languages makes you a force multiplier for any security team.

---

Features

• Full Sigma parser — handles all detection syntax including modifiers (|contains, |startswith, |endswith, |re, |all, |base64), condition expressions (1 of selection*, all of them), and aggregation pipes
• 4 SIEM backends — Splunk SPL, Elasticsearch Lucene, Microsoft Sentinel KQL, and grep/regex
• 40+ built-in detection rules — Windows (15), Linux (10), Network (8), Web (7) with MITRE ATT&CK mapping
• Field mapping pipelines — Windows Event Log, Sysmon, and Linux field name translation
• Rule validator — checks required fields, UUID format, ATT&CK tag syntax, and best practices
• Coverage reporter — generates MITRE ATT&CK matrix coverage analysis
• Rule tester — applies rules against JSON log samples to verify detections work
• Full CLI — convert, validate, coverage, test, list, demo commands
• 116 unit tests — comprehensive test coverage across all modules

---

Installation

git clone https://github.com/edwardmarez/sigmashield
cd sigmashield
pip install -r requirements.txt
pip install -e .

Requirements: Python 3.8+, PyYAML, Jinja2, colorama, tabulate

---

Quick Start

# See a full demo of all features
python -m sigmashield demo

# List all 40 built-in rules
python -m sigmashield list --rules-dir sigmashield/rules/

# Convert a rule to all 4 backends
python -m sigmashield convert --rule sigmashield/rules/windows/brute_force_login.yml --backend splunk
python -m sigmashield convert --rule sigmashield/rules/windows/brute_force_login.yml --backend elastic
python -m sigmashield convert --rule sigmashield/rules/windows/brute_force_login.yml --backend sentinel
python -m sigmashield convert --rule sigmashield/rules/windows/brute_force_login.yml --backend grep

# Convert entire rule library and save to files
python -m sigmashield convert --rules-dir sigmashield/rules/ --backend splunk --output ./converted/

# Validate all rules
python -m sigmashield validate --rules-dir sigmashield/rules/ --verbose

# Generate ATT&CK coverage report
python -m sigmashield coverage --rules-dir sigmashield/rules/ --output coverage.txt

# Test rules against sample logs
python -m sigmashield test \
    --rule sigmashield/rules/windows/powershell_encoded.yml \
    --logs sample_logs/windows_events.json

---

Multi-SIEM Conversion Example

The same Sigma rule produces a query optimized for each platform. Here is the Windows Brute Force Login rule converted to all 4 backends:

Sigma Rule (brute_force_login.yml):

title: Windows Brute Force Login Attempts
detection:
  selection:
    EventID: 4625
    LogonType:
      - 3
      - 10
  condition: selection | count() by IpAddress > 10

Splunk SPL

index=windows sourcetype="WinEventLog:Security" EventCode=4625 (LogonType=3 OR LogonType=10)
| stats count by IpAddress
| where count > 10

Elasticsearch Lucene

// Index: winlogbeat-*
winlog.channel:"Security" AND event.code:4625 AND (winlog.event_data.LogonType:3 OR winlog.event_data.LogonType:10)
// Aggregation: count() by IpAddress > 10

Microsoft Sentinel KQL

SecurityEvent
| where EventID == 4625 and LogonType in~ ("3", "10")
| summarize count() by IpAddress
| where count_ > 10

Grep (Incident Response)

grep -i -P '(?=.*EventID.*4625)(?=.*LogonType.*(3|10))' /var/log/windows/Security.evtx.log

---

Architecture

SigmaShield/
├── sigmashield/
│   ├── parser.py          # Sigma YAML → SigmaRule objects
│   │                      # Handles all modifiers and condition expressions
│   ├── backends/
│   │   ├── base.py        # Abstract BaseBackend with condition tree resolver
│   │   ├── splunk.py      # SPL with index/sourcetype context + stats
│   │   ├── elastic.py     # Lucene with ECS field mapping + index comments
│   │   ├── sentinel.py    # KQL with table routing + summarize
│   │   └── grep.py        # grep -E / grep -P with lookaheads for AND logic
│   ├── pipelines/
│   │   ├── pipeline.py    # FieldMappingPipeline base class (chainable)
│   │   ├── windows.py     # Windows Event Log field name mappings
│   │   ├── sysmon.py      # Sysmon event field mappings
│   │   └── linux.py       # Linux syslog/auditd field mappings
│   ├── validator.py       # Rule correctness + best practice checks
│   ├── coverage.py        # MITRE ATT&CK matrix coverage analysis
│   ├── testing.py         # Rule → JSON event matching engine
│   └── main.py            # CLI (argparse)
├── sigmashield/rules/     # 40 production-quality detection rules
│   ├── windows/           # 15 rules
│   ├── linux/             # 10 rules
│   ├── network/           # 8 rules
│   └── web/               # 7 rules
├── sample_logs/           # JSON event samples for rule testing
└── tests/                 # 116 unit tests

---

Detection Rules

Windows (15 rules)

Rule	Level	ATT&CK
Brute Force Login Attempts	HIGH	T1110, T1110.001
PowerShell Encoded Command	HIGH	T1059, T1059.001
Suspicious Service Installation	HIGH	T1543.003
LSASS Memory Dump	CRITICAL	T1003, T1003.001
Mimikatz sekurlsa Command	CRITICAL	T1003, T1003.001
Registry Run Key Modification	MEDIUM	T1547, T1547.001
Scheduled Task Creation	MEDIUM	T1053, T1053.005
PsExec Lateral Movement	HIGH	T1021, T1021.002
Pass-the-Hash Attack	HIGH	T1550
WMI Command Execution	MEDIUM	T1047
DLL Side-Loading	MEDIUM	T1574
UAC Bypass via eventvwr	HIGH	T1548, T1548.002
Shadow Copy Deletion	CRITICAL	T1490
RDP Lateral Movement	LOW	T1021, T1021.001
PowerShell Download Cradle	HIGH	T1059.001, T1105

Linux (10 rules)

Rule	Level	ATT&CK
Cron Job Persistence	MEDIUM	T1053
Sudo Privilege Escalation	MEDIUM	T1548
SSH Brute Force	MEDIUM	T1110, T1110.001
Reverse Shell	CRITICAL	T1059, T1059.004
/etc/passwd or /etc/shadow Access	HIGH	T1003
Kernel Module Load (Rootkit)	HIGH	T1547
Data Exfiltration via curl	MEDIUM	T1048
SUID Binary Abuse	MEDIUM	T1548.001
Docker Container Escape	HIGH	T1611
Log File Clearing	HIGH	T1070, T1070.002

Network (8 rules)

Rule	Level	ATT&CK
Port Scan Detection	MEDIUM	T1046
DNS Tunneling	MEDIUM	T1071, T1071.004
C2 Beacon Pattern	MEDIUM	T1071, T1071.001
Tor Exit Node Connection	MEDIUM	T1090
SMB Lateral Movement	MEDIUM	T1021, T1021.002
Nmap Scan Detection	MEDIUM	T1046
ICMP Data Exfiltration	MEDIUM	T1048
Password Spray Attack	HIGH	T1110, T1110.003

Web (7 rules)

Rule	Level	ATT&CK
SQL Injection	HIGH	T1190
Reflected XSS	MEDIUM	T1189
Path Traversal	HIGH	T1083
Web Shell Access	CRITICAL	T1505, T1505.003
Log4Shell (CVE-2021-44228)	CRITICAL	T1190
Command Injection	HIGH	T1190
Spring4Shell (CVE-2022-22965)	CRITICAL	T1190

---

Sigma Detection Syntax Reference

SigmaShield implements the full Sigma specification. Here are the key patterns:

Basic Field Matching

detection:
  selection:
    EventID: 4625           # Exact match
    LogonType:
      - 3                   # OR logic for lists
      - 10
  condition: selection

Modifiers

detection:
  selection:
    CommandLine|contains: '-enc'          # *-enc*
    CommandLine|startswith: 'powershell'  # powershell*
    CommandLine|endswith: '.ps1'          # *.ps1
    CommandLine|re: '(?i)invoke-.*'       # Regex
    CommandLine|contains|all:            # AND logic (all must match)
      - 'powershell'
      - '-bypass'
      - '-enc'
    Hash|base64: 'MaliciousString'        # Base64 encoded

Condition Expressions

# Single selection
condition: selection

# Exclude false positives
condition: selection and not filter

# Pattern matching (wildcard)
condition: 1 of selection*
condition: all of filter*

# All selections must match
condition: all of them

# Boolean expressions
condition: (selection_a or selection_b) and not filter

Aggregation

# Count threshold by field
condition: selection | count() by src_ip > 10

# Min/max/avg/sum
condition: selection | min(duration) by src_ip < 1000

---

Field Mapping Pipelines

Field names differ between SIEM platforms. Pipelines translate Sigma's generic names:

from sigmashield.backends import get_backend
from sigmashield.pipelines import get_pipeline
from sigmashield.parser import SigmaParser

# Parse rule
parser = SigmaParser()
rule = parser.parse_file("rule.yml")

# Apply Windows pipeline + convert to Splunk
pipeline = get_pipeline("windows")
backend = get_backend("splunk", pipeline=pipeline)
query = backend.convert_rule(rule)

Available pipelines: windows, sysmon, linux

Windows pipeline sample mappings:

Sigma Field	Windows Event Log
EventID	EventCode
Image	NewProcessName
ParentImage	ParentProcessName
TargetUsername	TargetUserName
SubjectUsername	SubjectUserName

---

Python API

from sigmashield import SigmaParser, SigmaValidator, CoverageReporter
from sigmashield.backends import get_backend
from sigmashield.testing import RuleTester

# Parse a rule
parser = SigmaParser()
rule = parser.parse_file("sigmashield/rules/windows/brute_force_login.yml")
print(f"Title: {rule.title}")
print(f"ATT&CK: {', '.join(rule.mitre_attack_techniques)}")

# Convert to Splunk
backend = get_backend("splunk")
query = backend.convert_rule(rule)
print(query)

# Validate rules
validator = SigmaValidator()
result = validator.validate_rule(rule)
if not result.is_valid:
    for finding in result.errors:
        print(f"ERROR: {finding.message}")

# Generate ATT&CK coverage report
reporter = CoverageReporter()
report = reporter.analyze_directory("sigmashield/rules/")
print(f"Coverage: {report.coverage_percentage:.1f}%")
print(report.format_text())

# Test rule against log events
tester = RuleTester()
result = tester.test_rule_against_file(rule, "sample_logs/windows_events.json")
print(f"Matches: {result.match_count} of {result.total_events} events")
for match in result.matches:
    if match.matched:
        print(f"  Matched event: {match.event.get('description', '')}")

---

CLI Reference

python -m sigmashield <command> [options]

convert — Convert rules to SIEM queries

# Single rule
python -m sigmashield convert \
    --rule sigmashield/rules/windows/brute_force_login.yml \
    --backend splunk

# All rules, save to files
python -m sigmashield convert \
    --rules-dir sigmashield/rules/ \
    --backend sentinel \
    --output ./converted/

# With field mapping pipeline
python -m sigmashield convert \
    --rules-dir sigmashield/rules/windows/ \
    --backend splunk \
    --pipeline windows

# Available backends: splunk, elastic, sentinel, grep

validate — Validate rule quality

# Validate single rule
python -m sigmashield validate --rule rule.yml

# Validate entire library
python -m sigmashield validate --rules-dir sigmashield/rules/ --verbose

coverage — ATT&CK coverage report

# Print report
python -m sigmashield coverage --rules-dir sigmashield/rules/

# Save to file
python -m sigmashield coverage --rules-dir sigmashield/rules/ --output coverage.txt

# JSON export for dashboards
python -m sigmashield coverage --rules-dir sigmashield/rules/ --json coverage.json

test — Test rules against log samples

# Test single rule
python -m sigmashield test \
    --rule sigmashield/rules/windows/brute_force_login.yml \
    --logs sample_logs/windows_events.json

# Test all rules
python -m sigmashield test \
    --rules-dir sigmashield/rules/ \
    --logs sample_logs/windows_events.json \
    --show-all

list — List available rules

# Simple list
python -m sigmashield list --rules-dir sigmashield/rules/

# Table format (requires tabulate)
python -m sigmashield list --rules-dir sigmashield/rules/ --format table

# JSON export
python -m sigmashield list --rules-dir sigmashield/rules/ --format json

# Filter by level
python -m sigmashield list --rules-dir sigmashield/rules/ --level critical

# Filter by product
python -m sigmashield list --rules-dir sigmashield/rules/ --product windows

# Filter by ATT&CK technique
python -m sigmashield list --rules-dir sigmashield/rules/ --technique T1059

demo — Full feature demonstration

python -m sigmashield demo

---

Integrating with Real SIEMs

Splunk

1. Save converted queries to .spl files
2. In Splunk: Search & Reporting → Alerts → Create Alert
3. Paste the SPL query, set trigger conditions, configure notifications
4. For continuous detection: use Splunk Enterprise Security Correlation Searches

Elasticsearch / Kibana

1. Convert rules with --backend elastic
2. In Kibana: Security → Rules → Create Detection Rule → Custom Query
3. Paste the Lucene query
4. Enable ECS (Elastic Common Schema) for field mappings to work correctly

Microsoft Sentinel

1. Convert rules with --backend sentinel
2. In Sentinel: Analytics → Create → Scheduled query rule
3. Paste the KQL query and set the schedule
4. Map fields to your data connectors if needed

Incident Response (grep)

For quick log hunting during an incident when you have direct server access:

# Convert and run immediately
python -m sigmashield convert \
    --rule sigmashield/rules/linux/reverse_shell.yml \
    --backend grep

# Output:
grep -i -P '(bash.*>.*\/dev\/tcp|0>&1)' /var/log/syslog

---

Writing Custom Sigma Rules

title: My Custom Detection Rule
id: <generate with python -c "import uuid; print(uuid.uuid4())">
status: experimental
description: |
  Describe what this rule detects, why it matters,
  and what attacker technique it covers.
author: Your Name
date: 2024-01-15
references:
  - https://attack.mitre.org/techniques/T1234/
tags:
  - attack.persistence
  - attack.t1234
logsource:
  category: process_creation  # OR product/service
  product: windows
detection:
  selection:
    Image|endswith: '\malicious.exe'
    CommandLine|contains:
      - '-evil'
      - '--bad'
  filter_legit:
    CommandLine|contains: 'legitimate_vendor'
  condition: selection and not filter_legit
fields:
  - CommandLine
  - Image
  - ParentImage
falsepositives:
  - List known false positive scenarios here
level: high  # informational, low, medium, high, critical

Validate your rule before deploying:

python -m sigmashield validate --rule my_rule.yml --verbose

---

MITRE ATT&CK Coverage

SigmaShield's 40 built-in rules cover 39 ATT&CK techniques across 10 tactics:

Tactic	Coverage
Credential Access	T1003, T1110, T1550
Persistence	T1053, T1543, T1547, T1505
Execution	T1059, T1047, T1059.004
Lateral Movement	T1021, T1021.001, T1021.002
Privilege Escalation	T1548, T1548.002, T1611
Defense Evasion	T1070, T1574, T1090
Initial Access	T1190, T1189
Discovery	T1046, T1083
Command and Control	T1071, T1071.004
Exfiltration	T1048

Generate your current coverage:

python -m sigmashield coverage --rules-dir sigmashield/rules/

---

Testing

# Run all tests
python -m pytest tests/ -v

# Run specific test module
python -m pytest tests/test_parser.py -v
python -m pytest tests/test_backends.py -v
python -m pytest tests/test_validator.py -v
python -m pytest tests/test_pipelines.py -v

# With coverage
pip install pytest-cov
python -m pytest tests/ --cov=sigmashield --cov-report=term-missing

Test coverage: 116 tests across 4 modules:

• test_parser.py — 35 tests: YAML parsing, modifiers, aggregation, condition resolution
• test_backends.py — 34 tests: All 4 backends, including full rule library conversion
• test_validator.py — 25 tests: Valid rules, warnings, errors, built-in rule set
• test_pipelines.py — 22 tests: All 3 pipelines, factory, backend integration

---

Project Structure

SigmaShield/
├── sigmashield/
│   ├── __init__.py
│   ├── __main__.py          # python -m sigmashield entry point
│   ├── main.py              # CLI with argparse
│   ├── parser.py            # SigmaRule, SigmaParser, FieldCondition, etc.
│   ├── validator.py         # SigmaValidator, ValidationResult, ValidationFinding
│   ├── coverage.py          # CoverageReporter, CoverageReport
│   ├── testing.py           # RuleTester, RuleTestResult
│   ├── utils.py             # Escape helpers, wildcard conversions, utilities
│   ├── backends/
│   │   ├── __init__.py      # get_backend() factory
│   │   ├── base.py          # Abstract BaseBackend
│   │   ├── splunk.py        # Splunk SPL backend
│   │   ├── elastic.py       # Elasticsearch Lucene backend
│   │   ├── sentinel.py      # Microsoft Sentinel KQL backend
│   │   └── grep.py          # grep/egrep CLI backend
│   ├── pipelines/
│   │   ├── __init__.py      # get_pipeline() factory
│   │   ├── pipeline.py      # FieldMappingPipeline, PipelineRule
│   │   ├── windows.py       # Windows Event Log field mappings
│   │   ├── sysmon.py        # Sysmon field mappings
│   │   └── linux.py         # Linux syslog/auditd field mappings
│   └── rules/
│       ├── windows/         # 15 Windows detection rules
│       ├── linux/           # 10 Linux detection rules
│       ├── network/         # 8 network detection rules
│       └── web/             # 7 web application detection rules
├── sample_logs/
│   ├── windows_events.json  # Windows event samples
│   ├── linux_events.json    # Linux syslog/auth samples
│   └── README.md
├── tests/
│   ├── test_parser.py
│   ├── test_backends.py
│   ├── test_validator.py
│   └── test_pipelines.py
├── requirements.txt
├── setup.py
├── LICENSE
└── README.md

---

Resources

• Sigma Project — Official Sigma specification and community rules
• Sigma Specification — Full syntax reference
• MITRE ATT&CK Framework — Adversary tactics and techniques
• MITRE ATT&CK Navigator — Coverage visualization
• Detection Engineering Weekly — Industry newsletter
• Sigma HQ Rules — 3000+ community rules
• Elastic Security Rules — Elastic's detection library
• Splunk Security Essentials — Splunk detection content

---

License

MIT License — Copyright (c) 2024 Edward Marez. See LICENSE for details.

---

Contributing

1. Fork the repository
2. Create a feature branch: git checkout -b feature/new-detection-rule
3. Add your Sigma rule to the appropriate rules/ subdirectory
4. Validate: python -m sigmashield validate --rule rules/your_rule.yml
5. Run tests: python -m pytest tests/
6. Submit a pull request

---

Built by Edward Marez — Detection Engineer