Pattern Development

Pattern Development Guide

Aegis patterns ship as Rust structs in crates/aegis-patterns/src/ and are compiled into every surface. Two more paths exist for extending detections without touching the compiled corpus:

1. Rust — core patterns, reviewed and shipped with the binary
2. YAML — contributions bundled into a pattern pack with aegis-bundler
3. .aegis.yml — per-project custom patterns merged in at scan time

Writing a Good Pattern

A pattern is a regex plus metadata. The regex uses the Rust regex crate (RE2 syntax): no lookarounds, . does not cross newlines, and nested quantifiers cannot backtrack (keep repetitions bounded anyway).

Best Practices

1. Use word boundaries

# BAD - matches substrings
sk-[A-Za-z0-9]{32}

# GOOD - whole tokens only
\bsk-[A-Za-z0-9]{32}\b

2. Be specific

# TOO GENERIC - many false positives
[A-Z]{2}-[A-Z]{5}

# SPECIFIC - identifiable prefix + assignment context
\bapi[_-]?token["\s]*[:=]["\s]*[A-Za-z0-9]{32,}

3. Set an entropy floor for secrets — low-entropy matches (password123) are usually placeholders:

min_entropy: Some(3.5)

4. Suppress safe idioms with exclude — when the exclude regex also matches the candidate span, the finding is suppressed. Use it to exempt documentation examples and placeholders. The exclude only sees the matched span, not the rest of the line.

5. Never use nested quantifiers that can explode — the RE2 engine will not hang, but unbounded nesting still wastes cycles:

# BAD
(a+)+$

# GOOD
\ba{8,32}\b

Rust Format (core patterns)

Add a Pattern to the appropriate module in crates/aegis-patterns/src/<category>.rs:

Pattern {
    name: "my-service-key".to_string(),
    category: "secrets".to_string(),
    match_pattern: r#"(?i)myservice[_-]?key\s*[:=]\s*['"][A-Za-z0-9]{16,}"#.to_string(),
    enabled: true,
    severity: "high".to_string(),      // critical | high | medium | low
    confidence: "high".to_string(),    // high | medium | low
    min_entropy: Some(3.5),
    description: "Detects hardcoded MyService keys".to_string(),
    reference: Some("https://docs.example.com/security".to_string()),
    tags: vec!["secrets".to_string(), "api-key".to_string()],
    env_var: false,   // true = only runs during `aegis scan --env`
    binary: false,    // true = may match inside binary files
    exclude: Some(r#"(?i)example|placeholder|your[-_]key"#.to_string()),
    file_extensions: Vec::new(), // empty = every text file
}

Then:

1. Add the pattern to crates/aegis-patterns/src/lib.rs dispatch if it opens a new category.
2. Run cargo run -p aegis-patterns --example generate_docs — the generated catalog must stay fresh; a CI test fails otherwise.
3. Add positive/negative fixtures to crates/aegis-cli/tests/pattern_fixtures.rs.
4. scripts/generate_examples.py backfills the liveness example — every shipped pattern must have a provably firing example (pattern_liveness.rs fails CI without one).

Naming rules: pattern names and categories are kebab-case (my-service-key); the hygiene tests reject anything else. Categories must already exist in aegis_patterns::by_category().

YAML Format (aegis-bundler)

YAML contributions skip the recompile step. Each pattern is a document in a YAML list (field names mirror the Rust struct):

- name: my-service-key
  category: secrets
  match: '(?i)myservice[_-]?key\s*[:=]\s*["''][A-Za-z0-9]{16,}'
  severity: high
  confidence: high
  min_entropy: 3.5
  description: Detects hardcoded MyService keys
  enabled: true

Bundle it and load it into a running MCP server:

cargo build --release -p aegis-bundler
./target/release/aegis-bundler ./patterns-dir ./my-patterns.bundle
# then: MCP method update_bundle with params ["./my-patterns.bundle", true]

Per-Project Custom Patterns (.aegis.yml)

A scan root may contain a .aegis.yml (or .aegis.yaml) file whose patterns are merged into the registry before the walk — useful for project-specific rules without shipping them globally. See the CLI guide.

Testing Your Pattern

Every pattern in the corpus is covered by three CI-enforced suites — mirror them for new rules:

• Liveness (pattern_liveness.rs): the pattern fires on its stored example, generated by scripts/generate_examples.py.
• Hygiene (registry_hygiene.rs): regexes compile, names and categories are kebab-case, severities/confidences are enumerated values, references are real HTTPS URLs, entropy floors sit in the Shannon range, and the generated docs are fresh.
• Precision/recall (corpus_precision_recall.rs): fixtures in crates/aegis-core/tests/corpus/ keep false positives at or below a measured gate.

Manual check:

cargo run -p aegis-cli -- scan --stdin <<< 'myservice_key = "AbCdEf1234567890AbCdEf1234567890"'

Submitting Patterns

1. Fork the repository and create a branch
2. Add the pattern (Rust or YAML) with fixtures and an example
3. Run the gates: cargo fmt --all, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace
4. Submit a pull request

See CONTRIBUTING for full guidelines.

Pattern Quality Checklist

• ✅ Specificity — uses word boundaries \b where appropriate
• ✅ Accuracy — minimal false positives on real code
• ✅ Clarity — name clearly describes what it detects (kebab-case)
• ✅ Testing — liveness example + precision fixtures pass
• ✅ Entropy — appropriate min_entropy for secret-shaped rules
• ✅ Metadata — description, tags, and a reference URL
• ✅ Docs — generated catalog regenerated and committed

Common Mistakes

Overly Generic

[0-9]{32}   # matches ANY 32-digit number

Missing Word Boundaries

password    # matches "password" anywhere, including prose

Wrong Engine Assumptions

(?<=prefix)token   # lookarounds do not exist in RE2 — express "prefix
                   # without capturing it" via the exclude field instead