Skip to content

docs(readme): add Detection Rules as Memory section#74

Merged
rolandpg merged 1 commit into
masterfrom
docs/detection-rules-readme
Apr 18, 2026
Merged

docs(readme): add Detection Rules as Memory section#74
rolandpg merged 1 commit into
masterfrom
docs/detection-rules-readme

Conversation

@rolandpg
Copy link
Copy Markdown
Owner

Follow-up to #70. Adds a README section documenting the Sigma + YARA first-class primitives for readers who don't click through to the blog post.

What's included

  • One-paragraph explainer + reference to the shared DetectionRule supertype
  • Upstream-spec references (SigmaHQ JSON schema, CCCS YARA metadata, YARA docs)
  • Python API snippet (ingest_sigma + ingest_yara)
  • CLI usage (single-file, bulk-directory, --dry-run)
  • Honest framing of the LLM rule explainer: synchronous on-demand in v1, async enrichment-queue wiring is v1.1 work (matches detection/explainer.py:7-9 docstring)

Voice + scope

  • Brand-voice compliant: no emoji, no exclamation marks, no hype adjectives ("powerful", "blazing-fast")
  • No comparative framing against Mem0 / Graphiti / LangMem
  • No SOC Prime / DetectFlow endorsement claims
  • Matches the existing ## Quick Start / ## How It Works / ATHF bridge voice
  • Placement: between ## MCP Server (Claude Code) and ## Integrations (alongside the ATHF bridge row)

Related

Follow-up to #70. Documents the Sigma + YARA first-class
primitives for README readers who don't click through to the blog
post. Covers: what ships, Python API + CLI usage, LLM rule
explainer (honestly framed as v1 synchronous, v1.1 async), and
upstream-spec references.

Brand-voice compliant (no emoji, no hype adjectives, no
comparative framing). Placement: between MCP Server section and
Integrations section, next to the existing ATHF bridge row.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 18, 2026 01:26
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new README section that documents Sigma + YARA “detection rules as memory” to make the detection-rule primitives discoverable without the blog post.

Changes:

  • Introduces a “Detection Rules as Memory (Sigma + YARA)” section explaining rule ingest and ontology/graph integration
  • Adds Python API and CLI usage examples for Sigma/YARA ingest (including --dry-run)
  • Adds references to upstream specs and documents explainer behavior + rate limiting

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md

## Detection Rules as Memory (Sigma + YARA)

Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so a single rule UUID is addressable across both formats.
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The statement that a “single rule UUID is addressable across both formats” looks inaccurate. In code, Sigma rule IDs come from the Sigma id field (or a sigma_<hash> fallback) while YARA uses CCCS meta.id (or a yara_<hash> fallback), so there isn’t a shared cross-format UUID namespace—only a shared DetectionRule field contract. Consider rephrasing to say both formats share the DetectionRule interface/fields rather than implying the same UUID identifies the same rule in both formats.

Suggested change
Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so a single rule UUID is addressable across both formats.
Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so both formats share the same rule interface and fields even though each format keeps its own rule identifier.

Copilot uses AI. Check for mistakes.
Comment thread README.md

## Detection Rules as Memory (Sigma + YARA)

Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so a single rule UUID is addressable across both formats.
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This paragraph claims tags become graph edges for “tools” and “malware families”, but the current mapping only upgrades Sigma attack.s#### tags to Malware and doesn’t emit a distinct Tool entity type (and YARA tag resolution only emits AttackPattern, Vulnerability, ThreatActor, or YaraTag). To avoid over-promising, consider narrowing the wording to what’s actually emitted today (e.g., ATT&CK techniques, CVEs, ATT&CK group/software IDs, and generic tags).

Suggested change
Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so a single rule UUID is addressable across both formats.
Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges for currently supported mappings such as MITRE ATT&CK techniques, CVEs, ATT&CK group/software IDs, and generic tags, all resolved against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so a single rule UUID is addressable across both formats.

Copilot uses AI. Check for mistakes.
Comment thread README.md
python -m zettelforge.sigma.ingest rules/ --dry-run
```

An LLM rule explainer (`zettelforge.detection.explainer.explain`) produces a structured JSON summary -- intent, key fields, evasion notes, false-positive hypotheses -- for any `DetectionRule`. It runs synchronously on demand in v1; async enrichment-queue wiring is v1.1. Rate-limited via `ZETTELFORGE_EXPLAIN_RPM` (default 60 calls/minute).
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The explainer output described here (“intent, key fields, evasion notes…”) doesn’t match the current RuleExplanation schema in zettelforge.detection.explainer, which returns fields like summary, mechanism, threat_model, false_positive_patterns, related_techniques, and confidence. Suggest updating the README wording to match the actual keys so users don’t build against fields that aren’t produced.

Suggested change
An LLM rule explainer (`zettelforge.detection.explainer.explain`) produces a structured JSON summary -- intent, key fields, evasion notes, false-positive hypotheses -- for any `DetectionRule`. It runs synchronously on demand in v1; async enrichment-queue wiring is v1.1. Rate-limited via `ZETTELFORGE_EXPLAIN_RPM` (default 60 calls/minute).
An LLM rule explainer (`zettelforge.detection.explainer.explain`) produces a structured JSON summary for any `DetectionRule`, including fields such as `summary`, `mechanism`, `threat_model`, `false_positive_patterns`, `related_techniques`, and `confidence`. It runs synchronously on demand in v1; async enrichment-queue wiring is v1.1. Rate-limited via `ZETTELFORGE_EXPLAIN_RPM` (default 60 calls/minute).

Copilot uses AI. Check for mistakes.
@rolandpg rolandpg merged commit b0b2d45 into master Apr 18, 2026
14 of 15 checks passed
@rolandpg rolandpg deleted the docs/detection-rules-readme branch April 18, 2026 01:30
rolandpg added a commit that referenced this pull request Apr 19, 2026
Detection-rules-as-memory, MCP Registry publication, SQLite concurrency
hardening, test-suite hygiene, and brand/docs polish.

Highlights:
- feat: Sigma + YARA as first-class memory entities with LLM rule
  explainer (#70) + Detection Rules as Memory README section (#74)
- feat: MCP Registry publication (server.json + mcp-name tag) (#75)
- fix: SQLite reader concurrency — 16 methods now hold _write_lock
  (closes #68, fixes a production read-during-write race) (#69)
- fix: 3 CI test regressions stabilized (#67)
- chore: test-suite hygiene — 280→305 passing, 17→10 skipped,
  2→0 xfailed; migrated langchain_retriever to Pydantic V2 ConfigDict
  (#62, #63, #64, #65)
- brand: neural-chain architecture diagram + light/dark parity,
  canonical security channels, refreshed social preview (#61)

See CHANGELOG.md for details.

Bumps: pyproject.toml, src/zettelforge/__init__.py, mkdocs.yml,
server.json, SECURITY.md.
@rolandpg rolandpg mentioned this pull request Apr 19, 2026
rolandpg added a commit that referenced this pull request Apr 19, 2026
Detection-rules-as-memory, MCP Registry publication, SQLite concurrency
hardening, test-suite hygiene, and brand/docs polish.

Highlights:
- feat: Sigma + YARA as first-class memory entities with LLM rule
  explainer (#70) + Detection Rules as Memory README section (#74)
- feat: MCP Registry publication (server.json + mcp-name tag) (#75)
- fix: SQLite reader concurrency — 16 methods now hold _write_lock
  (closes #68, fixes a production read-during-write race) (#69)
- fix: 3 CI test regressions stabilized (#67)
- chore: test-suite hygiene — 280→305 passing, 17→10 skipped,
  2→0 xfailed; migrated langchain_retriever to Pydantic V2 ConfigDict
  (#62, #63, #64, #65)
- brand: neural-chain architecture diagram + light/dark parity,
  canonical security channels, refreshed social preview (#61)

See CHANGELOG.md for details.

Bumps: pyproject.toml, src/zettelforge/__init__.py, mkdocs.yml,
server.json, SECURITY.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants