docs(readme): add Detection Rules as Memory section#74
Conversation
Follow-up to #70. Documents the Sigma + YARA first-class primitives for README readers who don't click through to the blog post. Covers: what ships, Python API + CLI usage, LLM rule explainer (honestly framed as v1 synchronous, v1.1 async), and upstream-spec references. Brand-voice compliant (no emoji, no hype adjectives, no comparative framing). Placement: between MCP Server section and Integrations section, next to the existing ATHF bridge row. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a new README section that documents Sigma + YARA “detection rules as memory” to make the detection-rule primitives discoverable without the blog post.
Changes:
- Introduces a “Detection Rules as Memory (Sigma + YARA)” section explaining rule ingest and ontology/graph integration
- Adds Python API and CLI usage examples for Sigma/YARA ingest (including
--dry-run) - Adds references to upstream specs and documents explainer behavior + rate limiting
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| ## Detection Rules as Memory (Sigma + YARA) | ||
|
|
||
| Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so a single rule UUID is addressable across both formats. |
There was a problem hiding this comment.
The statement that a “single rule UUID is addressable across both formats” looks inaccurate. In code, Sigma rule IDs come from the Sigma id field (or a sigma_<hash> fallback) while YARA uses CCCS meta.id (or a yara_<hash> fallback), so there isn’t a shared cross-format UUID namespace—only a shared DetectionRule field contract. Consider rephrasing to say both formats share the DetectionRule interface/fields rather than implying the same UUID identifies the same rule in both formats.
| Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so a single rule UUID is addressable across both formats. | |
| Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so both formats share the same rule interface and fields even though each format keeps its own rule identifier. |
|
|
||
| ## Detection Rules as Memory (Sigma + YARA) | ||
|
|
||
| Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so a single rule UUID is addressable across both formats. |
There was a problem hiding this comment.
This paragraph claims tags become graph edges for “tools” and “malware families”, but the current mapping only upgrades Sigma attack.s#### tags to Malware and doesn’t emit a distinct Tool entity type (and YARA tag resolution only emits AttackPattern, Vulnerability, ThreatActor, or YaraTag). To avoid over-promising, consider narrowing the wording to what’s actually emitted today (e.g., ATT&CK techniques, CVEs, ATT&CK group/software IDs, and generic tags).
| Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so a single rule UUID is addressable across both formats. | |
| Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges for currently supported mappings such as MITRE ATT&CK techniques, CVEs, ATT&CK group/software IDs, and generic tags, all resolved against the same ontology as every other note. A shared `DetectionRule` supertype carries `SigmaRule` and `YaraRule` subtypes, so a single rule UUID is addressable across both formats. |
| python -m zettelforge.sigma.ingest rules/ --dry-run | ||
| ``` | ||
|
|
||
| An LLM rule explainer (`zettelforge.detection.explainer.explain`) produces a structured JSON summary -- intent, key fields, evasion notes, false-positive hypotheses -- for any `DetectionRule`. It runs synchronously on demand in v1; async enrichment-queue wiring is v1.1. Rate-limited via `ZETTELFORGE_EXPLAIN_RPM` (default 60 calls/minute). |
There was a problem hiding this comment.
The explainer output described here (“intent, key fields, evasion notes…”) doesn’t match the current RuleExplanation schema in zettelforge.detection.explainer, which returns fields like summary, mechanism, threat_model, false_positive_patterns, related_techniques, and confidence. Suggest updating the README wording to match the actual keys so users don’t build against fields that aren’t produced.
| An LLM rule explainer (`zettelforge.detection.explainer.explain`) produces a structured JSON summary -- intent, key fields, evasion notes, false-positive hypotheses -- for any `DetectionRule`. It runs synchronously on demand in v1; async enrichment-queue wiring is v1.1. Rate-limited via `ZETTELFORGE_EXPLAIN_RPM` (default 60 calls/minute). | |
| An LLM rule explainer (`zettelforge.detection.explainer.explain`) produces a structured JSON summary for any `DetectionRule`, including fields such as `summary`, `mechanism`, `threat_model`, `false_positive_patterns`, `related_techniques`, and `confidence`. It runs synchronously on demand in v1; async enrichment-queue wiring is v1.1. Rate-limited via `ZETTELFORGE_EXPLAIN_RPM` (default 60 calls/minute). |
Detection-rules-as-memory, MCP Registry publication, SQLite concurrency hardening, test-suite hygiene, and brand/docs polish. Highlights: - feat: Sigma + YARA as first-class memory entities with LLM rule explainer (#70) + Detection Rules as Memory README section (#74) - feat: MCP Registry publication (server.json + mcp-name tag) (#75) - fix: SQLite reader concurrency — 16 methods now hold _write_lock (closes #68, fixes a production read-during-write race) (#69) - fix: 3 CI test regressions stabilized (#67) - chore: test-suite hygiene — 280→305 passing, 17→10 skipped, 2→0 xfailed; migrated langchain_retriever to Pydantic V2 ConfigDict (#62, #63, #64, #65) - brand: neural-chain architecture diagram + light/dark parity, canonical security channels, refreshed social preview (#61) See CHANGELOG.md for details. Bumps: pyproject.toml, src/zettelforge/__init__.py, mkdocs.yml, server.json, SECURITY.md.
Detection-rules-as-memory, MCP Registry publication, SQLite concurrency hardening, test-suite hygiene, and brand/docs polish. Highlights: - feat: Sigma + YARA as first-class memory entities with LLM rule explainer (#70) + Detection Rules as Memory README section (#74) - feat: MCP Registry publication (server.json + mcp-name tag) (#75) - fix: SQLite reader concurrency — 16 methods now hold _write_lock (closes #68, fixes a production read-during-write race) (#69) - fix: 3 CI test regressions stabilized (#67) - chore: test-suite hygiene — 280→305 passing, 17→10 skipped, 2→0 xfailed; migrated langchain_retriever to Pydantic V2 ConfigDict (#62, #63, #64, #65) - brand: neural-chain architecture diagram + light/dark parity, canonical security channels, refreshed social preview (#61) See CHANGELOG.md for details. Bumps: pyproject.toml, src/zettelforge/__init__.py, mkdocs.yml, server.json, SECURITY.md.
Follow-up to #70. Adds a README section documenting the Sigma + YARA first-class primitives for readers who don't click through to the blog post.
What's included
DetectionRulesupertype--dry-run)detection/explainer.py:7-9docstring)Voice + scope
## Quick Start/## How It Works/ ATHF bridge voice## MCP Server (Claude Code)and## Integrations(alongside the ATHF bridge row)Related
e6dcfa4)