[REVIEW] detection-engineering: add Sigma backend conversion evidence gates

[yanziwei]

Review request

The detection-engineering skill requires Sigma rule authoring, validation, and detection-as-code conversion, but it does not require evidence that the converted target-SIEM query preserves the Sigma rule semantics. A Sigma rule can be valid YAML while still failing in Sentinel, Splunk, Elastic, Chronicle, or QRadar because of field mapping drift, unsupported modifiers, changed case sensitivity, or expensive generated queries.

Proposed improvement

Add a backend conversion evidence gate requiring reviewers to capture:

• target backend and converter version
• exact conversion command and result
• field mapping evidence
• unsupported modifier/operator handling
• logic parity after conversion
• known-positive and known-negative sample execution evidence
• query runtime and backend limits

Bounty request

Reviewer tier / USD 25 if accepted.

[yanziwei]

Submitted implementation PR: #1799

[shensz2017]

/attempt

[shensz2017]

VALID GAP - Sigma backend conversion semantics

Why valid

A Sigma rule can pass YAML/schema validation while the deployed backend query is semantically wrong. This is especially common when a converter silently rewrites or drops modifiers such as contains|all, re, cidr, wildcard matching, case-insensitive matching, or list handling. It can also fail when Sigma fields map to the wrong backend field family, for example CommandLine versus an ECS, Sentinel table, Splunk sourcetype, Chronicle UDM, or QRadar property.

Concrete failure mode: a rule that detects a PowerShell command through a Sigma CommandLine|contains|all selection can convert into a syntactically valid query that misses known-positive events because one token becomes case-sensitive, a field mapping points at an unpopulated field, or a backend does not preserve the same grouping/negation semantics. That creates a false sense of deployment readiness even though the detection will not fire.

Review of PR #1799

The implementation is directionally correct and placed in the right lifecycle point: after detection-as-code conversion but before declaring the rule stable or deployment-ready. It also updates the report output and adds a pitfall entry, which means the requirement is visible both during execution and in final reviewer evidence.

Acceptance criteria I would expect

• Each target backend records the converter name/version, backend config, exact command, generated query, and warning/error output.
• Field mapping evidence distinguishes complete, partial, and missing mappings, including data model/table/index requirements.
• Unsupported modifiers/operators are explicitly listed and drive the deployment decision instead of being treated as a generic warning.
• Logic parity covers selections, filters, negation, grouping, and any correlation/window behavior that may change during conversion.
• Sample execution includes both known-positive and known-negative evidence, preferably with fixture IDs or raw-event references.
• Query performance evidence includes runtime, scanned volume, timeout/row-limit risk, and backend-specific constraints.

Suggested tightening

The PR already mentions whether conversion succeeded without manual edits, but the output table does not capture manual post-processing. I would add a Manual Edits/Post-processing column or fold it into Conversion Result, because locally patched queries are a major source of drift between Sigma source and deployed SIEM content.

I would also make Sample Execution require both TP and TN status explicitly rather than a single free-form pass value, so reviewers cannot mark the conversion ready after only proving that the query fires on one positive sample.