Skip to content

v0.17.0

Choose a tag to compare

@mostafa mostafa released this 23 Jun 10:23
5883af7

TL;DR
RSigma v0.17.0 is the "detection-engineering toolkit" release: the rule-side reporting suite that closes the program loop, plus the daemon output-delivery layer and live daemon introspection.

  • Detection-engineering reports: rule backtest replays an event corpus against a ruleset and diffs per-rule fire counts against declared expectations (#216); rule coverage maps a ruleset onto MITRE ATT&CK, exports a Navigator layer, and reports coverage gaps (#221); rule visibility turns the field-observability signal into a DeTT&CT administration pair and a visibility Navigator layer (#242); rule scorecard fuses backtest precision/recall, coverage, and fire volume into per-rule keep/tune/retire verdicts (#243).
  • Output delivery: detection results now flow through a per-sink async delivery layer with bounded queues, retry/backoff, batching, and an at-least-once ack-join across fan-out (#222); an OTLP output sink exports detections over OTLP/HTTP and OTLP/gRPC (#223); a generic, template-driven webhook sink delivers to Slack, Teams, Discord, PagerDuty, or any HTTP endpoint (#227).
  • Daemon introspection: engine status queries a running daemon from the command line (#237), engine tap records a redactable, replayable live event fixture (#238), and engine tail streams live detections to the terminal (#239).
  • Conversion reach: backend convert resolves targets native-first and delegates anything without a native backend to an installed sigma-cli, reaching the full pySigma backend ecosystem with no new dependency (#241).
  • rstix: Phase 2 adds STIX meta objects (#213) and relationship/sighting objects (#220), thanks to @SecurityEnthusiast; the crate is not releasable on its own yet.
  • Fibratus conversion fixes: emit the required version field so converted rules load (#219), and map file_access/file_event/create_remote_thread to their idiomatic macros (#217), thanks to @rabbitstack.
  • Faster NATS and daemon integration tests: deterministic waits replace fixed sleeps and long-poll timeouts, cutting each suite's runtime by roughly 7x with no production code changes (#240).
  • Security: bump the transitive quinn-proto (via reqwest) to 0.11.15 to clear RUSTSEC-2026-0185, a high-severity remote memory exhaustion advisory.

rule scorecard: fuse the rule-side reports into per-rule keep/tune/retire verdicts (#243)

A new rsigma rule scorecard subcommand fuses the toolkit's existing rule-side outputs into the per-rule keep/tune/retire verdict table a detection program reviews on a cadence. It reads JSON the toolkit already emits, so it adds no new collection or evaluation: it is an offline fusion-and-verdict layer over already-aggregated reports.

  • Inputs and the join. Joins the rule backtest report (precision proxy, recall, the corpus false-positive signal, per-rule fire counts) and the rule coverage report (per-rule ATT&CK mapping and the per-technique rule count for sole-coverage analysis), both required, into a per-rule record keyed by rule_id. Optionally enriches it with a Prometheus production-volume snapshot or /metrics endpoint (--metrics, joined by rule_title with colliding titles summed and flagged), a Prometheus query-API range window (--metrics-window) for last-fired, and a triage disposition feed (--triage) for the live false-positive ratio and MTTD/MTTR. Every cell records which input supplied it, and a missing optional input degrades the verdict rather than blocking it.
  • Verdict model. Bands default to the SOC quality-metrics thresholds and are configurable through flags and the scorecard config section: retire on a precision proxy below the retire floor (0.10) or zero volume across the corpus and the metrics window (a dead rule), tune on the review band or a live false-positive ratio over the ceiling (0.50), keep on a healthy precision proxy (0.80) with enough volume and a recent fire. A retire candidate that is the sole coverage for an ATT&CK technique is downgraded to tune with a coverage-risk note, so the program never silently drops coverage.
  • Output and CI. Renders through the global --output-format layer (table on a TTY, json/ndjson/csv/tsv) plus a --report markdown or HTML program artifact grouped by verdict (extension dispatch, --report-format override). --fail-on <none|tune|retire> turns it into a CI gate. Exit codes follow the house scheme: 0 success or under policy, 1 verdicts hit --fail-on, 2 an input is missing or unfetchable, 3 a bad flag or a malformed/version-mismatched report.
  • Config. A scorecard config section follows the layered-config conventions: the verdict thresholds carry single-source defaults (pinned to the clap flags by a drift-guard test), and every input (including the two required reports, scorecard.backtest/scorecard.coverage) and the report path can be supplied from the config file. Relatedly, rule coverage now also accepts its rule paths from coverage.rules.
  • No new dependencies. The Prometheus exposition-snapshot parser is hand-rolled (the single new untrusted-input surface, fuzzed by fuzz_scorecard_promtext); the query-API path reuses the existing ureq client. The backtest and coverage reports deserialize through structs shared with their producers, so the consumer and producers cannot drift.

rule visibility: DeTT&CT export and a visibility Navigator layer (#242)

A new rsigma rule visibility subcommand turns the shipped field-observability signal into the two artifacts blue teams consume for data-source maturity: a DeTT&CT administration pair and a visibility ATT&CK Navigator layer. Where rule coverage reports the detection axis ("which techniques your rules detect"), rule visibility reports the data axis ("which fields and logsources you actually see"), and the two Navigator layers stack to expose data-without-detection and detection-without-data cells.

  • Inputs and the join. Joins the rule logsource inventory and rule field set (from --rules) with the observed field signal (--observed <file|->: the engine eval --observe-fields JSON, a saved GET /api/v1/fields snapshot, or stdin; or --addr for a live daemon) through a bundled, overridable mapping table (--mapping[=<path|url>]). With no observed signal the command reports the rule-expected baseline.
  • Mapping table. A curated logsource/field -> ATT&CK data source/data component/technique table ships in-repo so the default invocation needs no network; --mapping reads a local JSON table or fetches a URL through the same 7-day cache the lint schema download uses. Rule logsources the table does not recognize are surfaced as a hygiene list.
  • Scoring. Visibility rides DeTT&CT's 0-to-4 scale, derived from the fraction of a data source's mapped rule fields that were observed. A data source whose mapped fields are all unobserved is a blind spot; an observed source no rule consumes is untapped. Scores are conservative seeds marked for analyst review, with data_quality dimensions carrying the seed value rather than fabricated precision.
  • Outputs. Writes a DeTT&CT data-source administration YAML (--dettect-data-sources), a technique-administration YAML (--dettect-techniques, visibility axis only), and a format 4.5 visibility Navigator layer (--navigator, scored 0-4). The report renders through the global --output-format layer (table/json/ndjson/csv/tsv).
  • CI signal. --fail-on-blind-spots exits 1 when a rule-expected data source has no observed telemetry. A visibility config section (mapping, fail_on_blind_spots) follows the layered-config conventions.

Reuse pySigma backends through sigma-cli delegation (#241)

rsigma backend convert now resolves targets native-first: it uses a native rsigma backend when one exists and otherwise delegates the conversion to an external sigma-cli when one is installed, so the full pySigma backend ecosystem (splunk, elasticsearch, kusto, qradar, loki, crowdstrike, and 30+ more) is reachable from the same command. It is a light subprocess wrapper with no new dependencies; no Python runtime is required unless a delegated target is actually used.

  • Native-first dispatch. postgres/postgresql/pg, lynxdb, and fibratus keep converting natively and always win; any other target is delegated. A future native backend transparently supersedes its delegated path.
  • Discovery. sigma-cli is found via the RSIGMA_SIGMA_CLI path override or a bare sigma on PATH. When a target has no native backend and sigma-cli is absent, the command exits 3 with install guidance (pipx install sigma-cli, sigma plugin install <target>).
  • Flag mapping. -t, -f, -p, --without-pipeline, -s, and -O key=value pass through to sigma convert verbatim; -O correlation_method=<m> maps to sigma-cli's -c/--correlation-method. The original rule files are handed to sigma-cli, which parses, pipelines, and converts them.
  • Output. sigma-cli stdout is relayed through the normal output handling (stdout, -o <file>, and the --output-format json envelope). A non-zero sigma-cli exit maps to 2 with its stderr relayed; a missing binary or a directory --output in delegated mode maps to 3.
  • Listing. backend targets appends the installed sigma-cli targets, and backend formats <target> shows a delegated target's formats.
  • Scope. CLI backend convert only; the MCP convert tool and the rsigma_convert library API convert with native backends. rsigma builtin pipeline names (ecs_windows, sysmon) are not translated; pass sigma-cli pipeline names or YAML paths in delegated mode.

Faster NATS and daemon integration tests (#240)

The nats_integration, cli_daemon_nats, and cli_daemon_dynamic suites spent most of their wall time waiting on fixed sleeps and long-poll timeouts rather than doing real work. Replacing those with deterministic waits cuts each suite's runtime by roughly 7x (30.7s to ~2.7s, 15.1s to ~4.4s, and 4.2s to ~1.9s) with no production code changes.

  • Shared-consumer NATS test. The first consumer's messages() pull stream prefetches the whole batch, so the second consumer in the shared group starved and its recv() blocked for the full ~30s pull-consumer expiry, which alone took the entire suite. Each receive is now bounded with a short timeout while still asserting that a consumer in the group receives a message.
  • NATS daemon state tests. The state-restore tests poll the SQLite state DB until the source position is persisted instead of sleeping a fixed 3s, the backward-replay test collects only the message it inspects (it was blocking the full timeout waiting for a second message a clean run never emits), and the no-output check uses an ordered canary detection rather than a fixed wait window.
  • Dynamic-pipeline tests. Event ingestion, /api/v1/reload, and /api/v1/sources/resolve are all asynchronous, so the tests now poll /api/v1/status for the observable counters and re-post events until the rebuilt engine takes effect, replacing the 500ms-to-3s sleeps that previously padded each step.

engine tail: stream live detections to the terminal (#239)

A new rsigma engine tail subcommand (and the GET /api/v1/detections/stream endpoint behind it) streams a running daemon's live detections, the detections-out counterpart to engine tap. Each result is the same EvaluationResult shape the sinks emit, captured after post-evaluation enrichment and regardless of which sinks are configured, so engine tail and a saved sink file are the same format.

  • Server-side filters. --level <severity> (minimum severity) and --rule <substring> (case-insensitive title/id match) are applied at the sink, so filtered-out results never cross the wire. --duration and --limit bound the stream; with neither, it streams until interrupted.
  • Lossy by design. Each session owns a bounded buffer and drops detections (counted) when full, so a slow tail client can never backpressure the sink task or stall the at-least-once ack-join. A dropped client connection tears the session down automatically. A trailing summary record reports {streamed, dropped}.
  • Opt-in. Disabled by default; enable with --enable-tail or daemon.tail.enabled: true. The config-file-only daemon.tail keys tune buffer_events (8192) and max_sessions (2); the endpoint returns 503 when disabled, 409 at the session cap, and 400 for a bad level.
  • Output. Rendered through the global --output-format layer (NDJSON when piped, pretty JSON on a TTY, plus csv/tsv/table). The client uses the synchronous ureq transport, so it builds without the daemon feature.
  • New metrics. rsigma_tail_active_sessions and rsigma_tail_detections_dropped_total.

engine tap: record the live event stream to a replayable fixture (#238)

A new rsigma engine tap subcommand (and the GET /api/v1/tap endpoint behind it) records a bounded window of a running daemon's live event stream as an NDJSON fixture, closing the "reproduce a missed detection locally" loop: capture what the daemon is actually seeing, optionally redact sensitive fields, then replay it against candidate rules with engine eval -e @fixture.ndjson.

  • Two capture stages. --stage decoded (the default) records what the engine evaluated (post-parse, post-event-filter), so the fixture is always valid NDJSON and replays without repeating the daemon's input flags. --stage raw records the input line as received, for debugging the parse/filter step.
  • Server-side redaction. --redact-fields user.email,src_ip redacts named dotted paths before the data leaves the daemon; raw values never cross the wire. Redaction is deterministic per-session salted hashing (rsigma:redacted:<hex>), so equal values map to equal tokens within a capture (correlation cardinality survives replay) while the per-session salt blocks dictionary reversal and cross-fixture linkage. A non-numeric path segment meeting an array fans out to every element (fail-closed).
  • Bounded and lossy by design. The capture can never apply backpressure to detection: each session owns a bounded buffer and drops events (counted) when full. --duration and --limit bound the window, a trailing summary record reports {captured, dropped, duration_ms, stage}, and a dropped client connection tears the session down automatically. The hook rides the same ArcSwap observer pattern as --observe-fields, so the idle hot-path cost is one load per batch.
  • Config and limits. Opt-in: disabled by default (it exfiltrates raw events), enabled with --enable-tap or daemon.tap.enabled: true; the endpoint returns 503 when disabled. The config-file-only daemon.tap keys tune buffer_events (8192), max_sessions (2), and max_duration (5m); the endpoint returns 409 at the session cap and 400 over max_duration.
  • New metrics. rsigma_tap_sessions_total, rsigma_tap_active_sessions, rsigma_tap_events_streamed_total, and rsigma_tap_events_dropped_total.
  • Security. The tap exfiltrates raw events, so it inherits the admin surface's TLS/mTLS protections, ships with a kill switch for hardened deployments, and keeps redacted fields off the wire entirely. The client uses the synchronous ureq transport, so it builds without the daemon feature.

engine status: query a running daemon from the command line (#237)

A new rsigma engine status subcommand fetches a running daemon's /api/v1/status snapshot and renders it through the shared output layer, so checking a daemon no longer requires curl.

  • Address resolution. --addr takes a host:port or full URL and defaults to daemon.api.addr from the resolved config; wildcard binds (0.0.0.0, [::]) map to loopback, and https:// URLs work for TLS deployments. It shares this convention with config reload.
  • Output. Rendered through the global --output-format layer: a TTY-aware default (pretty json on a terminal, ndjson when piped) plus a METRIC | VALUE table view and csv/tsv. The snapshot covers rules loaded, events processed, detections and correlations fired, correlation state entries, uptime, and the dynamic-source summary when configured.
  • No daemon feature required. The command uses the synchronous ureq client, so a build without the daemon feature can still inspect a remote daemon. It exits 3 when the daemon is unreachable or returns an error.

Webhook output sink: deliver detections to Slack, Teams, Discord, PagerDuty, or any HTTP endpoint (#227)

A generic, template-driven webhook sink turns a detection or correlation into a templated HTTP request. It is one configurable sink rather than a set of bespoke integrations: Slack, Microsoft Teams, Discord, and PagerDuty ship as field-parametric YAML recipes in the webhooks guide, while the engine stays service-agnostic.

  • Config. --webhook <FILE_OR_DIR> (repeatable) and the daemon.output.webhooks config key declare webhooks: entries, each with an id, kind: detection | correlation, a url, optional headers/body templates, and optional timeout, retry, rate_limit, scope, and queue_size. Validated at startup with field-scoped errors.
  • Templating. url, header values, and body are rendered per result with the same engine as enrichers (${detection.*}/${correlation.*} plus ${ENV_VAR} for secrets). The body is JSON-string-escaped so an attacker-influenced rule title or event field cannot break the payload.
  • Best-effort by design. Webhooks run as lossy (on_full=drop) leaves on the async delivery layer, so a third-party endpoint never blocks the at-least-once token release for durable sinks; anything undeliverable lands in the --dlq. Connection/timeout errors, 429 (honoring a capped Retry-After), and 5xx retry; other 4xx route straight to the DLQ.
  • Rate limiting and isolation. An optional per-entry token bucket spaces requests; each webhook runs its own bounded queue and worker, so a slow webhook cannot stall other sinks.
  • TLS to internal endpoints. A per-webhook tls: block adds a custom CA bundle (for a relay served by a private CA) and a client cert/key for mutual TLS toward the endpoint. Public endpoints use the system roots with no extra config.
  • Observability. rsigma_webhook_requests_total{webhook_id,outcome} and rsigma_webhook_request_duration_seconds{webhook_id}; queue depth, retries, drops, and DLQ routing read from the shared per-sink series keyed by the webhook id.
  • Egress and secrets. Webhooks use the daemon's egress-filtered HTTP client (--egress-policy); keep secrets in the environment and reference them with ${ENV_VAR}.

OTLP output sink: export detections over OTLP/HTTP and OTLP/gRPC (#223)

The daemon can now emit detection and correlation results to an OpenTelemetry collector, completing OTLP transport symmetry with the existing OTLP receiver.

  • Two transports. --output otlp://host:4317 exports over OTLP/gRPC; --output otlphttp://host:4318 exports over OTLP/HTTP (protobuf, posted to /v1/logs). Append ?compression=gzip for gzip on the wire. Both require a daemon-otlp build.
  • TLS. The otlps:// (gRPC) and otlphttps:// (HTTP) schemes enable TLS, verifying the collector against the bundled public roots by default. ?ca= verifies against a private CA, ?client_cert=/?client_key= enable mutual TLS, and ?tls_domain= overrides the verified server name.
  • Mapping. Each result becomes one OTLP LogRecord under an rsigma resource and instrumentation scope: the Sigma level maps to the OTLP severity (critical to FATAL, high to ERROR, medium to WARN, low to INFO, informational to DEBUG), the rule title is the log body, and the full serialized result is attached as structured attributes.
  • Delivery. The OTLP sink rides the async delivery layer: batched export with bounded retry and backoff, and terminal failures routed to the DLQ.

Async sink delivery layer: per-sink workers, retry/backoff, and isolation (#222)

Detection output now flows through a per-sink delivery layer instead of a single inline sink writer. Each --output sink runs its own bounded queue and worker task, so a slow or flaky network sink no longer immediately head-of-line blocks the others, and transient failures are retried instead of being dropped to the dead-letter queue on the first error.

  • Per-sink workers. Each sink drains its own bounded queue, batches opportunistically, retries with capped exponential backoff, and routes a result to the DLQ only after exhausting retries. Fan-out across --output sinks is isolated up to each sink's queue depth; a slow durable sink eventually applies backpressure, the honest cost of at-least-once.
  • At-least-once preserved. An ack-join releases a source's acknowledgment only once every sink has committed the result (delivered or DLQ-parked), so the NATS at-least-once contract survives fan-out. Results still in a worker queue at shutdown are left unacked and redelivered on restart.
  • Per-sink lossy mode. Append ?on_full=drop to an --output sink to drop results when its queue is full instead of applying backpressure, trading durability for never stalling. The default (?on_full=block) keeps at-least-once for durable sinks.
  • Tunable. --sink-retry-max, --sink-backoff-base-ms, --sink-backoff-max-ms, --sink-batch-max, and --sink-batch-flush-ms (and their daemon.output.* config keys) tune the shared delivery behavior; the per-sink queue depth follows buffer_size.
  • New metrics. rsigma_sink_queue_depth, rsigma_sink_retries_total, rsigma_sink_dropped_total, and rsigma_sink_delivery_failures_total, all labeled by sink.
  • Input-source metric parity. The HTTP (POST /api/v1/events) and OTLP (HTTP and gRPC) push receivers now feed rsigma_input_queue_depth and rsigma_back_pressure_events_total, which previously tracked only the stdin and NATS pull sources.

rule coverage: ATT&CK Navigator export and coverage-gap analysis (#221)

A new rsigma rule coverage subcommand maps a rule set onto MITRE ATT&CK. It reads the attack.* tags off every detection and correlation rule, exports an ATT&CK Navigator layer, and reports coverage gaps against external references, the companion to rule backtest in a detection-as-code pipeline.

  • Navigator export. --navigator <FILE> writes an ATT&CK Navigator layer (format 4.5) scored by rule count per technique, the same "score function count" semantics SigmaHQ uses, so a rsigma layer overlays cleanly on the SigmaHQ baseline. Sub-technique scores are kept exact.
  • Cross-references. --atomics diffs against the Atomic Red Team index (techniques with atomics but no rule, and rules whose technique has no atomic), --baseline diffs against a baseline Navigator layer (the SigmaHQ heatmap by default), and --targets diffs against a plain-text technique list. Bare --atomics/--baseline fetch their upstream defaults over HTTP with a 7-day on-disk cache and stale-cache fallback; both also accept a local path or an atomic-red-team atomics/ directory.
  • Sub-technique roll-up. A rule on attack.t1059.001 covers a T1059 target (reported as covered_via_subtechnique); a parent rule does not vouch for a specific sub-technique target.
  • Output and exit codes. The report renders through the shared output layer (table, json, ndjson/csv/tsv per-technique rows). --fail-on-gaps exits 1 when any requested cross-reference reports uncovered techniques; 2 for unreadable rules, 3 for an unfetchable cross-reference input.
  • Config. A coverage section (atomics, baseline, targets, fail_on_gaps) flows through rsigma config init/validate/show/schema, the RSIGMA_COVERAGE__* environment layer, --config, and --dry-run.
  • Internal. The multi-path rule loader shared with backend convert moved into a crate-level helper so the two commands cannot drift on rule loading.

rstix: Phase 2 — STIX relationship and sighting objects (#220)

Phase 2 adds typed STIX relationship and sighting objects (not releasable on its own until StixObject dispatch and Bundle parsing land).

  • model::sro: Relationship (STIX §5.1 — common properties via SdoSroCommonProps plus relationship_type, source_ref, target_ref, optional description, start_time, stop_time; RelSourceRef / RelTargetRef type aliases; charset and stop_time later than start_time enforced at deserialize), Sighting (STIX §5.2 — common properties plus sighting-specific fields including description, first_seen, last_seen, count, summary, sighting_of_ref, ObservedDataId for observed_data_refs, and WhereSightedRef for identity or location in where_sighted_refs; SightingOfRef type alias; Sighting::COUNT_MAX; count range and last_seenfirst_seen enforced at deserialize), and the SroObject enum (#[non_exhaustive]). Reference-target rules for source_ref, target_ref, and sighting_of_ref are documented in rustdoc and deferred until bundle-level typed parsing.
  • Deserialize: model::type_check hoisted from model::meta for shared "type" validation; each SRO type rejects mismatched JSON "type" in a single serde pass (ModelError::UnexpectedObjectType); no intermediate serde_json::Value parse. Leaf SRO and model::meta types require JSON "type" — a missing "type" field is a serde error, not silently defaulted.
  • QueryableStixObject: QueryValue::Id added; get_field exposes reference fields on SRO and meta types (for example source_ref, sighting_of_ref, created_by_ref).
  • Tests: roundtrip_strict minimal and rich fixtures under tests/fixtures/spec/sro/; negative fixtures for relationship type charset, time ordering, sighting count range, where_sighted_refs typing, cross-type "type" rejects, and missing "type"; unit coverage for SroObject and MetaObject QueryableStixObject delegation.
  • Docs: SRO invariant decisions in crates/rstix/README.md and docs/library/rstix.md.

Fibratus conversion: emit the required version field (#219)

Fibratus rules require a top-level version attribute (the rule content version, distinct from min-engine-version); the loader rejects a rule that omits it. The converted YAML envelope never emitted it, so every converted rule failed to load. Reported by @rabbitstack.

  • The envelope now emits version: right after id for both detection and correlation rules, regardless of emit_metadata. It defaults to 1.0.0 and is overridable with -O version=<value>.

Fibratus conversion: file and remote-thread macro fixes (#217)

Three Fibratus conversion bugs reported by @rabbitstack are fixed, so the converted rules now use the idiomatic macros the upstream loader expects instead of raw or unmapped predicates.

  • file_access rules now map to the open_file macro. The fibratus_windows pipeline had no file_access handler, so file open rules (Microsoft-Windows-Kernel-File ETW provider) emitted the raw Sigma fields FileName/Image with no event scope, which the loader rejects. The pipeline now renames FileName -> file.path and Image -> ps.exe and injects the open_file discriminator triple (evt.name = 'CreateFile' and file.operation = 'OPEN' and file.status = 'Success').
  • file_event rules now collapse to the create_file macro. The disposition guard was appended after the rule body and lacked the success-status clause, so the run never matched the macro and left a raw evt.name = 'CreateFile' ... and not (file.operation ~= 'OPEN') body. The full create_file triple is now injected contiguously and in macro order.
  • create_remote_thread rules now use the create_remote_thread macro instead of degrading to the bare create_thread. The pipeline injects the cross-process guards (evt.pid != 4, evt.pid != thread.pid) the macro requires.
  • Recognizer. The macro recognizer now accepts the De Morgan negated-equality form (not (field ~= 'x')) as equivalent to a macro's field != 'x' clause, so disposition and cross-process guards injected through the pipeline fold back into their macros. use_macros=false still emits the raw expansion.
  • Pipelines. The add_condition transformation gained an optional field_refs map that injects field-to-field comparisons (lowered through the fieldref modifier) rather than literals.

rule backtest: corpus replay with per-rule expectations (#216)

A new rsigma rule backtest subcommand replays an event corpus against a ruleset and diffs the per-rule fire counts against declared expectations, the per-rule fixture harness that engine eval --fail-on-detection could not provide (that check is corpus-global and passes when any rule fires).

  • Corpus replay. --corpus takes a file or a directory walked recursively, with extension dispatch (.ndjson/.jsonl as NDJSON, .evtx via the evtx feature, everything else through --input-format). Correlation state is reset per corpus file so each file is an independent time slice.
  • Expectations. An optional --expectations YAML asserts per rule (by id or title): at_least, at_most, or exactly, optionally scoped to one corpus file. A rule that fires with no covering expectation is surfaced as a potential false positive, governed by --unexpected (fail/warn/ignore).
  • Reports. The report renders through the shared output layer (table, json, ndjson/csv/tsv per-rule rows) and can be written to --report (JSON) and --junit (a hand-rolled JUnit XML, no new dependency). It carries per-rule fires, a per-corpus-file breakdown, the unexpected-fire set, and a per-logsource false-positive-density rollup.
  • Exit codes follow the house scheme: 0 all expectations met, 1 a failed expectation or a policy-failing unexpected fire, 2 unreadable rules, 3 a bad expectations file or corpus path.
  • Config. A backtest section (rules, corpus, expectations, unexpected, pipelines, and the syslog input knobs) flows through rsigma config init/validate/show/schema, the RSIGMA_BACKTEST__* environment layer, --config, and --dry-run.
  • Internal. The format-aware eval stream loop moved into a shared commands::eval_stream module so engine eval and rule backtest cannot drift on input parsing; eval behavior is unchanged.

rstix: Phase 2 — STIX meta objects (#213)

Phase 2 adds STIX meta objects (not releasable on its own until StixObject dispatch and Bundle parsing land).

  • model::meta: MarkingDefinition (STIX §7.2.1 optional common properties — created_by_ref, external_references, object_marking_refs, granular_markings; legacy TLP 1.x and current TLP 2.0 encodings; IS_NON_VERSIONABLE / is_non_versionable(); nine TLP UUID constants), ExtensionDefinition (created_by_ref required per §7.2.2), LanguageContent (contents as nested BTreeMap for stable JSON key order), and the MetaObject enum (#[non_exhaustive]).
  • Deserialize: each meta type validates JSON "type" against TYPE_NAME in a single serde pass (ModelError::UnexpectedObjectType); no intermediate serde_json::Value parse.
  • Tests: roundtrip_strict for complete types (meta objects, ExternalReference, GranularMarking, ExtensionMap); subset roundtrip for SdoSroCommonProps / ScoCommonProps fixtures that carry unmodeled SDO keys. Fixtures under tests/fixtures/spec/meta/ include minimal TLP markings, a rich marking-def with common properties, and cross-type reject coverage. Unit pins for all nine TLP ids.
  • Docs: STIX object model version vs TLP marking encoding in crates/rstix/README.md and docs/library/rstix.md.

Security: transitive quinn-proto bump for RUSTSEC-2026-0185

cargo audit flagged RUSTSEC-2026-0185, a high-severity (7.5) remote memory exhaustion in quinn-proto from unbounded out-of-order stream reassembly, published 2026-06-22. It reaches the tree transitively through reqwest -> quinn -> quinn-proto. A targeted cargo update -p quinn-proto --precise 0.11.15 moves the workspace and fuzz lockfiles from 0.11.14 to the fixed 0.11.15 with no other dependency changes.

v0.16.0...v0.17.0