v0.13.0

TL;DR
RSigma v0.13.0 is the "post-evaluation enrichment, server-side TLS, and field observability" release:

• Post-evaluation enrichment between engine.evaluate() and the sinks: four primitives (template, lookup, http, command), strict detection-vs-correlation kind separation, scope filters, on_error policies, six new Prometheus metrics, and a public register_builtin(name, factory) registry.
• Server-side TLS on the daemon API listener (Axum REST + Prometheus + OTLP/HTTP + OTLP/gRPC sharing one socket via ALPN), gated by the new daemon-tls Cargo feature, with optional mutual TLS and cross-platform cert hot-reload via POST /api/v1/reload.
• Field observability: opt-in --observe-fields on engine daemon and engine eval exposes the gap and broken-coverage signals via four /api/v1/fields/* endpoints and three Prometheus surfaces, sharing a RuleFieldSet + FieldCoverage join primitive across CLI and daemon.
• Detached dynamic sources: declare sources in standalone YAML loaded via --source <file_or_dir>, with a unified DaemonSourceRegistry and a new rsigma rule migrate-sources helper. Pipeline-embedded sources: is visible-deprecated this release.
• Library API: MatchResult and CorrelationResult collapse into a single EvaluationResult (RuleHeader + ResultBody), wire shape preserved. Deprecated CLI aliases are now hidden from rsigma --help. The reserved-but-empty attack subcommand group is removed.
• Dependency bumps: jsonschema 0.46.5, jaq-core / jaq-std 1.x to 3.0 with jaq-json 2.0 (Radically Open Security audit fixes), assert_cmd 2.2.2, plus CI action bumps and two VS Code Dependabot security fixes (@azure/msal-node ^5.2.2, brace-expansion ^5.0.6).

Unknown-field discovery API (#149)

The engine daemon learns to surface two halves of detection coverage live from inside the process: which event fields are not referenced by any loaded rule (gap signal) and which rule fields have never appeared in an event (broken-coverage signal). RSigma owns both rule parsing and event ingestion end-to-end, so this view does not need an external pipeline.

Two new flags on rsigma engine daemon (off by default; zero overhead when not set):

Flag	Default	Purpose
--observe-fields	off	Enable the field observer. When enabled, every event evaluated by the engine task has its dotted field paths recorded.
--observe-fields-max-keys <N>	10000	Hard ceiling on distinct field names. Existing keys keep counting once the cap is hit; new keys are dropped and counted as overflow.

Four new HTTP endpoints.

Method	Path	Description
GET	/api/v1/fields	Snapshot bundling summary + unknown + missing for a one-shot dashboard read.
GET	/api/v1/fields/unknown	Event fields not referenced by any rule. Sorted by descending count.
GET	/api/v1/fields/missing	Rule fields never seen in events. Each entry includes up to 10 rule titles with a truncated flag for fields that span more rules.
DELETE	/api/v1/fields/observer	Clear the observer's counters and return {previous_keys, previous_events}.

Each list endpoint accepts ?limit=N&offset=M (default limit=100, cap 1000) and returns total + next_offset for deterministic pagination. All four return 503 Service Unavailable with {"error":"field observation disabled","hint":"..."} when --observe-fields is not set.

Three new Prometheus surfaces.

Metric	Type	Description
rsigma_fields_observed_total	counter	Total events scanned by the opt-in field observer.
rsigma_fields_observer_unique_keys	gauge	Distinct field names currently tracked.
rsigma_fields_observer_overflow_dropped_total	counter	New-key insert attempts dropped because the observer was at capacity.

The gauges refresh on every /metrics scrape and after every successful /api/v1/fields/* call, so a Prometheus alert on rsigma_fields_observer_overflow_dropped_total fires the moment an operator's --observe-fields-max-keys choice is too low for the deployment.

Shared extraction with rsigma rule fields. The rule-field side of the join lives in a new rsigma_eval::fields module (RuleFieldSet) that both the CLI subcommand and the daemon import. The daemon caches the post-pipeline set on RuntimeEngine via ArcSwap and refreshes it on every successful load_rules(), so the HTTP handlers run lock-free against a stable view even during hot reloads.

Shared join primitive. FieldObservation::coverage(&RuleFieldSet) -> FieldCoverage lives in rsigma-eval and partitions an observation snapshot into the unknown / intersection / missing buckets in one pass. Both the daemon's HTTP handlers and the eval report consume this, so the partition semantics cannot drift across runtimes.

Implementation cost. Default-off; the engine task takes a single ArcSwap load per batch when no observer is attached and skips field iteration entirely. With --observe-fields set, the only added work is one Event::field_keys() walk per parsed event (one String allocation per leaf path, depth-capped at 64; flat formats like KvEvent return Cow::Borrowed) plus a short std::sync::Mutex lock to update counters. Memory is bounded by --observe-fields-max-keys (10k default ≈ a few hundred KB; keys stored as Arc<str> so snapshots refcount-bump rather than copy).

Offline coverage report. rsigma engine eval mirrors the daemon's field-observability surface with three new flags: --observe-fields enables observation; --observe-fields-max-keys <N> (default 10000, validated as NonZeroUsize so 0 is rejected at parse time); --observe-fields-report <PATH> writes the JSON report to a file (defaults to stderr if omitted so detections on stdout stay machine-consumable; clap-requires --observe-fields so the typo case fails fast). The report has the same shape as GET /api/v1/fields, so the same jq queries work against either runtime. To make this possible without coupling engine eval to the daemon Cargo feature, FieldObserver lives in rsigma-eval (which every consumer already links) and uses std::sync::Mutex to keep rsigma-eval dependency-light. rsigma-runtime keeps a pub use rsigma_eval::{FieldObserver, FieldObservation, FieldObservationEntry, FieldCoverage} re-export so existing imports continue to compile unchanged.

Docs. Endpoint reference under "Field observability" in docs/reference/http-api.md; flag rows in docs/cli/engine/daemon.md and docs/cli/engine/eval.md; metric rows in docs/reference/metrics.md; combined daemon/eval workflow in docs/guide/observability.md.

Server-side TLS for the daemon API listener (#128)

The engine daemon API listener now terminates TLS in-process for every protocol that already shares --api-addr: the Axum HTTP REST API (/healthz, /readyz, /metrics, /api/v1/*), OTLP/HTTP on POST /v1/logs, and OTLP/gRPC via LogsService/Export. Operators can drop the sidecar reverse proxy they previously needed for confidentiality, integrity, and agent-to-daemon pinning.

New Cargo feature. daemon-tls on rsigma-cli gates the TLS surface and pulls in rustls (with the aws-lc-rs provider, matching the NATS client TLS path and inheriting upstream FIPS-mode work), tokio-rustls, rustls-pemfile, rustls-pki-types, x509-parser, and hyper/hyper-util. The default build is unchanged.

Six new flags on rsigma engine daemon.

Flag	Env	Default	Purpose
--tls-cert <PATH>	--	--	PEM-encoded leaf certificate (chain). Requires --tls-key.
--tls-key <PATH>	--	--	PEM-encoded private key (PKCS#8, PKCS#1, or SEC1). Requires --tls-cert.
--tls-key-password <PASS>	RSIGMA_TLS_KEY_PASSWORD	--	Password for an encrypted --tls-key. Currently rejected with a clear hint pointing at openssl rsa for offline decryption; reserved for a future release.
--tls-client-ca <PATH>	--	--	PEM bundle of trusted CAs. Enables mutual TLS: clients without a CA-signed cert are rejected during the handshake.
--tls-min-version <1.2|1.3>	--	1.3	Minimum negotiated TLS protocol version.
--allow-plaintext	--	off	Opt-in for plaintext on a non-loopback --api-addr.

Plaintext refusal policy. When daemon-tls is built in, the daemon refuses to start on any non-loopback address unless either --tls-cert/--tls-key or --allow-plaintext is supplied. Loopback (127.0.0.0/8, ::1) always allows plaintext to keep local development friction-free.

Unified serving path. The implementation collapses the previous split between axum::serve (for plaintext non-OTLP) and tonic::transport::Server::serve_with_incoming_shutdown (for OTLP) into a single axum::Router built via tonic::service::Routes::into_axum_router. For TLS, a small custom axum::serve::Listener wraps the TcpListener and performs the tokio-rustls handshake on every accepted connection. ALPN advertises both h2 and http/1.1, so the same socket continues to serve REST + Prometheus + OTLP/HTTP + gRPC after TLS termination.

Cross-platform cert hot-reload. Cert rotation funnels through the daemon's central debounced reload task, which is triggered by POST /api/v1/reload (works on every platform, including Windows), SIGHUP (Unix), or a YAML change picked up by the file watcher. All three paths re-read the certificate and key from disk and atomically swap the active rustls::ServerConfig via Arc<ArcSwap<…>>. Inflight TLS connections are not dropped; failed reloads keep the previous certificate active, bump rsigma_reloads_failed_total, and log an error so a typo in the cert path cannot black-hole the listener. Encrypted-key support and ACME/Let's Encrypt automation are intentionally out of scope; operators rotate cert files (cert-manager, certbot, Vault PKI, ...) and trigger a reload.

Two new Prometheus metrics.

Metric	Type	Description
rsigma_tls_certificate_expiry_seconds	gauge	Seconds until the active TLS server certificate's not_after. Signed: negative once expired. Updated at startup and after every successful reload.
rsigma_tls_active_connections	gauge	Currently active TLS-terminated connections on the API listener. Decrements on connection close (including handshake failure).

A single WARN is logged at startup (and after every successful reload) when the active cert expires within 30 days, so operators can plug the line into existing log-based alerting alongside the longer-horizon Prometheus alert on rsigma_tls_certificate_expiry_seconds.

Docs. Full reference under "TLS termination for the API listener" in docs/reference/security.md; flag table in docs/cli/engine/daemon.md; agent recipes (Grafana Alloy, Vector, Fluent Bit, OpenTelemetry Collector) with tls/mTLS blocks in docs/guide/otlp-integration.md; quick-start note in docs/getting-started/quick-start.md; new row in docs/reference/feature-flags.md; two new alerts in docs/reference/metrics.md.

Deprecated CLI aliases hidden from --help (#125)

The 12 flat top-level CLI aliases (eval, daemon, parse, validate, lint, fields, condition, stdin, convert, list-targets, list-formats, resolve) introduced as visible-deprecated forwarders in v0.12.0 (PR #124) are now hidden from rsigma --help via #[command(hide = true)]. The dispatch arms and the deprecation_warn helper are otherwise unchanged, so:

• Every alias still runs successfully and still prints the migration warning on stderr.
• rsigma <alias> --help is still routable and renders the same flag list as the new grouped form, so scripts that introspect a subcommand keep working.
• rsigma --help now lists only the four noun-led groups (engine, rule, backend, pipeline) plus help.

The warning text was updated from "This alias will be hidden in the next release and removed in v1.0." to "This alias is hidden from --help and will be removed in v1.0." to reflect the new lifecycle stage. Removal at v1.0 is tracked in #126.

Detached dynamic sources (#135)

Dynamic source declarations are decoupled from pipeline YAML files. Sources are now a first-class daemon-level concept declared in standalone YAML files and loaded via the new --source <file_or_dir> flag (repeatable). Both pipelines and enrichers reference sources by source_id as before; the daemon resolves them through a unified DaemonSourceRegistry that enforces collision-error semantics (same ID in two sites is a startup error with both paths in the message).

Pipeline-embedded sources: is deprecated. Existing pipeline files that declare sources: continue to work but emit a tracing::warn! at parse time recommending --source and rsigma rule migrate-sources. The deprecation runs over three releases: visible-deprecated this release, hidden from docs next release (#136), removed at v1.0 (#137).

New subcommand. rsigma rule migrate-sources -p <pipeline-dir> -o <out> extracts every pipeline-embedded sources: block into a standalone file, deduplicating by source ID with collision detection, and rewrites the pipeline files with the sources: block removed. Supports --strategy single (default, one consolidated file) and --strategy per-pipeline.

CLI flag additions. --source-file on rsigma pipeline resolve and --source on rsigma rule validate --resolve-sources so offline tooling can validate pipelines that reference external sources.

API change. GET /api/v1/sources now returns an origin field on each entry (external:<path> or pipeline:<name>) instead of the previous pipeline field.

Post-evaluation enrichment (#134)

The daemon now runs a configurable enrichment pipeline between engine.evaluate() and the sinks. Each detection or correlation gets context (asset owner, IP reputation, identity, GeoIP, KEV flag, runbook URL, ...) injected into its RuleHeader::enrichments map before serialization, so every downstream consumer sees the same structured data without re-fetching it.

New flag. rsigma engine daemon --enrichers <PATH> points at a YAML file with max_concurrent_enrichments: <N> (default 16) plus a list of enricher entries. The file is hot-reloaded on SIGHUP, file-watcher events, and POST /api/v1/reload; a reload that fails validation logs the error and keeps the previous pipeline active, so a typo never silently degrades production to "no enrichment".

Four primitives. Every entry declares a type: from a fixed set, modeled on what Splunk (lookup + rest), Cribl Stream (Lookup + HTTP + Code + Eval), and Vector (enrichment_tables + remap) all converged on:

type	Surface
template	Pure string interpolation. No I/O. Used for runbook URLs and synthetic identifiers.
lookup	Reads a dynamic source (as declared today via pipeline sources:) from the existing Arc<SourceCache> by source_id, with an optional jq / JSONPath / CEL extract to slice the cached value and an optional default for cache miss or no extract match. Zero-network-cost.
http	Per-result reqwest request with template-expanded URL, headers, and optional body. Optional response cache keyed on (method, url, body_hash) with configurable TTL is mandatory in practice for any rate-limited API.
command	Per-result tokio::process::Command invocation with template-expanded argv and environment. Stdout capped at 10 MiB; output parsed as JSON (default) or raw string.

The IRQL-style catalog (enrich_ip_employee, enrich_ip_geoip, enrich_hash_virustotal, enrich_cve_kev, enrich_url_runbook, enrich_ip_passive_dns) ships as field-parametric YAML recipes in docs/guide/enrichers.md, not Rust code. External crates that need a Rust-coded named enricher (bundled data, complex parser, stable contract, non-obvious algorithm) register one via the public register_builtin(name, factory) API.

Strict kind separation. Every enricher declares kind: detection | correlation. The kind drives two checks. At config load time, a kind: detection enricher may only reference ${detection.*} template variables and a kind: correlation enricher may only reference ${correlation.*}; cross-namespace references are rejected with a clear error pointing at the offending field. At runtime, the pipeline skips enrichers whose declared kind does not match the current EvaluationResult body variant before invoking enrich(), so a detection-kind enricher pays no cost on correlation results and vice versa. Available variables are documented in the Kind and template namespaces section of docs/guide/enrichers.md.

scope filtering and on_error policies. Within its declared kind, an enricher can be limited via scope.rules (rule ID or title glob), scope.tags (tag-set intersection with prefix.* wildcards), and scope.levels (severity membership). Axes are AND-ed; an empty axis is not a filter. On failure, on_error selects between skip (drop the enrichment, keep the result), null (inject null), and drop (drop the entire result). The default is skip, so an enrichment outage never silently swallows detections.

Six new Prometheus metrics. All six are pre-registered at startup, so every label triple renders with # HELP and # TYPE lines and zero counts on the first scrape, before any event has fired:

Metric	Labels
rsigma_enrichment_total	enricher_id, kind, status (success/skip/error/timeout/drop)
rsigma_enrichment_duration_seconds	enricher_id, kind
rsigma_enrichment_queue_depth	--
rsigma_enrichment_http_cache_hits_total	enricher_id
rsigma_enrichment_http_cache_misses_total	enricher_id
rsigma_enrichment_http_cache_expirations_total	enricher_id

Filtered (kind- or scope-mismatched) enricher calls do not increment any counter, so cardinality stays bounded by the number of configured enrichers.

Library API. rsigma-runtime exports the Enricher async trait, EnrichmentPipeline, EnricherKind, OnError, Scope, the four primitive types (TemplateEnricher, LookupEnricher, HttpEnricher, CommandEnricher), HttpEnricherClient, HttpResponseCache, OutputFormat, the MetricsHook trait, and the register_builtin(name, factory) registry. Reserved names (template, lookup, http, command) are rejected at registration time; duplicate registrations of the same name are rejected to keep the registry append-only.

Documentation. New docs/guide/enrichers.md (config schema, the four primitives, recipes catalog, promotion criteria, output shape, metrics) and docs/developers/adding-enrichers.md (testing pattern, metrics wiring, naming conventions). docs/cli/engine/daemon.md, docs/library/runtime.md, and docs/reference/metrics.md updated. The crates/rsigma-cli/README.md gains a full enrichment surface section that mirrors the docs-site guide.

New dependencies. humantime and arc-swap in rsigma-cli (humantime for 5s / 1h duration parsing in the YAML; arc-swap for the hot-reload swap), globset and jaq-core / jaq-std / jaq-json in rsigma-runtime (globset for scope.rules / scope.tags patterns; the jaq additions wire the enrichment extract flow through jaq 3.0). wiremock added as a dev-dependency in both crates for HTTP enricher integration tests.

Unified evaluation result type (#132)

MatchResult and CorrelationResult are collapsed into a single EvaluationResult via composition. The five fields shared between detection and correlation today (rule_title, rule_id, level, tags, custom_attributes) move into a new RuleHeader struct along with a new optional enrichments map. Kind-specific fields live in DetectionBody and CorrelationBody, behind a #[serde(untagged)] ResultBody enum.

Wire shape preservation. Both the header and the body flatten into the parent JSON object via #[serde(flatten)], so each NDJSON line remains a single flat object: same field set, same values, same skip_serializing_if behavior. Downstream consumers continue to distinguish detection from correlation by presence of correlation_type (correlation-only). The one cosmetic change is key order on rules with a non-empty custom_attributes map: custom_attributes is now emitted between the rule header fields and the kind-specific body fields rather than after them. JSON objects are unordered per spec, so this is invisible to compliant consumers; the golden snapshot tests at crates/rsigma-eval/tests/wire_shape_golden.rs pin the new ordering for both kinds.

Library API is breaking but pre-1.0. The old MatchResult, CorrelationResult, and the struct shape of ProcessResult { detections, correlations } are replaced by:

• EvaluationResult (the single result type)
• RuleHeader, DetectionBody, CorrelationBody, ResultBody (the composable parts)
• ProcessResult (now a type alias for Vec<EvaluationResult>; detections come first, correlations after, in evaluation order)
• ProcessResultExt extension trait on [EvaluationResult] exposing detections() / correlations() iterators and detection_count() / correlation_count()

Migration on the consumer side:

Before	After
m.rule_title, m.tags, etc.	m.header.rule_title, m.header.tags, ...
m.matched_fields, m.event	m.as_detection().unwrap().matched_fields, m.as_detection().unwrap().event
m.correlation_type, m.group_key, ...	m.as_correlation().unwrap().correlation_type, ...
result.detections.len()	result.detection_count()
result.correlations.iter()	result.correlations()
result.detections[0]	result.detections().next().unwrap()

Internally, the three duplicated for m in &result.detections / for m in &result.correlations loops in the file, stdout, and NATS sinks collapse to one for m in result loop.

A new Criterion bench (crates/rsigma-eval/benches/result_serialize.rs) pins serialize throughput of the new design against a byte-for-byte copy of the old types across four representative inputs; the derived #[serde(flatten)] path is within ±4% of the baseline on every sample.

Drop reserved attack subcommand

The empty attack command group that v0.12.0 reserved as a forward declaration for MITRE ATT&CK tooling is removed. The corresponding Commands::Attack clap variant, the AttackCommands enum, the dispatcher branch, the help-text test assertion, and the "reserved; populated by the upcoming MITRE ATT&CK contributor PR" README line are gone. The CLI now exposes four groups instead of five (engine, rule, backend, pipeline); the attack namespace remains available for a future contributor PR to populate but is no longer reserved ahead of time.

Dependency and security bumps (#145)

Rolls up five open Dependabot PRs and closes two Dependabot security alerts. Rust: jsonschema 0.46.5, assert_cmd 2.2.2 (#141), and jaq-core / jaq-std 1.x to 3.0 with the new jaq-json 2.0 (#142, #143) -- the jaq 3.0 release ships the Radically Open Security audit fixes and a new Loader + Compiler + Ctx API that both apply_jq sites in rsigma-runtime and rsigma-cli are ported to; valid jq expressions in extract: and --jq are unaffected. CI: cargo-deny-action 2.0.18, taiki-e/install-action 2.78.0, zizmor-action 0.5.5 (#144). VS Code extension: top-level npm overrides bump @azure/msal-node to ^5.2.2 (drops the vulnerable uuid 8.x, closes GHSA-w5hq-g745-h8pq, #138) and brace-expansion to ^5.0.6 (closes CVE-2026-45149).

Other changes

• Documentation (PR #131): version references no longer hardcode the current release in the docs site -- rsigma.version now reads from Cargo.toml at build time via the macros plugin, so the docs auto-bump on every release rather than drifting behind. docs/guide/performance-tuning.md gains a "Rule loading at scale" section covering the v0.12.0 single-rebuild batched loaders and amortized O(1) add_rule with verified Criterion numbers at 1K / 10K / 100K rules. The rsigma-parser README intro paragraph's stale lint count (65) was bumped to 66 to match every other authoritative location.
• Enrichment wording: the lookup enricher's startup error message and docs/guide/enrichers.md describe sources as "configured on the daemon" rather than "declared in your pipeline sources: block" so the copy stays accurate after a forthcoming release lets sources be declared independently of pipelines.
• README and home page: Detection Engineering Weekly #157 added to the "featured in" list (README.md and docs/index.md) with a quote calling out RSigma's dynamic-pipelines model.
• Contributing guidelines: the docs/ MkDocs site is now listed as a release deliverable in CONTRIBUTING.md alongside the crate READMEs, with a page-to-change matrix that maps each kind of change (new CLI flag, new daemon config key, new library API, new metric, new feature flag) to the page that must stay in sync.

v0.12.0...v0.13.0