v0.20.0

TL;DR
RSigma v0.20.0 is the "intermediate representation, reverse conversion, and rstix STIX closure" release: a new HIR crate becomes the compile and convert backbone, SIEM queries convert back to Sigma YAML, rstix gains a TAXII 2.1 client alongside graph, marking, and store modules, and the schema classifier learns the major cloud log sources.

• Intermediate representation: a new rsigma-ir HIR crate with lower_rule, a serializable HIR restart cache, and opt-in optimization passes, with rsigma-eval compile and the rsigma-convert Backend trait now IR-native end to end while backend output stays byte-identical (#360, #362, #363, #364, #368, #370).
• Reverse conversion: a pluggable framework that parses a SIEM query dialect into the IR, raises it to a Sigma rule, and emits canonical YAML, with Elastic Lucene as the reference frontend, exposed through rsigma rule reverse and the MCP reverse_convert tool (#348, #371).
• rstix threat intel: an async TAXII 2.1 client with mTLS, pagination, auth, and resilience (#373); spec-exact STIX 2.1 closure with graph traversal, TLP marking resolution, and an object store (#327); and serde-gated wire-format validator dependencies (#352), thanks to @SecurityEnthusiast.
• Detection routing: a built-in cloud schema signature bundle covering AWS CloudTrail and VPC Flow, Azure, GCP, Microsoft 365, GitHub, Okta, OneLogin, Kubernetes, Docker, and osquery (#318).
• Reliability: deterministic bloom pre-filter seeding, so the per-field substring pre-filter's bit layout is stable across runs and platforms with correctness unchanged.
• Dependencies: two rolled-up Dependabot batches across the Rust workspace, CI actions, and the VS Code extension (#354, #357).

rstix: TAXII 2.1 Client (taxii feature) (#373)

• taxii — async HTTP client for TAXII 2.1 endpoint groups (discovery, API Root, collections, objects, manifest, versions, status). Channels (§6) are RESERVED in the spec and not implemented. Includes DNS SRV via _taxii2._tcp and TaxiiClientConfig::dns_nameserver() for custom resolvers.
• Wire format — TaxiiEnvelope for object exchange (not Bundle); Accept / Content-Type: application/taxii+json;version=2.1 on every request; manifest requests also accept STIX JSON; trailing-slash URL rules; HTTPS enforced by default (allow_insecure_http for tests); max_content_length enforced before POST.
• Pagination — TaxiiPaged<T> returns page bodies plus X-TAXII-Date-Added-First / Last headers; cursor + header fallback on objects, manifest, object-by-id, and versions streams.
• Auth — BearerAuth, BasicAuth, ApiKeyHeader via TaxiiAuthProvider; secrets use secrecy::SecretString with redacted Debug.
• Resilience — RetryPolicy (503/429/network with backoff, honoring Retry-After on 503/429); full TaxiiError mapping including HTTP 415; optional PreflightPolicy for client-side permission guards.
• mTLS — ClientCertificate::from_pem and ClientCertificate::from_pkcs12_der (pure-Rust PKCS#12 via p12-keystore) embedded in build_rustls_config; single rustls TLS stack.
• Live harness fixes — DANE TLSA prefetch honors dns_nameserver().
• Tests — cargo test -p rstix --features taxii --test taxii_client. Optional live harness: tests/taxii-live/README.md (TLS, mTLS, PKCS#12 mTLS, SRV, TLS 1.3, DANE).
• Conformance — TLS 1.2+, required success Content-Type, strict pagination headers, RFC 2782 SRV weighted selection, HTTP 416 stream recovery, HTTP Date clock skew for added_after, WWW-Authenticate parsing, API Root/collection capability checks, POST auto-poll (PostSubmitPolicy), optional SPKI pinning and DANE (ServerTrustPolicy), TaxiiDiscovery::default_api_root(), optional StatusDetail.version.

Reverse conversion: SIEM queries to Sigma YAML (#348, #371)

A pluggable reverse-conversion framework, the mirror of the forward Backend engine: a query dialect is parsed into the intermediate representation, raised to a Sigma rule, and emitted as YAML. Elastic Lucene ships as the reference frontend.

• rsigma-parser — new emit_rule_yaml / emit_collection_yaml, the inverse of parse_sigma_yaml. The emitter is a deterministic canonical form (named detections, logsource custom fields, and custom attributes are sorted), reconstructs field|modifier keys, escapes literal wildcards in values while leaving re/cidr/fieldref values raw, and renders the condition from ConditionExpr. An empty logsource is emitted as {} so it re-parses.
• rsigma-ir — new raise_rule (IrRule to SigmaRule), the inverse of lower_rule: it reconstructs the modifiers each IrMatcher variant implies, collapses homogeneous value lists, keeps conditions selector-preserving, and rejects numeric dynamic-source references. The canonical ir_pattern_to_sigma moved here (re-exported from rsigma-convert).
• rsigma-convert — new reverse module: a Frontend trait plus a QueryDialect table drive a shared tokenizer and precedence-climbing boolean parser, and assemble_rule turns a boolean tree of leaves into named Sigma selections and a condition (AND-merged selections, same-field OR value lists, negated branches as filters). reverse_collection converts a batch, collecting per-query errors. The LuceneFrontend parses the Lucene / Elasticsearch query_string subset (field:value with wildcards, quoted phrases, /regex/, [a TO b] / {a TO b} ranges, comparison shorthand, field:(a OR b) groups, _exists_, keyword terms, and AND/OR/NOT with grouping) and rejects boosting, fuzzy/proximity, and non-numeric ranges with a structured error.
• rsigma CLI — new rsigma rule reverse --from <dialect> (a single entry point for every dialect, mirroring backend convert --target) reads queries from an inline argument, from --file (repeatable, files or directories: each file is one query, a directory contributes every query file it holds), or from stdin, takes --title/--id/--level/--status and --logsource-* hints a query cannot carry, and prints Sigma YAML (or writes it with -o; a directory -o receives one <name>.yml per query, batch runs title each rule from its file name). Each emitted rule is parsed back before printing, so a rule that would not round-trip never reaches the operator.
• rsigma-mcp — new reverse_convert tool (with a dialect parameter) exposes reverse conversion over MCP, returning the drafted YAML or an error envelope for inexpressible constructs (13 tools total).

Deterministic bloom pre-filter seeding

The engine's per-field substring bloom pre-filter seeded its hasher from ahash's runtime RNG, so its exact bit layout, and therefore which non-matching values landed as false positives, varied across process runs (surfacing as a flaky bloom_index unit test). The filter now uses fixed seeds and is deterministic across runs and platforms. Correctness is unchanged: the bloom only ever over-approximates, never producing a false negative, regardless of seed.

Intermediate representation crate and IR-backed compile (#360, #362, #363, #364, #368, #370)

• rsigma-ir — new sync-only crate with HIR types (IrRule, IrDetection, IrMatcher, IrCondition, IrCorrelation, IrFilter) and lower_rule / lower_* that resolve modifiers before eval. Quantified selectors are preserved so evaluation stays count-based. The matcher model is faithful and lossless: string matches keep a wildcard-aware, original-case IrPattern and encoding modifiers stay explicit as IrEncoding steps, so lowering never lowercases, compiles regexes, or expands encodings.
• rsigma-eval — compile_rule routes through lower_rule → compile_to_compiled, which performs the physical work (lowercasing, regex/aho-corasick, encoding expansion). Public physical API (CompiledRule, evaluate_rule, Engine) is unchanged.
• rsigma-ir HIR cache — all HIR types derive serde::{Serialize, Deserialize} (and the embedded rsigma-parser types gained Deserialize), so a lowered rule set round-trips. A new cache module serializes a slice of IrRule to a versioned, self-describing blob (a HirCacheHeader with HIR_SCHEMA_VERSION plus the rules) via encode_rules / decode_rules, with to_json for a debug export. decode_rules version-checks the header before decoding the rules. CBOR is the binary format because the embedded LogSource's #[serde(flatten)] map has an unknown length that fixed-layout encoders reject.
• rsigma-eval restart cache — Engine::save_hir / Engine::load_hir persist and reload the engine's lowered rules through the HIR cache, so a warm start skips parse, pipeline, and lowering. The engine retains the post-pipeline HIR for rules added via the parsed-rule paths; filter injections and precompiled rules are not captured, so re-apply filters after load_hir.
• rsigma-ir — opt-in, semantics-preserving HIR optimization passes in optimize: flatten_condition (merge nested same-kind boolean groups, collapse Not(Not), unwrap single-child groups, drop idempotent duplicate siblings), eliminate_dead_detections (prune detections no condition can reference, honoring them/glob selector patterns and recursing into Conditional), and common_subexpressions (a non-mutating report of repeated detection items). optimize_rule runs the structural passes in order. The passes are not run by the default eval or convert paths, so compiled-matcher behavior and byte-identical backend output are unchanged; a differential test confirms each pass preserves the match decision and the set of matched selections and fields.
• rsigma-convert — the Backend trait is IR-native end to end. A rule is lowered to IrRule once and both detection and condition walks run over the HIR (convert_rule_via_ir → convert_ir_detection / convert_ir_detection_item). Value leaves take the faithful HIR (convert_field_str over IrStrOp + wildcard-aware IrPattern, convert_field_regex with RegexFlags, convert_field_compare_op with CompareOp, convert_keyword_str / convert_keyword_num) instead of SigmaString / Modifier, so a backend never touches rsigma-parser to emit a value match. PostgreSQL, LynxDB, and Fibratus golden outputs are byte-identical.

Dependency bumps (#354, #357)

Rolls up the open Dependabot PRs into a single merge. Rust (workspace Cargo.lock and fuzz/Cargo.lock): bytes 1.12.0 to 1.12.1 (#322), regex 1.12.4 to 1.13.1 (#326, #319), jsonschema 0.46.9 to 0.48.1 (#325), rmcp 2.1.0 to 2.2.0 (#323) with a co-required sse-stream 0.2.3 to 0.2.4 bump (rmcp 2.2.0 calls the renamed SseStream::from_bytes_stream API), and the patch-updates group (#356) daachorse 3.0.2 to 3.0.3 plus uuid 1.23.4 to 1.23.5. rsigma-parser Cargo.toml pins for yamlpath/yamlpatch move from 1.25 to 1.26 (lockfile already on 1.26.1; yamlpath 1.27.0 is held back until yamlpatch publishes a matching release). CI (all repinned by commit SHA, batched via the actions-updates group, #324, #355): taiki-e/install-action v2.82.8 to v2.83.2, github/codeql-action/upload-sarif v4.36.3 to v4.37.0, and actions/setup-node v4.4.0 to v6.4.0. VS Code extension: typescript 6.0.3 to 7.0.2 (#321) and @types/node 26.1.0 to 26.1.1 (#320). Held back: rusqlite 0.39 to 0.40.1 (#234) on MSRV 1.88.

rstix: serde-gate wire-format validator dependencies (#352)

• Wire MUST at parse (DD-DM-001) — STIX §6.4 / §6.5 / §6.15 require well-formed domain-name, email-addr, and url values; rstix rejects malformed values at default serde parse rather than accepting them and reporting later. Documented in crates/rstix/README.md.
• idna, url, email_address, base64, encoding_rs — optional deps enabled by the serde feature (base64 also under pattern); --no-default-features builds omit them. Default serde parse still enforces DD-DM-001 unchanged.
• pattern implies serde — Pattern Engine evaluation uses Bundle, StixObject, and typed SCO/SDO types that live behind the serde feature; --no-default-features --features pattern now compiles. CI adds a lean-feature matrix for --no-default-features on rstix.

rstix STIX 2.1 spec-exact closure with Graph + Marking + Store (graph, marking, store features) (#327)

• Wire MUST at parse (Decision A): domain-name, email-addr, and url values use IDNA / RFC 5322 / RFC 3986 validation at the default serde boundary (idna, email_address, url are required dependencies).
• SCO *_enc (§3.1 / §3.9.1): IANA charset validation and _enc-without-base pairing on spec-defined file.name_enc and directory.path_enc, plus any _enc keys in common.extra; negative fixtures in tests/fixtures/spec/sco/.
• Spec-audit closure: observed-data deprecated objects with embedded SRO; standalone unknown top-level keys via common.extra; granular selector semantics, language-content nested rules, STIX-W0031, and location ISO 3166 / region-ov via Bundle::validate().
• Serialization conventions: wire-facing JSON property bags use BTreeMap (stable key order for strict round-trip and JCS); internal STIX-id indexes use HashMap — aligned with PR #213 review (ExtensionMap, LanguageContent.contents, common.extra).
• Language-content (§7.1.1): recursive object mirroring, "" list placeholders, unknown target fields silently ignored (no advisory).
• Granular markings: selector resolution on custom object wire JSON; custom objects validated in the pipeline schema phase.
• Email-message (§6.6): RFC 2047 encoded-word decoding on ingest for header string fields.
• Encryption algorithm: pipeline property_types check aligned to SHOULD warning (W0010), matching Bundle::validate().

Adds three independent modules for Graph + Marking + Store:

Graph (graph feature):

• StixGraph::from_bundle — indexes SRO relationship edges and all typed _ref / _refs properties (including nested SCO extension refs); duplicate object ids rejected with GraphError::DuplicateObjectId.
• EdgeTraversal — plan-style chain: from(id).out_edges_matching(pred).targets_as::<T>().
• RelationshipExpander — multi-hop expansion from any start node (expand_from) or an IndicatorId (expand_from_indicator); collects identity, infrastructure, indicator, malware, threat-actor, campaign, attack-pattern, course-of-action, and vulnerability summaries.
• SroEdgePayload — typed access to underlying relationship or sighting SRO on graph edges; sighting edges indexed with type "sighting".
• in_refs / incoming SRO traversal — bidirectional graph walking on SRO and inlined ref edges.
• model/ref_paths — path-aware ref inventory shared with bundle ref validation.

Marking (marking feature):

• TlpV2Level — all five predefined TLP 2.0 UUIDs; TLP:AMBER+STRICT vs TLP:AMBER with distinct permits_disclosure.
• MarkingResolver — effective_for_object (most restrictive), effective_for_property / effective_for_selector (granular selectors with JSON path resolution), permits_disclosure(audience), and EffectiveMarking::language_tags for granular language markings.

Store (store feature):

• StixStore trait (object-safe) + MemoryStore with versioned SDO/SRO storage, full-text search index, delete, and export_bundle.
• FsStore (store-fs feature) — filesystem-backed durability with JSON object files under a store root.
• SCO asserted-id preservation — store key is source id; UUIDv5 fingerprint reported via FingerprintConflict in ImportReport.
• StixQuery builder for typed store queries with pagination (QueryCursor / next_cursor), type-indexed scans, text_search, SCO content updates, and StoreError::InvalidQuery for out-of-range cursors.

Cloud Schema Signature Bundle (#318)

Built-in schema signatures cover AWS CloudTrail (aws_cloudtrail), AWS VPC Flow Logs (aws_vpcflow), Azure Activity/SignIn/Audit logs (azure_activitylogs, azure_signinlogs, azure_auditlogs), GCP Cloud Audit (gcp_audit), Microsoft 365 unified audit log (m365_audit), GitHub Audit (github_audit), Okta System Log (okta_system_log), OneLogin (onelogin_events), Kubernetes audit (k8s_audit), Docker events (docker_events), and osquery (osquery_result). Each signature uses multi-field markers (high specificity, no misfires) and implies a SigmaHQ taxonomy-compatible product/service logsource for conflict-based pruning. Off-taxonomy sources (k8s, docker, osquery) ship as logsource.custom dimensions to stay within the no-blessed-vocabulary line, and AWS VPC Flow Logs ships as product: aws plus custom: {source: vpcflow}. A gcp_audit.yml pipeline strips the data. prefix that SigmaHQ's gcp.audit rules use (data.protoPayload.*) so they match native Cloud Logging events (protoPayload.*). Per-source golden fixtures (test_event.json + expected_classification.yaml) with a walk test, an end-to-end GCP routing test, and specificity-ordering unit tests guard correctness and prevent regressions. A new cloud collection recipes guide covers Vector, OTel, and Fluent Bit configs per source.

v0.19.0...v0.20.0