Skip to content

feat(rstix): Graph + Marking + Store and STIX 2.1 spec-exact closure - #327

Merged
mostafa merged 2 commits into
timescale:mainfrom
SecurityEnthusiast:feat/rstix-stix21-graph-marking-store
Jul 17, 2026
Merged

feat(rstix): Graph + Marking + Store and STIX 2.1 spec-exact closure#327
mostafa merged 2 commits into
timescale:mainfrom
SecurityEnthusiast:feat/rstix-stix21-graph-marking-store

Conversation

@SecurityEnthusiast

Copy link
Copy Markdown
Contributor

Summary

This PR completes the Graph + Marking + Store vertical slice for rstix and closes the remaining STIX 2.1 spec-audit differentials (36/36 resolved). It adds three independent optional features (graph, marking, store / store-fs), enforces wire MUST rules at the default serde parse boundary (Decision A), aligns the Validation Pipeline with advisory semantics, and refactors shared ref-path inventory out of stix_object.rs.

Graph (graph feature)

  • StixGraph::from_bundle — zero-copy property graph over a parsed Bundle.
  • SRO edges — indexes relationship (source → target) and sighting (sighting id → sighting_of_ref); full payloads via SroEdgePayload / Edge::relationship() / Edge::sighting().
  • Inlined refs — indexes all typed _ref / _refs properties (including nested SCO extension refs) with property paths.
  • TraversalStixGraph::from(id).out_edges_matching(pred).targets_as::<T>(), plus out_refs / in_refs (including dangling targets).
  • RelationshipExpander — multi-hop expansion from any start node or from an IndicatorId; collects identity, infrastructure, indicator, malware, threat-actor, campaign, attack-pattern, course-of-action, and vulnerability summaries.
  • model/ref_paths.rs — path-aware ref inventory shared with bundle ref validation (~350 lines moved from stix_object.rs; collect_internal_refs() delegates here).
  • Tests: tests/graph.rs, fixtures under tests/fixtures/graph/.

Marking (marking feature)

  • TlpV2Level — all five predefined TLP 2.0 UUIDs; TLP:AMBER+STRICT vs TLP:AMBER with distinct permits_disclosure.
  • TlpV1Level — legacy TLP 1.x ingestion for ATT&CK feeds (#[allow(deprecated)] on impl for -D warnings CI).
  • MarkingResolver — indexes marking-definition objects from a bundle:
    • effective_for_object (most restrictive TLP wins)
    • effective_for_property / effective_for_selector (granular selectors with JSON path resolution)
    • permits_disclosure(audience) and EffectiveMarking::language_tags
  • Granular selector resolution reuses model/validate.rs syntax rules and runs on custom object wire JSON when allow_custom is enabled.
  • Tests: tests/marking.rs, fixtures under tests/fixtures/marking/.

Store (store / store-fs features)

  • StixStore trait (object-safe) + MemoryStore:
    • Versioned SDO/SRO storage (append on content change; dedup identical re-upserts)
    • Full-text search index, type-indexed scans, pagination (QueryCursor / next_cursor)
    • delete, export_bundle, StixQuery builder (text_search, modified_after, labels, …)
  • SCO asserted-id preservation — store key is always the source id; UUIDv5 fingerprint reported via FingerprintConflict in ImportReport (never silently rewrites SCO ids).
  • FsStore (store-fs) — durable JSON-on-disk store with atomic writes (temp file + rename).
  • Tests: tests/store.rs, tests/store_fs.rs, fixtures under tests/fixtures/store/.

STIX 2.1 spec-exact closure

Decision A — wire MUST at parse

Required workspace deps: idna, email_address, url, base64, encoding_rs.

Field Enforcement
domain-name.value IDNA (UTS #46) + label rules
email-addr.value RFC 5322 addr-spec
url.value RFC 3986 (http, https, ftp)

SCO *_enc (§3.1 / §3.9.1)

  • Spec-defined typed fields: file.name_enc, directory.path_enc only.
  • validate_sco_string_encoding_pair + validate_extra_enc_pairings for any _enc keys in ScoCommonProps::extra (IANA charset + pairing rules).
  • New vocab/iana_charset.rs closed vocabulary helper.
  • Negative fixtures: file-name-enc-without-name.json, file-name-enc-invalid.json, file-extra-enc-without-base.json, file-with-name-enc.json.

Email-message (§6.6)

  • model/rfc2047.rs — RFC 2047 encoded-word decoding on ingest for subject, message_id, body, received_lines, additional_header_fields.
  • Vendor _enc siblings (e.g. pattern fixture subject_enc) remain in common.extra; get_field reads them for pattern evaluation.

Language-content (§7.1.1)

  • Recursive object mirroring, "" list placeholders, unknown target fields silently ignored (no advisory).
  • Conformance test excludes bundle-language-content-unknown-field.json (spec MUST ignore).

Granular markings / custom objects (§7.2.3 / §11.2)

  • Selector resolution on custom object wire JSON in Validation Pipeline schema phase.
  • Custom objects validated when allow_custom is enabled.

Spec-audit items closed (previously open)

Gap Resolution
Granular selector semantics Bundle::validate() + custom wire JSON
Language-content nested rules Recursive mirroring + list-length checks in Bundle::validate()
Standalone unknown top-level keys common.extra / MarkingDefinition.extra; fixture identity-standalone-extra.json
Observed-data deprecated objects with SRO ObservedDataEmbeddedObject::Sro; fixture observed-data-deprecated-objects-sro.json
Location ISO 3166 / region-ov Bundle::validate() SHOULD warnings
STIX-W0031 TLP v1 Bundle::validate() advisory
Partial SCO *_enc Spec inventory complete; validate_extra_enc_pairings() for common.extra

Validation Pipeline alignment

  • ScoEncWithoutBaseProperty / ScoEncInvalidCharset — property names carried in tagged wire errors; invalid charset maps to E0013 (closed vocab), not I0002.
  • Encryption algorithm — pipeline property_types aligned to SHOULD warning (W0010), matching Bundle::validate().
  • validate_diagnostic_coverage.rs — E0013 case for unknown opinion; encryption → W0010.
  • validate_conformance.rs — language-content unknown-field fixture excluded per §7.1.1.

Serialization conventions

  • Wire-facing JSON property bags: BTreeMap (ExtensionMap, common.extra, LanguageContent.contents, SCO hashes, Bundle.extra_properties() values).
  • Internal id indexes: HashMap (Bundle.id_index, graph adjacency, store buckets, marking resolver).
  • Documented in crate README and docs/content/library/rstix.md.

Documentation

  • crates/rstix/README.md — Graph/Marking/Store sections, wire-format validation, serialization map conventions, expanded testing layout, updated invariant table.
  • docs/content/library/rstix.md — feature status, extensions, serialization conventions, full feature-flag table.
  • docs/content/reference/architecture.md, feature-flags.md — Graph + Marking + Store complete; Validation Pipeline complete.
  • CHANGELOG.md[Unreleased] entry.

Test plan

  • cargo fmt --all -- --check
  • cargo check --workspace --all-targets --all-features --locked
  • cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
  • cargo test --workspace --all-features --locked
  • cargo +1.88.0 check --workspace --all-targets --all-features --locked
  • RUSTDOCFLAGS="-D warnings -D rustdoc::broken-intra-doc-links" cargo doc --workspace --all-features --locked --no-deps
  • cargo test -p rstix --features validate,serde,pattern,graph,marking,store,store-fs --locked
  • Docs: cd docs && npm ci && npm run docs:build && npm run docs:validate
  • WASM build + smoke (if wasmtime installed)
  • MCP smoke
  • cargo llvm-cov --workspace --all-features --locked
  • zizmor --pedantic .github/workflows

Add optional graph, marking, and store features; enforce wire MUST rules at
parse; close remaining spec-audit differentials; refactor shared ref-path
inventory; and parse MITRE ATT&CK bundles with allow_custom (x-mitre-* types
and in-bundle custom ref kind checks).
Signed-off-by: Meghdad Shamsaei <meghdadshamsaei@gmail.com>

@mostafa mostafa left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SecurityEnthusiast Thank you for your contribution! LGTM. 🚀

@mostafa
mostafa merged commit 5953ffb into timescale:main Jul 17, 2026
17 checks passed
@SecurityEnthusiast
SecurityEnthusiast deleted the feat/rstix-stix21-graph-marking-store branch July 17, 2026 10:38
@mostafa mostafa mentioned this pull request Jul 22, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants