refactor(eval): unify MatchResult and CorrelationResult into a single EvaluationResult#132
Merged
Merged
Conversation
…onResult Collapse the two parallel result types into a single `EvaluationResult` built from a shared `RuleHeader` plus an `#[serde(untagged)]` `ResultBody` enum (`Detection` | `Correlation`). The five fields shared today (`rule_title`, `rule_id`, `level`, `tags`, `custom_attributes`) move into `RuleHeader`, plus a new optional `enrichments` map. Kind-specific fields live in `DetectionBody` / `CorrelationBody`. The wire NDJSON shape is preserved exactly: both bodies are flattened into the parent JSON object via `#[serde(flatten)]`, no `result_kind` discriminator is added. Downstream consumers continue to disambiguate variants via `correlation_type` presence (today's contract). `ProcessResult` collapses to a `pub type ProcessResult = Vec<EvaluationResult>`. The engine still emits detections first, then correlations, in the same order as today. A new `ProcessResultExt` trait exposes `detections()` / `correlations()` iterators and `detection_count()` / `correlation_count()` methods so call sites stay readable. A new Criterion bench at `crates/rsigma-eval/benches/result_serialize.rs` compares the new design's serialize throughput against a byte-for-byte copy of the old types across four representative inputs (V1 baseline vs V2 derived `#[serde(flatten)]` vs V3 hand-written `Serialize`). V2 lands within +/-4% of V1 on every sample, so the derive path is what ships. The bench also asserts byte-identical wire shape across all three variants on the sample inputs before any timing is collected. Library API is breaking but pre-1.0: `MatchResult` and `CorrelationResult` are replaced by `EvaluationResult`, `ResultBody`, `RuleHeader`, `DetectionBody`, `CorrelationBody`, plus `ProcessResultExt`. The three duplicated `for m in &result.detections / .correlations` loops in the file, stdout, and NATS sinks collapse to one `for m in result`. 1336 workspace tests pass with --all-features; cargo fmt clean; clippy --workspace --all-targets --all-features clean.
- CHANGELOG: Unreleased entry for the unified result type, breaking library API change, consumer migration table, and wire-shape preservation note. - crates/rsigma-eval/README.md: rewrite the "Output Types" section around EvaluationResult, RuleHeader, DetectionBody, CorrelationBody, FieldMatch, EventRef, and the new ProcessResultExt trait. - docs/library/eval.md: update the type table, detection doctest, and correlation example to use the new types and iterators. - README.md and assets/architecture.mmd: replace the old result-types output block with the unified composition. - crates/rsigma-eval/tests/wire_shape_golden.rs: new golden NDJSON snapshot tests pinning the byte-exact serialization for one detection line and one correlation line, plus the downstream disambiguation contract (correlation_type only on correlations, matched_fields only on detections) and the enrichments-None skip behavior.
A non-empty `custom_attributes` map now serializes between the rule header fields and the kind-specific body fields, not at the end of the line as it used to. JSON objects are unordered per spec, so this is invisible to compliant consumers; pin the actual byte ordering with a new golden snapshot so a future change is intentional, not silent. Tighten the wording in the CHANGELOG entry and the result.rs module docstring to record that the field set, values, and skip_serializing_if behavior all match the previous layout, with custom_attributes being the lone position change.
This was referenced May 20, 2026
mostafa
added a commit
that referenced
this pull request
May 21, 2026
Records the shipped surface of PR #134 under `## [Unreleased]`, immediately above the existing `### Unified evaluation result type (#132)` entry. Covers the new `--enrichers <PATH>` flag, the four primitive `type:` values (`template`, `lookup`, `http`, `command`), the strict kind separation and namespace-validated templates, the `scope` filtering axes and `on_error` policies, the six new Prometheus metric families (all pre-registered at startup), the public library API surface (`Enricher`, `EnrichmentPipeline`, the four primitive types, `HttpResponseCache`, `register_builtin`), and the new dependencies pulled in (`humantime`, `arc-swap`, `globset`, `jaq-core`, `jaq-std`, plus `wiremock` as a dev-dep). Style notes: - All repo-relative paths use plain inline code rather than markdown links, matching the rest of the CHANGELOG and avoiding broken cross-links when `docs/release-notes.md` pulls the file in via `include-markdown`. - The `lookup` row notes "as declared today via pipeline `sources:`" so the entry reads accurately for the release that ships PR #134, without committing the wording to pipeline-only declaration forever. Verification: mkdocs build --strict is clean (release-notes.md embeds the CHANGELOG via include-markdown).
mostafa
added a commit
that referenced
this pull request
May 21, 2026
Audit pass over `git log v0.12.0..main` plus this branch's own commits caught four user-visible items that were not yet recorded in the Unreleased section. Adds a small `### Drop reserved 'attack' subcommand` section for the namespace removal (commit 6369ac8, direct to main with no PR), and a new `### Other changes` bulleted list covering: - PR #131 docs cleanup (version macros now self-serve from Cargo.toml, the new "Rule loading at scale" section in the performance-tuning guide, the rsigma-parser README intro count bumped from 65 to 66 to match the rest of the docs). - The option-B wording fix shipped in 778c83e (`lookup` enricher error and docs say "configured on the daemon" instead of "declared in your pipeline `sources:` block"). - Detection Engineering Weekly #157 added to the featured-in list in README and docs/index.md. - CONTRIBUTING.md gains a Documentation section that lists the docs/ MkDocs site as a release deliverable alongside crate READMEs, with a page-to-change matrix. The four items each ship in the same release as #134 post-evaluation enrichment and #132 unified result types, so they fit naturally under the current Unreleased heading. Verification: mkdocs build --strict clean (release-notes.md embeds CHANGELOG.md via include-markdown).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Collapses the two parallel result types (
MatchResultandCorrelationResult) into a singleEvaluationResultbuilt from a sharedRuleHeaderplus an#[serde(untagged)]ResultBodyenum. The five fields that are shared today (rule_title,rule_id,level,tags,custom_attributes) live inRuleHeaderalong with a new optionalenrichmentsmap; the kind-specific fields stay inDetectionBody/CorrelationBody.ProcessResultbecomes aVec<EvaluationResult>(detections first, correlations after, in evaluation order). The three duplicated sink loops infile.rs,stdout.rs, andnats_sink.rscollapse to one. A newProcessResultExtextension trait gives call sites readabledetections()/correlations()/detection_count()/correlation_count()views without forcing pattern matching.Wire shape
NDJSON output keeps the same field set, same values, and same
skip_serializing_ifbehavior. Both the header and the body flatten into the parent JSON object via#[serde(flatten)], so each line stays a single flat object. Downstream consumers continue to distinguish detection from correlation by the presence ofcorrelation_type.One cosmetic change: on rules with a non-empty
custom_attributesmap,custom_attributesis now emitted between the rule header fields and the kind-specific body fields rather than at the end of the line. JSON objects are unordered per spec, so this is invisible to compliant consumers. The new golden snapshot tests atcrates/rsigma-eval/tests/wire_shape_golden.rspin the new ordering for both kinds (with and withoutcustom_attributes/enrichments).Library API (breaking, pre-1.0)
m.rule_title,m.tags, ...m.header.rule_title,m.header.tags, ...m.matched_fields,m.eventm.as_detection().unwrap().matched_fields,.eventm.correlation_type,m.group_key, ...m.as_correlation().unwrap().correlation_type, ...result.detections.len()result.detection_count()result.correlations.iter()result.correlations()result.detections[0]result.detections().next().unwrap()Performance
A new Criterion bench at
crates/rsigma-eval/benches/result_serialize.rscompares the new design's serialize throughput against a byte-for-byte copy of the old flat structs across four representative inputs (small/realistic, detection/correlation). The new design is within +/-4% of the baseline on every sample; the bench's equivalence preflight also asserts byte-identical wire shape across the two layouts on the sample inputs.Test plan
cargo test --workspace --all-features(1341 tests pass)cargo fmt --all -- --checkcleancargo clippy --workspace --all-targets --all-features -- -D warningscleancargo bench --bench result_serialize(within +/-4% of baseline on every sample; equivalence preflight passes)custom_attributescase (crates/rsigma-eval/tests/wire_shape_golden.rs)