Skip to content

test: SigmaHQ corpus regression for dynamic pipelines#93

Merged
mostafa merged 1 commit into
mainfrom
test/corpus-dynamic-pipelines
May 7, 2026
Merged

test: SigmaHQ corpus regression for dynamic pipelines#93
mostafa merged 1 commit into
mainfrom
test/corpus-dynamic-pipelines

Conversation

@mostafa
Copy link
Copy Markdown
Member

@mostafa mostafa commented May 7, 2026

Summary

  • Add 5 dynamic pipeline fixture files with file-based sources (no network required in CI) covering: field mapping extraction, allowlists, multi-format sources (JSON/YAML/CSV/lines), all three extract languages (jq/JSONPath/CEL), and include expansion.
  • Extend the sigma-corpus CI job with two new regression steps:
    1. Validate the full SigmaHQ ruleset with all dynamic pipelines applied (--resolve-sources)
    2. Golden file comparison: run rsigma resolve on each fixture and diff against committed expected output
  • Source data files and golden output committed for reproducibility.

Fixture coverage

Pipeline Sources Extract Formats
field_mapping 1 file jq JSON
allowlist 2 files jq JSON, lines
multi_format 4 files none JSON, YAML, CSV, lines
extract_languages 3 files jq, JSONPath, CEL JSON
include_expansion 1 file none JSON (transformation array)

Test plan

  • rsigma validate with all 5 dynamic pipelines passes locally
  • rsigma resolve golden file comparison passes locally (all 5 match)
  • CI sigma-corpus job passes with new steps

Add 5 dynamic pipeline fixtures with file-based sources (CI-safe, no
network required) covering field mapping, allowlists, multi-format
sources (JSON/YAML/CSV/lines), extract languages (jq/JSONPath/CEL),
and include expansion.

Extend the sigma-corpus CI job with two new steps:
- Validate the full SigmaHQ corpus with all dynamic pipelines applied
  (--resolve-sources), verifying no crashes or errors
- Golden file comparison: run rsigma resolve on each fixture and diff
  against committed expected output
@mostafa mostafa merged commit 77735d4 into main May 7, 2026
12 checks passed
@mostafa mostafa deleted the test/corpus-dynamic-pipelines branch May 7, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant