Skip to content

feat(eval): built-in cloud/SaaS/container schema signatures - #318

Merged
mostafa merged 5 commits into
mainfrom
feat/cloud-schema-signatures
Jul 16, 2026
Merged

feat(eval): built-in cloud/SaaS/container schema signatures#318
mostafa merged 5 commits into
mainfrom
feat/cloud-schema-signatures

Conversation

@mostafa

@mostafa mostafa commented Jul 16, 2026

Copy link
Copy Markdown
Member

Summary

Adds always-on, built-in schema signatures so rsigma recognizes common cloud, SaaS, and container event shapes out of the box and derives a SigmaHQ taxonomy-compatible logsource for conflict-based rule pruning.

Sources recognized:

  • AWS CloudTrail (aws_cloudtrail) and VPC Flow Logs (aws_vpcflow)
  • Azure Activity, SignIn, and Audit logs (azure_activitylogs, azure_signinlogs, azure_auditlogs)
  • GCP Cloud Audit (gcp_audit)
  • Microsoft 365 unified audit log (m365_audit)
  • GitHub Audit (github_audit)
  • Okta System Log (okta_system_log) and OneLogin (onelogin_events)
  • Kubernetes audit (k8s_audit), Docker events (docker_events), and osquery (osquery_result)

Each signature uses multi-field markers for high specificity with no misfires, and maps to an implied product/service logsource. Off-taxonomy sources (Kubernetes, Docker, osquery) carry logsource.custom dimensions instead of inventing product values SigmaHQ does not use, and VPC Flow Logs ships as product: aws plus custom: {source: vpcflow}.

Details

  • Signatures live in builtin_signatures() with the implied logsource in builtin_schema_logsource() and a specificity-ordered builtin_schema_names(), guarded by a consistency test.
  • A gcp_audit.yml pipeline strips the data. prefix that SigmaHQ's gcp.audit rules use (data.protoPayload.*) so they match native Cloud Logging events (protoPayload.*). An end-to-end test proves a native event matches only after the pipeline runs.
  • The Microsoft 365 signature targets the Office 365 Management Activity common schema and maps to service: audit, where SigmaHQ's native-field rules live. SigmaHQ's exchange, threat_detection, and threat_management services are written against a separately normalized shape (eventSource/eventName/status) that is not part of the Management Activity common schema, so raw events are intentionally not classified into those services.

Tests

  • Per-source golden fixtures (test_event.json + expected_classification.yaml) with a walk test asserting the resolved schema name, specificity, and implied logsource, plus a check that every fixture maps to a known schema.
  • Specificity-ordering and no-shadowing unit tests.
  • End-to-end GCP routing test.

cargo test -p rsigma-eval, cargo clippy --all-targets --all-features -- -D warnings, cargo fmt --check, and docmd validate all pass.

Docs

  • New cloud collection recipes guide with Vector, OTel, and Fluent Bit configs per source.
  • Updated schema signature reference and schema routing tables.

mostafa added 5 commits July 16, 2026 17:47
Add always-on signatures for AWS CloudTrail and VPC Flow Logs, Azure
Activity/SignIn/Audit logs, GCP Cloud Audit, the Microsoft 365 unified
audit log, GitHub Audit, Okta System Log, OneLogin, Kubernetes audit,
Docker events, and osquery. Each signature uses multi-field markers for
high specificity and implies a SigmaHQ taxonomy-compatible product and
service logsource for conflict-based pruning, with off-taxonomy sources
carried as logsource.custom dimensions.
Add per-source fixtures (test_event.json + expected_classification.yaml)
and a walk test that classifies each event and asserts the resolved
schema name, specificity, and implied logsource, plus a check that every
fixture maps to a known built-in schema.
SigmaHQ's gcp.audit rules reference fields under the data. prefix
(data.protoPayload.*), while native Cloud Logging events use bare
protoPayload.*. Strip the prefix via field_name_prefix_mapping so the
rules match native events, and add an end-to-end test proving a native
event matches only after the pipeline runs.
Document the built-in signatures and their implied logsource, the
schema-derived routing tables, and per-source collector recipes (Vector,
OTel, Fluent Bit). Add the CHANGELOG entry for the bundle.
@mostafa
mostafa force-pushed the feat/cloud-schema-signatures branch from b249bcc to 9600a25 Compare July 16, 2026 15:52
@mostafa
mostafa merged commit ca8b16c into main Jul 16, 2026
15 checks passed
@mostafa
mostafa deleted the feat/cloud-schema-signatures branch July 16, 2026 16:05
@mostafa mostafa mentioned this pull request Jul 22, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant