feat(ctem): RFC-012 + de-synthesize BAS (Tier-1 Phase 0 — honesty) - #270
Merged
Conversation
The attack-simulation engine reported a passing security control ('detected')
whenever an operator merely configured a detection_source string — no technique
was ever executed. That manufactures false security assurance.
Phase 0 (honesty): the synthetic executeSimulationTechnique now flags every run
as an unverified simulation — output carries verified:false,
execution_mode:'simulated', and a disclaimer — and the detection/prevention
text no longer claims 'Validated against X' / 'detected by security controls'.
detection_validated is false even when a source is configured (config presence
is an expectation, not proof). dry_run no longer returns a fake 'detected'.
RFC-012 documents the real path: reuse the shipped RFC-011 validation dispatch
(agent safe-check executor + completion hook) to run real techniques async and
correlate evidence via validation_evidence.simulation_run_id (already plumbed
through Ingest, passed nil today). Phase 1 = real safe-check dispatch, api-only.
No schema/UI-contract change in Phase 0; the misleading claim is removed, not
the feature. Index updated (docs/rfcs/README.md).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make Validation honest, and design the real path (Tier-1 Phase 0)
The CTEM maturity assessment scored Validation 2.1/5 — the weakest phase — because BAS is synthetic.
executeSimulationTechniquereported a passing security control (detected) whenever an operator merely typed a SIEM/EDR name into a config field. No technique is executed. That's worse than a missing feature: it manufactures false 'your controls caught this attack' assurance.This PR
RFC-012 (
docs/rfcs/RFC-012-real-bas-execution.md) — the design of record for real BAS execution: reuse the shipped RFC-011 validation rails (agent safe-check executor +validatecommand dispatcher + completion hook) to run real techniques async, and correlate evidence viavalidation_evidence.simulation_run_id(already plumbed throughIngest, passedniltoday). Phased rollout; Phase 1 is api-only.Phase 0 — honesty (code): the synthetic path now
verified:false,execution_mode:"simulated", with a disclaimer;detection_validated:falseeven when a source is configured (config presence ≠ proof);dry_runno longer returns a fakedetected.No schema or UI-contract change — the misleading claim is removed, not the feature.
Why phased (not the full async dispatch now)
Real execution flips
RunSimulationsync→async and touches the shared command-completion path that the live proof-of-fix (#266) depends on. That deserves its own focused PR against the RFC spec, not a rushed change bundled with a docs/honesty fix.Tests
"Validated against Splunk"is gone.detection_validated:false.go vet ./..., lint (new-from develop), compliance suite green.