feat: per-run analyst validation card for single-technique runs (roadmap #7) - #94
Merged
Conversation
…map #7) After an ad-hoc 'replicant run', the analyst otherwise reverse-engineers the pivot entities, window, and expected rule from a raw JSON manifest, the manual toil the tool sells against. run() now writes a copy-pasteable card beside the manifest (<stem>.card.md) and RunResult.summary() names it. The card is deterministic, derived from the plan's own events, and authors no rule logic (blueprint boundary): its search is a hunt pivot that LOCATES the run's events, not a detection rule. - Ties to #3: a marked run's find-events search keys on flexString1=<run_id>, so the exact events are one search away; an unmarked (loopback/--to-file/--no-send) run pivots on the top src/dst and the emitted window instead. - Ties to #5: the card carries the transferability verdict and note, so it states what a green result does and does not prove, plus the loopback-only delivery caveat. - Names objective, ndr_rule/ndr_uc, held/varied CEF fields, pivot entities, and the emitted window (real under --pace plan). Positive control: dropped the write_validation_card call and the four behavioural card tests went red, restored them green. 1079 tests; gates clean.
…euses DUBAI_TZ From /code-review on PR #94, verified before fixing: 1. (main) The card's window and count came from the uncompressed plan.events, so a --speed run's card claimed the raw timeline as 'real'. run() now builds the card from compress_timeline(plan.events, speed)[:count] - what actually went out - fixing both the --speed window and the overstated count on a graceful stop. New test exercises --speed (via --no-send so it does not wait the plan timeline); positive control observed red against plan.events, green. 2. The card build/write was outside the try/except, so a card-write OSError could fail a send that had already completed. Now wrapped: a secondary artifact logs a warning and leaves card_path None instead of raising. 3. Unmarked pivot query no longer leads with a dangling 'and' when a technique carries no src/dst (joined, not concatenated). 4. Reuse: the card imports DUBAI_TZ from audit.manifest instead of a fifth hard-coded timezone(timedelta(hours=4)). 1080 tests; black/ruff/mypy clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Roadmap 2026-09 item 7, the day-one workflow F1/F2 do not touch. After an ad-hoc
replicant run, the analyst otherwise reverse-engineers the pivot entities, window, and expected rule from a raw JSON manifest — the manual toil the tool sells against.What changed
run()writes a copy-pasteable card beside the manifest (<stem>.card.md);RunResult.summary()names it. Deterministic, derived from the plan's own events, and authors no rule logic (blueprint boundary): its search is a hunt pivot that locates the run's events, not a detection rule.flexString1=<run_id>(the exact events, one search away); an unmarked run (loopback/--to-file/--no-send) pivots on the top src/dst and the emitted window instead. The test asserts the card's run-id search matches the marker the run actually stamped.ndr_rule/ndr_uc, held/varied CEF fields, pivot entities, and the emitted window (the real timeline under--pace plan).Rendered REP-011 card (parser-only) confirmed to name the pivot (src IPs, user, held field
duser), the 30m window, and the full GeoIP-enrichment reason a green result would not prove.Positive control: dropped the
write_validation_cardcall and the four behavioural card tests went red, restored them green. 1079 tests; black/ruff/mypy clean.Web surfacing of the card is a follow-up (CLI-first identity; the card file is written for every run regardless). Decision record:
docs/roadmap-2026-09.md.