Skip to content

feat(engine): shadow-bake model-vs-deterministic cut divergence comparator - #309

Merged
thejefflarson merged 2 commits into
mainfrom
thejefflarson/jef-569-shadow-bake-with-model-vs-deterministic-cut-comparator-then
Aug 1, 2026
Merged

feat(engine): shadow-bake model-vs-deterministic cut divergence comparator#309
thejefflarson merged 2 commits into
mainfrom
thejefflarson/jef-569-shadow-bake-with-model-vs-deterministic-cut-comparator-then

Conversation

@thejefflarson

Copy link
Copy Markdown
Owner

Summary

Re-scoped JEF-569: a view-only comparator that lets the shadow bake measure
whether the ADR-0034 model-chosen cut-set agrees with what pure determinism
alone would have proposed, plus the human-read exit criterion that informs the
first enforce flip.

  • engine::cut_divergence (new module): for each breach-relevant entry with
    a decisive cut-choice decision this pass, classifies the model's cut-set
    against the deterministic fallback set respond::containment_for +
    respond::quarantine_workload_link would themselves have proposed for the
    same chains — agree / model_over_cut / model_under_cut / mixed. A
    fresh Uncertain (or no decision at all) is skipped — no fresh signal, never
    a divergence claim.
  • Journal line: journal::Decision::CutDivergence (durable, restored on
    replay into the new state::DivergenceLog ring, mirroring how a reversion
    restores today).
  • View: GET /api/divergence.json — a read-only, no-store JSON feed on
    the existing dashboard router (same auth/CSP layering as every other
    /api/*.json route). Deliberately not wired into the Preact tab/nav system
    (that's a separate surface with its own likely-parallel ticket); it's a raw
    array, not a strip-enveloped tab.
  • docs/adr/0037-shadow-bake-arm-readiness.md: the exit criterion an
    operator reads before the single ADR-0021 enforce flip at the ADR-0035
    ladder's narrowest rung — bake duration, zero unexplained model_over_cut on
    a clean workload, the ADR-0033 bench threshold, a model_under_cut
    spot-check, continuous coverage. Informs the flip; nothing here or in code
    auto-arms
    (auto-arming is explicitly rejected, same as ADR-0021).
  • Multi-hop fixture: cut_divergence_tests.rs's multi_hop_incident_snapshot
    — entry → pivot (critical CVE) → objective (KEV CVE, mounts the secret) — so
    the comparator is exercised on downstream divergence, not just the entry.

No PROTECTOR_*_ENABLE toggle — on by default in shadow, per CLAUDE.md.
View-never-mutates is compiler-enforced: every function in the path takes
shared references and returns owned data.

Divergence classification + journal/view shape

pub enum DivergenceClass { Agree, ModelOverCut, ModelUnderCut, Mixed }

pub struct DivergenceRecord {
    pub entry: String,
    pub class: DivergenceClass,
    pub model_cuts: Vec<String>,         // node keys the model named this pass
    pub deterministic_cuts: Vec<String>, // what containment_for + quarantine_workload_link would propose
}

Journaled as Decision::CutDivergence { entry, class, model_cuts, deterministic_cuts };
served read-only as a raw JSON array at /api/divergence.json.

Test plan

  • npm --prefix engine/web run build
  • cargo fmt --all -- --check
  • cargo clippy --all-targets -- -D warnings
  • cargo nextest run --workspace — 1133 passed, 2 skipped (pre-existing), 0 failed
  • New tests: cut_divergence_tests.rs (classification: agree / over-cut / under-cut /
    mixed, the multi-hop fixture, decisive-NoAttack under-cut, Uncertain/no-decision
    skip, a purity/idempotence test), state/divergence.rs's ring-eviction + class-count
    tests, a journal round-trip test for Decision::CutDivergence, and a dashboard
    HTTP-level test (GET-only, no-store, raw-array, POST 405s).

Scope notes / risks

  • DECISION NEEDED (flagged, not blocking): wiring the divergence feed into the
    Preact tab/nav system (a real "Divergence" view a human clicks into, vs. today's
    raw JSON feed) is deliberately deferred — sibling tickets are touching
    dashboard/view_model/props and the Preact client in parallel this sprint, and a
    tab addition would touch the same Tab enum / nav / bundle surface. /api/divergence.json
    exists today as the stable contract a follow-up tab can read from with no backend
    change.
  • engine/src/engine/mod.rs is now 949 lines (repo cap is 1000) — still under, but a
    future addition to this orchestrator should split it into a module directory rather
    than grow it further.
  • No engine-level (journal_tests.rs) integration test for divergence-restore-across-restart
    was added (that file is already 657 lines); coverage instead sits at the journal
    unit-test boundary (round-trip) + the state::DivergenceLog ring boundary, the same
    test-boundary split the file's own docs describe for other decision kinds.
  • Touches DashboardState's field list and its ~7 struct-literal construction sites
    (dashboard preview example scenarios + two test-module builders) — each is a single
    additive line, the same mechanical footprint every prior dashboard-view ticket in
    this repo's history has had.

Closes JEF-569.

thejefflarson and others added 2 commits August 1, 2026 02:08
…rator

Adds a view-only comparator (engine::cut_divergence) that, for each
breach-relevant entry with a decisive cut-choice decision, classifies the
model's chosen cut-set against the deterministic fallback set
containment_for + quarantine_workload_link would themselves have proposed
for the same chains (agree / model_over_cut / model_under_cut / mixed).
Journaled per entry (Decision::CutDivergence, durable and replay-restored)
and surfaced read-only at GET /api/divergence.json. On by default in
shadow, no PROTECTOR_*_ENABLE toggle; every function takes shared
references and returns owned data, so computing/journaling/viewing a
classification cannot arm or mutate anything (ADR-0016).

Also adds docs/adr/0037-shadow-bake-arm-readiness.md, the human-read exit
criterion (bake duration, zero unexplained model-over-cut on a clean
workload, the ADR-0033 bench threshold, a model-under-cut spot-check,
continuous coverage) that informs the single ADR-0021 enforce flip at the
ADR-0035 ladder's narrowest rung — the comparator only informs; nothing
here auto-arms.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP
@thejefflarson
thejefflarson enabled auto-merge (squash) August 1, 2026 09:34
@thejefflarson
thejefflarson merged commit a4b7b0b into main Aug 1, 2026
5 checks passed
@thejefflarson
thejefflarson deleted the thejefflarson/jef-569-shadow-bake-with-model-vs-deterministic-cut-comparator-then branch August 1, 2026 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant