test(bench): add route-forwarded-backend fixture to cut-choice bench (ADR-0034) - #314
Merged
thejefflarson merged 1 commit intoAug 1, 2026
Conversation
…(ADR-0034) Adds a route_forwarded_entry_cve CASE to judge_bakeoff_cutchoice.py: a backend reached only via an Ingress route (no direct Service exposure), rendered in the ENTRY position with its own loaded-at-runtime CVE. Expected assessment=attack, contain=only that backend — mirroring entry_only_log4j but with a distinct node identity (checkout-api-7f9d4c8b6d-x2p9k) so it's clearly the route-forwarded case. This validates that the Ingress exposure observer's Exposure::Internet promotion of a route-forwarded backend feeds the T2b cut-choice prompt through the same edge-CVE lane as a directly-exposed entry. Confirmed locally: the script imports and CASES/render() build the prompt without error (dry parse only, no model call). The AUTHORITATIVE deployed-judge validation (does this fixture pass on qwen3:1.7b via the cluster port-forward, and do downstream_cve_only / downstream_behavioral still hold with no drift) is pending and runs separately against the deployed judge.
Owner
Author
|
Deployed A/B validation (main loop, deployed qwen3:1.7b on pod pfwwl) — PASS: Acceptance met: the new fixture scores attack + correct cut-set on the deployed judge, and the two named drift checks are unchanged. Note (pre-existing, out of scope for this PR): |
thejefflarson
deleted the
thejefflarson/jef-698-deployed-pod-bench-route-forwarded-backend-fixture
branch
August 1, 2026 21:36
thejefflarson
added a commit
that referenced
this pull request
Aug 1, 2026
…, drop stale exemption
The breadcrumb sweep mangled two `// JEF-317, ON-NODE PENDING` trailing
comments in agent/protector-agent-ebpf/src/vmlinux.rs into `/, ON-NODE
PENDING` inside a `const _: () = { ... }` compile-time offset guard —
invalid Rust that would break the ebpf cross-compile. Restore the `//`
markers (JEF-317 dropped per the sweep).
#314 (which was concurrently editing scripts/judge_bakeoff_cutchoice.py)
is now merged, so its temporary EXEMPT_FILES entry is stale. Sweep the 3
remaining breadcrumbs there (keep the CUT TRAP / REFUTE TRAP mechanism
names; ADR-0034 is already cited inline) and remove the exemption, leaving
the guard with a single structural exemption (its own source file).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP
thejefflarson
added a commit
that referenced
this pull request
Aug 1, 2026
… sweep breadcrumbs (#316) * ci(guard): enforce the committed-content self-containment guardrail + sweep breadcrumbs Adds engine/tests/self_containment_guard.rs, a repo-wide nextest guard (alongside file_size_guard.rs) that fails the build on a JEF-nnn ticket reference or a linear.app URL in tracked source comments/docstrings and docs, with narrow, documented exemptions for the guardrail's own naming text in CLAUDE.md and for scripts/judge_bakeoff_cutchoice.py (mid-edit on a concurrent branch). Sweeps every existing JEF-nnn/linear.app breadcrumb the guard would flag out of tracked source and docs, replacing each with the ADR it records, the module it lives beside, or the inline reasoning already next to it — preserving the rationale, never the ticket number. Comment/doc-prose only; no code-logic change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP * ci(guard): fix mangled ebpf comment marker + fold judge_bakeoff sweep, drop stale exemption The breadcrumb sweep mangled two `// JEF-317, ON-NODE PENDING` trailing comments in agent/protector-agent-ebpf/src/vmlinux.rs into `/, ON-NODE PENDING` inside a `const _: () = { ... }` compile-time offset guard — invalid Rust that would break the ebpf cross-compile. Restore the `//` markers (JEF-317 dropped per the sweep). #314 (which was concurrently editing scripts/judge_bakeoff_cutchoice.py) is now merged, so its temporary EXEMPT_FILES entry is stale. Sweep the 3 remaining breadcrumbs there (keep the CUT TRAP / REFUTE TRAP mechanism names; ADR-0034 is already cited inline) and remove the exemption, leaving the guard with a single structural exemption (its own source file). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds one
CASEtoscripts/judge_bakeoff_cutchoice.py(the T2b cut-choice bench, ADR-0034):route_forwarded_entry_cve— a backend reached only via an Ingress route (no direct Service exposure) rendered in the ENTRY position with its own loaded-at-runtime CVE.workload/public/Pod/checkout-api-7f9d4c8b6d-x2p9k— distinct from the existingweb-frontend-5d8/web-cache-0pair so it's unambiguously the route-forwarded case, not a rename ofentry_only_log4j.entry_only_log4j's shape exactly (CVE loaded-at-runtime on the entry, clean downstream cache, same containment menu format) but represents a workload that only becomes an internet-facing entry because a route forwards to it.attack{checkout-api-7f9d4c8b6d-x2p9k}only (the route-forwarded backend itself, mirroring the entry-only minimal-cut discipline)This validates the now-merged Ingress exposure observer (
feat(engine): ingress exposure observer — route-transitive Exposure::Internet through declared L7 routes (ADR-0038), #313): a route-forwarded backend becomes anExposure::InternetENTRY and its loaded CVE should promote through the same edge-CVE lane the judge already handles for a directly-exposed entry.Test plan
python3 -c "..."dry-imports the script and confirmsCASESgrows from 9 → 10, andrender()builds the prompt for the new case without error (no network/model call — see command below).wc -l scripts/judge_bakeoff_cutchoice.py→ 275 lines, well under the repo's 1,000-line hard cap.cargo fmt --check→ clean (no Rust files touched by this change)./soundcheck:pr-reviewself-review → no Critical/High findings (pure data-literal test fixture addition, no new I/O/network/deserialization surface)./simplifyself-review (single-pass, no Agent-tool fan-out available in this context) → already clean; new case reuses existingmenu()/CACHE/CACHE_LINE/OBJShelpers, no new code path.Pending (not run here, per ADR-0033 deployed-pod discipline): the AUTHORITATIVE deployed-judge validation — running this fixture against the cluster-deployed
qwen3:1.7bvia a port-forward (OLLAMA_URL=http://localhost:11436/api/chat python3 scripts/judge_bakeoff_cutchoice.py qwen3:1.7b), confirming it scoresattack/correct cut-set, and confirmingdownstream_cve_onlyanddownstream_behavioralshow no drift. This requires cluster access this environment doesn't have and is expected to run in the main loop.Closes JEF-698