fix(stella-pipeline): bound the oracle trace at the verifier-prompt ingress (#1787) - #1982
Merged
Merged
Conversation
…ngress The trusted evidence summary rode into the verdict prompt with no length bound on its one growing channel: oracle_trace gains an observation per verification round, and the repair gate can keep granting rounds as long as a measured budget affords them — the diff has a token budget, the trusted zone had none (#1787's folded-in item). Bounded where the prompt value is constructed (the #1932 rule), to the newest 24 observations with the drop stated in-band, so a verifier reads 'earlier observations exist' rather than a trace that silently starts mid-run. The stored ladder snapshot keeps the full trace; provenance rendering is deliberately untouched. Refs #1787
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
Contributor
Reviewer's GuideBounds the oracle trace rendered into the verifier evidence summary by introducing a capped, in-band-noted rendering function and adds tests to witness both the clipping behavior for pathological traces and the no-op behavior for ordinary traces. File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
…mon::boot (#1970) ## What & why `main`'s workspace doc gate is red: `crates/stella-cli/src/daemon/boot.rs`'s module doc intra-doc-links `SkipReason::NoResumePoint`, but `daemon::boot` is a **private** module and `SkipReason` is `pub(super)`, so rustdoc resolves nothing and `rustdoc::broken_intra_doc_links` fails under `-D warnings`. The link landed in #1939 and was invisible until now, which is the part worth recording: `cargo doc` bails on the first crate that fails, so `stella-store`'s private-link error (fixed in #1965) masked this one entirely. Fixing one rustdoc break surfaces the next one down — expect to iterate, not to assume the first fix was the last. The fix is the same shape as #1965's: name the item in prose rather than link something rustdoc cannot see, with a parenthetical saying why. ## The witness - [ ] No witness needed (docs) — because: this is a doc-comment-only change, and the gate itself is the check. `RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps` exits **101** on `main` and **0** with this change, verified by exit code rather than by reading output. ## The gate - [x] `cargo doc --workspace --no-deps` under `-D warnings` (exit 0, the whole workspace — re-run after the fix to confirm nothing further was masked) - [x] `cargo fmt --check` - [x] CLA signed Refs #1939, #1965. ## Summary by Sourcery Documentation: - Clarify daemon boot module documentation by naming SkipReason::NoResumePoint in text instead of using an intra-doc link that rustdoc cannot resolve.
…nd, incl. a silently deleted #1793 witness (#1971) ## What `main` at b5ab7f8 did not build: `cargo clippy -p stella-pipeline --all-targets -- -D warnings` failed. A compile error in the lib masks the test target, so it read as one failure and was really **four**, from two different merges. ### From #1953 (the #1778 research stage) 1. **`management_prompt/tests.rs` (E0004)** — `ModelCallRole::Research` is a new variant and `management_system_block`'s match is exhaustive by design. `Research` rides the sub-agent primitive, so its system prompt travels on the `SubAgentSpec`, never through `metered_raw_call` — it joins the never-dispatched arm, and `ALL_ROLES` grows to 15. 2. **`pipeline.rs` (`too_many_arguments`)** — the new `research` parameter pushed `plan_stage` to 8 args, one over the cap. Bundled `budget`/`total` into the `Spend` struct every stage downstream of the fan-out already takes, rather than `#[allow]`-ing the lint. `pipeline.rs` is a god file, so the doc comment is written to land the file back at **exactly** its 3462 ceiling — no growth. ### From #1951 — a same-seam clobber #1951 rewrote `tests/verification_hardening.rs` wholesale and dropped three items #1945 had added to it hours earlier, in a file its own subject (per-candidate verifier degradation) never needed to touch: 3. **`PassingShell` + `shell_call_result` (E0425 ×2)** — the child module `flip_halt_arming.rs` was left referencing two doubles that exist nowhere in the tree. 4. **`a_revision_halts_at_the_step_where_the_tracked_test_flips`** — the configured-command **witness for #1793**. Deleting it failed no gate, because the crate had already stopped compiling for reason 3. **#1793 has been shipping with half its witness silently gone.** Restored verbatim from eddf970. ## The structural fix, not just the restore Restoring the witness took the parent file to 1557 lines, which `file-size` rejects outright — and the baseline takes no new entries. So this splits rather than exempts, and the split is the one the content was asking for: **both** #1793 witnesses and the two doubles they share now live in `verification_hardening/flip_halt_arming.rs`, the module already named for the concern. Parent drops to 1434. That the two witnesses were ever in separate files is what let the clobber happen quietly. With the cluster in one file, the same wholesale rewrite is a merge conflict instead of a silent deletion — the module doc records why. ## Witness Not a pure refactor: items 3 and 4 restore two witness tests, and neither passes vacuously. Each asserts a scripted-prompt count, so a `PassingShell` that omitted the `[exit code: 0]` marker `flip_halt::exit_status` parses would leave the halt unarmed, the revision would consume the steps scripted beyond the flip, and the count would be wrong. ``` test ...flip_halt_arming::a_revision_halts_at_the_step_where_the_tracked_test_flips ... ok test ...flip_halt_arming::an_authored_witness_arms_the_revision_flip_halt ... ok ``` - `cargo test -p stella-pipeline` — **605 passed, 0 failed** - `cargo clippy -p stella-pipeline --all-targets -- -D warnings` — clean - `make guards-fast` — green, `file-size` and `god-files` included ## Overlap with #1965 This branch originally also retired the stale `crates/stella-protocol/src/event.rs` baseline entry (1454 lines against a recorded 2965 — a fifth red gate on main). **#1965 landed the same fix while this was in flight**, so that work was dropped here in favour of theirs on rebase; the remaining commit is only the split. No duplicate baseline edit.
…-bound # Conflicts: # crates/stella-pipeline/src/management_prompt/tests.rs # crates/stella-pipeline/src/pipeline/scope_stage.rs # crates/stella-pipeline/src/pipeline/tests/verification_hardening/flip_halt_arming.rs
This was referenced Aug 7, 2026
macanderson
added a commit
that referenced
this pull request
Aug 7, 2026
…ngress (#1787) (#2002) ## Why this PR exists **#1787's fix is not in `main`.** PR #1982 carried it, but its base was the topic branch `unbreak-main-pipeline`, whose own PR (#1975) was **closed, not merged**. #1982 then merged into that dead branch, so the oracle-trace bound landed nowhere `main` can see, and nothing is carrying that branch forward. It also merged in a **broken** state. While the base was being reconciled with `main`, git's auto-merge of the two independently-written unbreaks concatenated both sides, leaving: - `struct PassingShell` and `fn shell_call_result` **defined twice** - `async fn a_revision_halts_at_the_step_where_the_tracked_test_flips` defined twice - a duplicate `ModelCallRole::Research` match arm (unreachable pattern) None of that compiles. `unbreak-main-pipeline` currently holds it; `main` is unaffected. This PR is the clean landing: **`main` plus `evidence.rs`, and nothing else.** ## What it does (#1787) Bounds the oracle trace at the verifier-prompt ingress. The trace grows once per verification round and the repair gate can keep granting rounds while a measured budget affords them — so unlike the diff, which rides under a token budget, this channel had **no ceiling at all**. - `MAX_ORACLE_TRACE_OBSERVATIONS = 24` — sized far above a normal run (baseline plus a handful of rounds) so the bound only bites a pathological loop. - `bounded_oracle_trace` keeps the **newest** observations and states the drop **in-band** (`…N earlier observation(s) omitted → …`), so the verifier reads "earlier observations exist" rather than a trace that silently starts mid-run. - The **stored snapshot keeps the full trace**; only the prompt ingress is clipped — the structural-bound rule from #1932. ## Witnesses - `a_pathological_oracle_trace_is_clipped_with_the_drop_stated` — a 100-observation trace renders clipped to the newest 24 with the omission counted in-band. - `an_ordinary_oracle_trace_renders_unchanged` — the bound does not touch a normal run, so this cannot ship as "always clip". Observations alternate pass/fail in the fixture so a clipped render is distinguishable from a repeated one. ## Verification - `cargo test -p stella-pipeline` — **585 pass**, 0 fail, including both witnesses above - `cargo fmt --check -p stella-pipeline` — clean - Diff vs `main` is exactly one file: `crates/stella-pipeline/src/pipeline/evidence.rs` (+74/−2) ## CI is red on `main`'s breaks, not this diff This branch is merged up to current `main`. Every failing step fails in a file this PR does not touch, and each already has a dedicated unbreak in flight: | Failing step | Where | Covered by | |---|---|---| | `check-file-size` | `scripts/file-size-baseline.txt` (parallel-merge skew) | **#2003**, **#2008** | | `cargo fmt --check` | not this crate's file | **#2005** | | clippy: unused `spend` / unused `mut` | `pipeline/scope_stage.rs:34` — a dead local `#1985` left behind | **#2000** | | rustdoc: unresolved `CompactionRewrite` | `stella-protocol` | **#2010** | The clippy one is worth naming precisely, since it is `stella-pipeline`: `main`'s `scope_stage.rs` binds `let mut spend = Spend { budget, total };` and then never uses it — the loop constructs a fresh `Spend` inline per iteration. `spend` occurs exactly once in the file. That is `main`'s dead local, untouched by this PR. No competing unbreak is included here on purpose — six are already open against `main`, and duplicating one is how `main` gets re-broken. ## Note on the dead branch `unbreak-main-pipeline` still holds the duplicate-definition breakage and the only copy of #1982's merge. It is not reachable from `main` and its PR is closed, so nothing needs to be reverted — but it should not be revived without first taking `main`'s copies of `flip_halt_arming.rs`, `management_prompt/tests.rs` and `scope_stage.rs`, which is what this PR does. Filed as #2001. Closes #1787
macanderson
added a commit
that referenced
this pull request
Aug 7, 2026
…ngress (#1787) (#2012) > Supersedes #1982, which was auto-closed when the branch it was stacked on > went away. Same single commit, now rebased directly on `main`. > > Note: `main` currently fails `cargo clippy -p stella-pipeline` with five > pre-existing warnings unrelated to this change (fixed in #2009), so this > PR's clippy step inherits them until that lands. ## What & why The last unbounded ingress into the verdict prompt, and the item #1787 folds in at the end of its body: > Also worth folding in: the trusted evidence summary has no length bound > (`oracle_trace` grows per observation; the diff has a token budget, the > trusted zone does not) — `pipeline/evidence.rs` since the extraction. `verifier_evidence_summary` renders `oracle_trace` in full. That trace gains an observation per verification round, and the repair gate (#1479) keeps granting rounds for as long as a measured budget affords them — so the one channel that grows without limit was also the one channel with no ceiling. Every other input to that prompt is bounded: the diff has a token budget, recall frames have `bound_recalled_frames`, and `Verdict::reasoning` got its cap in #1932. Bounded to the newest 24 observations, with the drop **stated in-band**: ``` oracle_trace=[…76 earlier observation(s) omitted → candidate:pass → candidate:fail → …] ``` Three choices worth naming, because each has a wrong-looking alternative: - **Newest kept, oldest dropped.** The recent runs are what the verdict weighs; a trace clipped from the front would hand the verifier a history that stops before the evidence. - **Stated, not silent.** A trace that silently began mid-run reads as the whole run — the verifier would draw conclusions about a first observation that was not the first. - **Clipped where the value is constructed**, not at a downstream consumer. That is the same "structural, not by convention" rule #1932 applied to `reasoning`, and it is why the stored `LadderSnapshot` and `verdict_provenance` are deliberately untouched: the bound is on the *prompt ingress*, not on the record. 24 is sized far above a normal run (a baseline plus a handful of rounds), so the bound only ever bites a pathological loop. It is a named constant next to its rationale rather than a literal. ## The witness - [x] This PR includes a witness test `a_pathological_oracle_trace_is_clipped_with_the_drop_stated` — a 100-observation trace renders exactly 24 entries behind the `…76 earlier observation(s) omitted →` marker, and still ends on the newest observation. `an_ordinary_oracle_trace_renders_unchanged` is the other half, and the one that matters for regression: a 5-observation trace is asserted **byte-identical to `render_oracle_trace`**, main's own unbounded function, which is still present and still used for provenance. So every prompt the bound does not bite is unchanged to the byte — which also means no verdict-reuse digest (#1431) moves for an ordinary run. Honest note on "fails on main": these pin a bound that does not exist on `main`, so the failure there is that `bounded_oracle_trace` is not defined — the same shape as #1932's witnesses for the `reasoning` cap, and the shape any "add a missing ceiling" change has. The behavioural claim is carried by the second test, which compares against main's function directly rather than against a copied expectation. ## The gate - `cargo test -p stella-pipeline` — 585 + 5 + 2 + 5 + 6 + 4 = **607 passed, 0 failed** - `cargo clippy -p stella-pipeline --all-targets` — clean - `cargo fmt --check -p stella-pipeline` — clean - `scripts/check-file-size.sh` — OK, none grew (`evidence.rs` was extracted from `pipeline.rs` precisely so this kind of channel can be added without touching a god file, and that still holds) Full workspace left to CI. ## Nothing left behind `Refs #1787`, deliberately not `Closes` — this is the folded-in bound only. Item 1 (a provider-parity-aware structured verdict output path, invariant 8) remains open and is being approached from a different angle in #1964; item 2 shipped as #1932 and item 3 as #1951. Refs #1787 ## Summary by Sourcery Bound the oracle trace rendered in verifier evidence summaries and added tests to cover the new trusted-zone length cap. New Features: - Introduce a bounded oracle trace renderer for verifier prompts that limits the trusted evidence summary to the newest observations while indicating omissions in-band. Tests: - Add witness tests ensuring long oracle traces are clipped with an explicit omission marker and that ordinary short traces remain byte-identical to the unbounded renderer.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The last unbounded ingress into the verdict prompt, and the item #1787 folds in
at the end of its body:
verifier_evidence_summaryrendersoracle_tracein full. That trace gains anobservation per verification round, and the repair gate (#1479) keeps granting
rounds for as long as a measured budget affords them — so the one channel that
grows without limit was also the one channel with no ceiling. Every other input
to that prompt is bounded: the diff has a token budget, recall frames have
bound_recalled_frames, andVerdict::reasoninggot its cap in #1932.Bounded to the newest 24 observations, with the drop stated in-band:
Three choices worth naming, because each has a wrong-looking alternative:
a trace clipped from the front would hand the verifier a history that stops
before the evidence.
run — the verifier would draw conclusions about a first observation that was
not the first.
That is the same "structural, not by convention" rule fix(stella-pipeline): bound the verdict reasoning at the point it is constructed (#1787) #1932 applied to
reasoning, and it is why the storedLadderSnapshotandverdict_provenanceare deliberately untouched: the bound is on the promptingress, not on the record.
24 is sized far above a normal run (a baseline plus a handful of rounds), so
the bound only ever bites a pathological loop. It is a named constant next to
its rationale rather than a literal.
The witness
a_pathological_oracle_trace_is_clipped_with_the_drop_stated— a 100-observationtrace renders exactly 24 entries behind the
…76 earlier observation(s) omitted →marker, and still ends on the newest observation.an_ordinary_oracle_trace_renders_unchangedis the other half, and the onethat matters for regression: a 5-observation trace is asserted byte-identical
to
render_oracle_trace, main's own unbounded function, which is stillpresent and still used for provenance. So every prompt the bound does not bite
is unchanged to the byte — which also means no verdict-reuse digest (#1431)
moves for an ordinary run.
Honest note on "fails on main": these pin a bound that does not exist on
main,so the failure there is that
bounded_oracle_traceis not defined — the sameshape as #1932's witnesses for the
reasoningcap, and the shape any"add a missing ceiling" change has. The behavioural claim is carried by the
second test, which compares against main's function directly rather than
against a copied expectation.
The gate
cargo test -p stella-pipeline— 585 + 5 + 2 + 5 + 6 + 4 = 607 passed, 0 failedcargo clippy -p stella-pipeline --all-targets— cleancargo fmt --check -p stella-pipeline— cleanscripts/check-file-size.sh— OK, none grew (evidence.rswas extracted frompipeline.rsprecisely so this kind of channel can be added without touchinga god file, and that still holds)
Full workspace left to CI.
Nothing left behind
Refs #1787, deliberately notCloses— this is the folded-in bound only.Item 1 (a provider-parity-aware structured verdict output path, invariant 8)
remains open and is being approached from a different angle in #1964; item 2
shipped as #1932 and item 3 as #1951.
Refs #1787
Summary by Sourcery
Bound the oracle trace rendered in the verifier prompt to a fixed number of recent observations and document the truncation in-band while preserving full traces in stored provenance.
Enhancements:
Tests: