fix(stella-tools): keep the head and tail of over-cap bash/custom output instead of dropping the tail (#1889)

[macanderson]

Problem

bash.rs and custom.rs capped tool output at MAX_OUTPUT_BYTES = 100 KB — ~28k estimated tokens, ~19% of the 150k compaction budget in one result. A single large result does not trigger the compaction that would reclaim it (compact_measured returns early when the rest of the transcript is small), and the 8-step retention horizon then keeps it verbatim: one cargo test --workspace or npm ci printing 100 KB cost ~224k input tokens, not 28k (#1889, the deliberate residue of #1842).

On top of the budget shape, the crate had grown three independent elision spellings: exec::truncate_middle (50/50 split, one marker format), bash.rs's inline copy (50/50, a second marker format), and custom.rs::truncate_middle_out (40/60, the second marker format again). Three copies is the shape that lets one drift — and they already had, on both the split and the marker.

Decision

Head + tail elision through one shared helper, and a 64 KB cap.

• One spelling: exec::truncate_middle_capped(s, max_bytes) is now the crate's single model-facing elision primitive. exec::truncate_middle, bash, and custom all cut through it; custom.rs's private copy (and its boundary helpers) are deleted. It sits in exec.rs because that module already owns the output-cap policy for every other runner (truncate_middle, CappedStream, truncate_preview), and exec.rs is not a god file.
• Split: 40% head / 60% tail. Tail-biased because a failing command's densest signal is at the end — the final test summary, the last error, the exit status. This is lesson L-S3, already ratified in custom.rs; bash moves from 50/50 to match rather than the reverse, and the shared function makes future divergence structurally impossible. Both cuts land on UTF-8 char boundaries (the existing discipline of truncate_preview / the old inline code).
• Marker names the elided byte count and the cap: [… N bytes truncated: output exceeded the 65536-byte cap; the head and tail are kept …] — the model can account for every byte and knows the bound it is working under, mirroring CappedStream's "say which cap did this" convention.
• Cap: 100 KB → 64 KB (~12% of the compaction budget) — the exact point tools: one read_file can inject ~114k tokens (400KB cap) that retention then protects for 8 steps #1842 ratified for read_file (400 KB → 64 KB, PR fix(stella-tools): cut read_file's payload cap from 76% of the context budget to 12% (#1842) #1888), for the same multiplier argument. It stays the same order of magnitude and preserves P1: budgets: fill in every deferred cap, deadline, and retention window in one sitting #616's ratio argument: still 2.2x exec::MAX_OUTPUT_BYTES (30k), so the shell remains the agent's wide sensory channel while read_output pages stay cheap. custom.rs now aliases bash's constant (pub(crate) use) instead of carrying a copy, so the two cannot drift.
• Retention-aging is deliberately not here. The issue's option (b) — aging an oversized result out of retention early — belongs to the engine's retention fold, tracked by fix(stella-core): gate retention aging on reclaimable bytes, not a result count #1819 (reclaimable-bytes gate) and the Implement the file-budget design (docs/design/file-budget.md) — Status: Proposed, currently untracked #1438 file-budget umbrella. This PR bounds the cost at the source; those bound what survives the horizon.

Exemplar for the consolidation shape: the crate's own shell_quote, which collapsed five drifting copies into one exec.rs primitive with a "a new one must be too" contract; this PR does the same for elision.

Witness

over_cap_output_keeps_first_and_last_lines_with_a_named_elision, once per surface (bash::tests, custom::tests): a command/script emitting a first sentinel line, MAX_OUTPUT_BYTES of filler, and a last sentinel line yields a result containing both sentinel lines, a marker naming the elided byte count and the cap, bounded by the cap plus the marker. Every size is derived from the constant — nothing hard-codes a human-readable size, the assertion shape #1842 caught going stale.

Verified failing on the old code the artisanal way — both tests spliced onto the parent commit:

test bash::tests::over_cap_output_keeps_first_and_last_lines_with_a_named_elision ... FAILED
test custom::tests::over_cap_output_keeps_first_and_last_lines_with_a_named_elision ... FAILED
  panicked: 'the marker names the cap it enforced'

…and passing on this branch. exec::tests additionally pin the helper itself: the marker's elided count is arithmetically exact (derived, not hard-coded), tail budget ≥ head budget (L-S3), both cuts survive landing mid-multibyte-char, and at-or-below-cap input is byte-identical.

Verification

• cargo test -p stella-tools: 723 passed (lib) + all integration suites green, 0 failed.
• cargo clippy -p stella-tools --all-targets -- -D warnings clean; cargo fmt -p stella-tools --check clean.
• Workspace-wide validation left to CI per build economy (no public API changed; every touched item is pub(crate)).

Closes #1889

Refs #1842 #1819 #1438

Summary by Sourcery

Unify and tighten stdout/stderr truncation across exec, bash, and custom tools to keep both the head and tail of oversized outputs under a smaller shared cap.

Bug Fixes:

• Ensure over-cap bash and custom tool output retains both the first and last lines instead of dropping the tail under some sizes.

Enhancements:

• Introduce a shared truncate_middle_capped helper in exec that performs UTF-8-safe, tail-biased head+tail elision with an explicit marker naming the elided byte count and cap.
• Reduce the bash/custom output cap from 100 KB to 64 KB to better bound transcript and compaction costs, while preserving the intended ratio to exec output limits.
• Alias custom tool output caps and elision behavior to bash so their output budgets and truncation semantics cannot drift apart.

Tests:

• Add unit tests for truncate_middle_capped to validate exact elided byte accounting, UTF-8 boundary safety, and no-op behavior at or below the cap.
• Add integration tests for bash and custom tools that verify over-cap outputs keep both sentinel lines, include a cap- and byte-count-bearing elision marker, and remain within the bounded size.

[sourcery-ai]

Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
stella-cli-docs	Ignored		Aug 6, 2026 6:42pm

[sourcery-ai]

Reviewer's Guide

Centralizes and unifies middle-out output truncation across exec, bash, and custom tools by introducing a shared UTF-8-safe head+tail elision helper with a 64KB cap, wiring bash/custom to it, tightening caps, and adding tests that assert sentinel preservation and marker correctness.

File-Level Changes

Change	Details	Files
Introduce shared truncate_middle_capped helper in exec with UTF-8-safe 40/60 head+tail split and explicit elision marker.	Refactor truncate_middle to delegate to truncate_middle_capped using exec::MAX_OUTPUT_BYTES. Implement truncate_middle_capped(&str, max_bytes) that keeps 40% head and 60% tail, computes elided byte count, and inserts a marker naming both the elided bytes and the cap. Add floor_char_boundary and ceil_char_boundary helpers to safely align head/tail cuts to UTF-8 char boundaries. Add tests to assert marker correctness, UTF-8 boundary safety, and no-op behavior at or below the cap.	crates/stella-tools/src/exec.rs
Make bash use the shared elision helper and lower its output cap to 64KB, while preserving its role as a wider channel than exec.	Change bash::MAX_OUTPUT_BYTES from a private 100_000-byte constant to a pub(crate) 64 * 1024-byte constant with expanded documentation tying it to compaction/retention budget reasoning. Replace bash’s inline middle-out truncation logic with a call to crate::exec::truncate_middle_capped(&combined, MAX_OUTPUT_BYTES). Add an integration-style test that runs a bash command emitting first and last sentinel lines plus filler to exceed the cap, asserting both sentinels, presence of the cap and elided-byte marker text, and an overall length bounded by roughly cap plus marker.	crates/stella-tools/src/bash.rs
Make custom tools reuse bash’s output cap and the shared elision helper, removing custom’s private truncation implementation.	Replace custom::MAX_OUTPUT_BYTES definition with pub(crate) use crate::bash::MAX_OUTPUT_BYTES so bash and custom share a single cap constant. Delete truncate_middle_out plus its floor_boundary/ceil_boundary helpers, removing the custom-specific elision implementation and marker format. Update run_custom to call crate::exec::truncate_middle_capped for both stdout (success) and stderr (error) paths. Retain and rename the oversized_output test to validate middle-out elision still occurs, and add a new test mirroring bash’s sentinel test to assert first/last lines are preserved and the marker names both cap and truncated bytes.	crates/stella-tools/src/custom.rs

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#1889	Change bash tool output capping so that oversized outputs no longer drop the tail, instead using a bounded head+tail elision (or equivalent) with an appropriate cap that avoids a single result consuming an outsized share of the compaction budget.	✅
#1889	Make custom-tool output capping use the same mechanism and cap as bash, so their behavior and limits cannot drift.	✅
#1889	Add tests (witnesses) for the chosen mechanism that are derived from the cap constant and verify that over-cap outputs preserve both head and tail and include a clear elision marker (naming at least the cap and elided size).	✅

Possibly linked issues

• tools: bash and custom-tool output caps carry the same context-budget shape as read_file did #1889: The PR applies head+tail elision with a 64KB cap to bash/custom, consolidates truncation logic, and adds witnesses as requested.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.