runner: keep readCapped's truncation on a rune boundary

[Soph]

https://entire.io/gh/entireio/cli/trails/1255

readCapped embeds repo docs (CLAUDE.md, AGENTS.md, README.md, go.mod) into the prompt entire runner tune sends to a model. Its cap is a byte budget applied as s[:maxLen], while both callers describe it in characters:

tuneDocCap    = 6000 // max chars embedded per doc (CLAUDE.md etc.)
tuneReadmeCap = 2000

A doc whose 6000th byte lands inside a multi-byte rune therefore put an invalid UTF-8 sequence into the prompt. Nothing downstream validates it, so it was invisible from the call site.

The fix

A continuation byte at the cut is exactly what "we cut mid-rune" means, so back off the continuation bytes — at most UTFMax-1 of them, the furthest a rune's start can be. utf8.RuneStart answers that from one byte.

Validating the prefix instead (utf8.ValidString(s[:cut])) answers a different question and is the wrong tool twice over: it walks the whole 6KB, and it returns false for a doc that is not UTF-8 at all, so chasing it back finds no valid prefix and drops the file's content in favour of a bare truncation marker. Invalidity our cut did not cause is the file's own, and the under-cap path already passes those bytes through.

Two traps, both pinned by tests

Getting here took two wrong turns, and the tests exist so neither comes back:

• An unbounded backoff empties a non-UTF-8 file. A latin-1 README came back as nothing but …(truncated)… — worse than the mid-rune cut being fixed.
• The floor has to be explicit, not implied by an iteration count. At maxLen=1 over a file of continuation bytes, a count-based bound indexes s[-1] and panics. Latent — the smallest cap any caller passes is go.mod's 400 — which is the reason to pin it rather than shrug.

Tests

These are the first tests readCapped has had: the boundary itself, a short file, a missing one, a non-UTF-8 doc keeping its content, and every cap from 0 to 7 across three body shapes.

The last one asserts the contract as a disjunction — the cut lands on a rune boundary, or it keeps the file's own bytes — rather than "not empty". An empty result is correct below one rune's width, where no non-empty valid prefix exists; an earlier draft of the test asserted non-empty and was wrong about the contract, not the code.

Why it's separate

Split out of #2290 (os.Root coverage follow-ups) on review feedback: a byte cap cutting a rune in prompt text has nothing to do with filesystem anchoring, and this is the piece of that branch that stands alone. #2290 no longer touches runner_gather.go.

Verification

mise run lint 0 issues, mise run test:ci exit 0, and GOOS=windows|linux|darwin go vet ./... all clean.

🤖 Generated with Claude Code

---

Note

Low Risk
Localized change to doc truncation for tuning prompts with thorough tests; no auth, network, or data-path impact.

Overview
readCapped (used when entire runner tune embeds CLAUDE.md, README.md, etc. into the model prompt) no longer uses a naive s[:maxLen] cut when content exceeds the byte cap. It walks back at most utf8.UTFMax-1 bytes to land on a rune start, and if no start exists in that window (e.g. latin-1 or continuation-byte-only content), it keeps the original byte cut so truncated output still includes file bytes instead of only the truncation marker.

Adds the first readCapped tests: valid UTF-8 rune-boundary truncation, short/missing files, non-UTF-8 content preservation, bounded backoff, and small caps (0–7) to prevent an index-out-of-range panic on tiny caps.

^{Reviewed by Cursor Bugbot for commit d096983. Configure here.}

[Copilot]

🟢 Approval recommended

The change is localized, preserves the intended byte-cap behavior, and includes thorough tests for the key edge cases and regressions.

Pull request overview

This PR hardens readCapped (used by entire runner tune to embed repo docs into a model prompt) so truncation respects UTF-8 rune boundaries, preventing invalid UTF-8 from being introduced when truncating by a byte budget.

Changes:

• Adjust readCapped truncation to back off up to utf8.UTFMax-1 continuation bytes to land on a rune boundary (when possible), otherwise preserve the original byte cut for non-UTF-8 content.
• Add unit tests covering rune-boundary truncation, short and missing files, non-UTF-8 preservation, and tiny cap edge cases (0–7) to prevent index-underflow panics.

File summaries

File	Description
cmd/entire/cli/runner_gather.go	Updates readCapped to avoid mid-rune truncation while keeping non-UTF-8 bytes intact under truncation.
cmd/entire/cli/runner_gather_test.go	Adds comprehensive tests to pin truncation semantics and regressions around small caps and non-UTF-8 inputs.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.