Skip to content

fix(profile): require a written reason to weaken the prompt-budget cap#248

Merged
drewstone merged 1 commit into
mainfrom
fix/prompt-budget-requires-reason
Jul 25, 2026
Merged

fix(profile): require a written reason to weaken the prompt-budget cap#248
drewstone merged 1 commit into
mainfrom
fix/prompt-budget-requires-reason

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

The 40,000-byte system-prompt ceiling could be raised — or muted via warnOnly — with no justification. That silent escape hatch is how concatenated reference material kept shipping: one product ran a 103,602-byte prompt behind a 120,000 ceiling and its turns failed outright.

Weakening the cap now requires overBudgetReason, checked on every compose rather than only once a prompt outgrows the raised number — so the silent raise itself is what fails. The error names the remedy (corpusSkills/userSkillMounts/composeSkills({ mode: 'mounted' }), keep inline only what must be obeyed without a tool call). Lowering the cap still needs no reason.

Breaking for callers that raise the cap. Fleet audit: only gtm-agent raises it (46,000, for a measured reason) — insurance is at 28,000 and tax pins the default, both unaffected. gtm needs overBudgetReason added immediately after this publishes.

Verified: tsc --noEmit exit 0; full suite 3,146 tests passing; 31/31 in the profile suite including new coverage that the raise-without-reason and warnOnly-without-reason paths throw, and that lowering needs no reason.

composeAgentProfile enforces a 40,000-byte system-prompt ceiling, but the cap
could be raised — or muted with warnOnly — silently. That escape hatch is how
reference material concatenated into a prompt kept shipping: one product ran a
103,602-byte prompt behind a 120,000 ceiling, and its turns failed outright.

Weakening the cap now needs overBudgetReason, and the check runs on every
compose rather than only once a prompt has outgrown the raised number, so the
silent raise is the step that fails. The error names the fix: move reference
material to resources.files via corpusSkills/userSkillMounts or
composeSkills({ mode: 'mounted' }) and keep inline only what the agent must obey
without a tool call. Lowering the cap still needs no justification.

@tangletools tangletools left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 72abc9bc

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-24T23:51:29Z

@tangletools tangletools left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Value Audit — sound-with-nits

Verdict sound-with-nits
Concerns 2 (2 weak-concern)
Heuristic 0.0s
Duplication 0.0s
Interrogation 89.4s (2 bridge agents)
Total 89.4s

💰 Value — sound

Adds a required written reason to raise or mute the 40KB system-prompt cap, closing the silent escape hatch that let oversized prompts ship — minimal, well-tested, and squarely in the codebase's grain.

  • What it does: Introduces an optional overBudgetReason field on ComposeProfileBudget and a pure assertBudgetPolicy() guard (src/profile/index.ts:151-166) that throws if maxSystemPromptBytes is raised above the 40,000 default OR warnOnly is set without a non-empty reason. Crucially the guard runs BEFORE the size check on every compose (src/profile/index.ts:202), so the silent raise itself fails — not ju
  • Goals it achieves: Stop the recurring failure mode where concatenated reference material bloated a system prompt past the model's effective range (the cited 103,602-byte prompt behind a 120,000 ceiling caused outright turn failures), by making the weakening step — not the eventual overflow — the loud failure. Forces a conscious, in-code, durable justification for any product decision to weaken the cap, and steers th
  • Assessment: A clean, proportionate change. It adds one optional interface field, one pure function, and runs at the single existing composition choke point with no new side effects or I/O. The four behavioral branches are each covered by a test (src/profile/index.test.ts:313, 327, 337, 343, 348). The remedy text references only primitives this same module already exports, so it points at real lift rather than
  • Better / existing approach: none — this is the right approach. Searched src/ for any prior 'require-a-reason-to-weaken-a-guard' pattern (grep overBudgetReason|require.*reason|justification) and found none — this is new mechanism. The guard correctly sits inside the existing assertSystemPromptWithinBudget choke point rather than introducing a second enforcement site, and the standalone /prompt assembler path reuses it via the
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 2
  • Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content

🎯 Usefulness — sound-with-nits

A well-placed fail-loud guard that makes weakening the 40KB prompt cap require a written reason, extending the existing budget gate in its own grain; the only gap is that the policy check piggybacks on the size check and so skips the no-systemPrompt edge.

  • Integration: Reachable and live: assertBudgetPolicy runs at the top of assertSystemPromptWithinBudget (src/profile/index.ts:202), which is the single composition gate called from composeAgentProfile (src/profile/index.ts:300) and directly from the test-vertical mini-app (tests/vertical/mini-app.ts:144). Every fleet product that composes a profile hits it; the PR body correctly identifies gtm-agent as the sole
  • Fit with existing patterns: Fits the established pattern exactly. The byte-budget gate itself was added for the same recurring failure (a 122,659-byte prompt returned empty answers, per the DEFAULT_MAX_SYSTEM_PROMPT_BYTES doc at src/profile/index.ts:124-127); this just closes the silent escape hatch on top of it. No competing mechanism exists — grep for assertSystemPromptWithinBudget/composeAgentProfile shows these are the o
  • Real-world viability: Holds on realistic paths: the check is pure synchronous validation with no concurrency surface; whitespace-only/empty overBudgetReason is trimmed and rejected (src/profile/index.ts:156); lowering the cap correctly needs no reason (raisedCap is false when maxSystemPromptBytes <= default); the warnOnly+reason path still warns on actual overflow (src/profile/index.ts:215). No existing test passes a r
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 1

💰 Value Audit

🟡 overBudgetReason is validated by .trim() !== '' — trivially satisfiable [maintenance] ``

A single character ('x') satisfies the gate (src/profile/index.ts:156). This is almost certainly intentional: the goal is friction-by-decision (forcing someone to type a sentence and own it in the diff), not prose-quality policing, and heavier validation would be over-engineering. Noting only so a reviewer is aware the gate is a speed bump, not a wall — if a product later ships a lazy one-word reason, the doctrine-enforcement value comes from code review on the reason string, not from this check

🎯 Usefulness Audit

🟡 Policy check is gated on systemPrompt existing, so 'every compose' is slightly overstated [integration] ``

composeAgentProfile only calls assertSystemPromptWithinBudget when typeof systemPrompt === 'string' (src/profile/index.ts:299-300), and assertBudgetPolicy lives inside it (src/profile/index.ts:202). So a product that sets { maxSystemPromptBytes: 120_000 } (or warnOnly: true) with no reason but whose merged profile has no systemPrompt string would slip past the guard. The PR body claims the check fires 'on every compose'; it actually fires on every compose that yields a prompt. In practice ever


What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass What it asks
Heuristic Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication Do added function/class names already exist elsewhere in the repo?
Value Audit What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

value-audit · 20260724T235447Z

@tangletools

Copy link
Copy Markdown

✅ No Blockers — 72abc9bc

Review health 100/100 · Reviewer score 89/100 · Confidence 65/100 · 2 findings (2 low)

glm: Correctness 89 · Security 89 · Testing 89 · Architecture 89

Reviewer score is advisory once the run is complete and the verdict has no blockers.

Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.

🟡 LOW No direct test exercising assertBudgetPolicy through assertSystemPromptWithinBudget (the exported entry point) — src/profile/index.test.ts

The policy is tested indirectly through composeAgentProfile, and the one assertSystemPromptWithinBudget test (line 367) uses maxSystemPromptBytes:100 (below default) so it never hits the policy gate. Since assertSystemPromptWithinBudget is a separately exported function that products call from their own /prompt assembler, a direct test asserting it throws on a raised cap without reason would close the coverage gap. Low impact: composeAgentProfile delegates to it, so indirect coverage exists.

🟡 LOW Behavioral throw is a breaking change for external consumers weakening the cap — src/profile/index.ts

Any external consumer of composeAgentProfile/assertSystemPromptWithinBudget that currently passes { warnOnly: true } or { maxSystemPromptBytes > 40000 } without overBudgetReason will now throw at runtime where it previously passed. This is the deliberate enforcement and is documented in the JSDoc for overBudgetReason, but since @tangle-network/agent-app is a published peer-dependency package this warrants a changelog entry and a semver signal (minor at minimum, arguably major) so fleet apps upgrading don't hit a runtime throw in production compose paths. No in-repo caller is affected (grep confirmed).


tangletools · 2026-07-24T23:55:01Z · trace

@tangletools tangletools left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approved — 2 non-blocking findings — 72abc9bc

Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.

Full immutable report for this review: trace

Summary comment for this run: full summary


tangletools · 2026-07-24T23:55:01Z · immutable trace

@drewstone
drewstone merged commit 0a26b3c into main Jul 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants