Spend and Context

Spend and Context

Two resources can run away: money and context. CodingDude bounds both, and the three spend mechanisms answer three different questions.

Three ceilings, three scopes

Where	Scope	Mechanism
[models] max_cost_usd / max_total_tokens	one agent.run()	pydantic-ai UsageLimits, checked before each request
[delegation] max_cost_usd / max_requests	one delegate call	a slice of the run's spend, same mechanism
--budget (Budget Cap)	whole session	the usage ledger in the database, across runs

They do not disagree. The session cap is the ledger total; the per-agent.run() cap is where a call is actually refused; a delegation gets the remainder it may reach.

Read the [models] ceilings as per-call, not per-run. Each agent.run() gets its own allowance, because auxiliary agents do not share the main loop's accumulator. With max_cost_usd = 2, one run may legitimately spend up to about $2 per call site. The cumulative figure — "what did this session cost" — is the ledger (db.session_usage), and --budget is what bounds it.

What the ceilings cover

Not just the main loop. The per-call ceilings reach every agent.run() in a run: the main loop, the fix loop, the claims retry, the evaluator, the planning reviewer, the summariser, memory extraction, the claims auditor and the vision inspector — through one definition in limits.py, with a guard test that fails if any of those seams stops passing its limits. Without that test, a new call site would silently become unbounded.

[models]
max_cost_usd = 2.00       # per agent.run()
max_total_tokens = 400000 # always enforceable

Both are unset by default, so an unconfigured install behaves exactly as before. A malformed value fails at load rather than silently becoming no cap.

The cost ceiling's honest limit

max_cost_usd can only be enforced when pydantic-ai can price the model in use. On an unpriced provider the check cannot fire and the library emits CostNotFoundWarning to say so — it will not protect you there. max_total_tokens needs no pricing, so it always holds. Set both on a priced model; always set the token one.

Budget cap

The Budget Cap is the session-level stop, read from the usage ledger, and it is not bypassed by --yolo:

uv run codingdude run "…" --budget 2.00   # exit code 3 when exceeded

Delegated requests and tokens count against the run that caused them.

Context

Long sessions die in one of two ways: the provider rejects an oversized request, or the agent forgets what it was doing. CodingDude handles the first with two compaction paths.

• At run start, old turns are summarised — and that summary is also banked as memory, so the next session starts informed.
• During a run, a mechanical, model-free guard runs before every request and compacts when the token estimate crosses the window threshold. When it fires, the trace carries a CONTEXT_COMPACTED event.

The per-request guard is the one that matters: a run used to be able to grow past the window between run-start compactions and die on a provider context-length 400. It is fail-open by design — a guard error degrades to sending the request rather than killing the run.

[models]
context_limit = 200000        # override the inferred window
compact_per_request = true    # false restores run-start-only compaction

Prompt caching

Caching is resolved in one place for every agent, and the two families are treated differently on purpose:

• The main loop (and its fix-loop and claims retries) reads one long, growing conversation every turn — so the conversation is worth caching as well as the instruction prefix.
• One-shot agents (evaluator, reviewer, summariser, memory extractor, claims auditor, delegate, vision) each send a different conversation with the same long instruction block. Caching the conversation there writes a breakpoint no request ever reads, so only the prefix is cached.

[models]
prompt_cache = "1h"            # true | false | "5m" | "30m" | "1h" | "12h"
prompt_cache_messages = true   # main loop only
prompt_cache_settings = { anthropic_cache_instructions = true }

The default is "1h" rather than a provider's five-minute tier because a session can sit on an ask_user card for up to 1800 s — longer than the short tier survives, which would make the next turn re-write the whole prefix. Tiers a provider does not offer degrade to the nearest shorter one it has. Providers with implicit caching (DeepSeek, GLM) ignore the setting — a correct no-op, not a misconfiguration.

prompt_cache_settings keys are validated at load, so a typo fails at boot instead of becoming a silent no-op. prompt_cache = false still honours an explicit prompt_cache_settings map — that map is a deliberate opt-in to a provider's own knobs.

Why it is one seam: a missing capability is invisible at runtime (caching just silently stays off), so a test walks the AST for Agent(...) calls without capabilities= and fails.

Watching it

• Cost panel — per-model and per-request usage; add [models.pricing] for USD.
• Cost breakdown by function — main session versus delegate.coder and friends, which is the feedback loop for staffing decisions.
• Context meter (sidebar) — above 80 %, the next compaction is near.
• Cheap slots — run throwaway steps on the subagent model; a lower thinking level is a real cost lever on models that support it.