RFC: A real-time budget decision plane for AI agent runs

[iamapsrajput]

AI agents don't consume tokens the way chat does. An agent runs a loop: observe, think, act, repeat, and each iteration resends the accumulated context. By step 20 of a run with file reads, a single call can exceed 50K input tokens. Reported cases from the past year include a developer hitting $4,200 in API fees over one weekend of autonomous refactoring. The mechanism is structural: unbounded loops resending growing context.

Existing gateway budgets attach to keys, users, or teams over daily or monthly windows. The damage unit for agents is the run: one autonomous session that needs a ceiling in dollars, not a monthly quota it can exhaust in an hour. And when today's budget checks do fire, the request dies with an opaque error the agent can't adapt to.

This RFC proposes a real-time budget decision plane: a run-scoped budget authority that atomically reserves estimated spend before provider calls (reserve → commit → refund, so parallel tool calls can't race past a ceiling), reconciles against actual usage after calls, fails closed on unknown prices, and exposes machine-readable budget state so agents can downshift to cheaper, capability-valid models before they exhaust the run.

Full document: RFC

Delivery is planned as a LiteLLM pre-call hook first (composes with existing deployments, replaces nothing), a standalone sidecar later. Apache 2.0.

Where I most want pushback

Reply to any of these by number:

1. Would your team ever enable blocking for production agent runs, or is auto-downgrade the practical ceiling of enforcement? What guarantees would blocking require beyond the correctness section (RFC §5)?
2. How do you identify an agent run today: header, OTel trace/span, framework-level ID, or not at all?
3. Which enforcement mode would you actually start with? Is actuals_only (no estimation trust required) the necessary on-ramp?
4. Is worst-case reservation (effective_max_output_tokens basis) acceptable, or does it over-reserve so aggressively for parallel workloads that soft-gate margins are preferable?
5. Would clients tolerate headers-only budget state on successful responses, with full state behind a decision lookup endpoint, or do they need the opt-in response envelope?
6. Where should this live: gateway plugin, sidecar, or SDK middleware inside the agent framework?
7. What's missing from the problem-detail schema for your agents to act on it automatically?

Background: this design grew out of the budget layer of an LLM router I've built and maintained since 2025. I'll keep RFC.md updated as feedback lands; a changelog at the bottom of the document tracks what changed between versions.