Replies: 1 comment
|
Having run scheduled agents under hard spend ceilings for a while, the split that worked for us was this. The CLI knows per-turn usage, but only the orchestrator can hold a cross-round budget, because the runaway you describe (11 fix rounds re-reading 650k tokens) is invisible at any single turn. Each round looked reasonable on its own. The loop was the problem, so the cap belongs where the loop lives, and the per-pane usage Herdr can see is exactly the feed it needs. Two sizing notes that saved us pain. Size the ceiling as a belt, not a governor, roughly 10 to 50x a normal cycle, because a tight cap aborts legitimately heavy runs and ends up disabled. And log per-run spend to a ledger even before capping anything. The trend is the early warning, and ours alarms on a rolling-median comparison rather than an absolute number, since slow growth is the common failure and the blowout is rare. Reference architecture where this is documented: Context is a budget, not a constant |
Uh oh!
There was an error while loading. Please reload this page.
Herdr doesn't report any token or cost figure, and that makes unattended loops hard to bound safely.
The shape of the problem: a build → review → fix loop that fails to converge doesn't crash, it just keeps paying. I had a builder reused across 11 fix rounds, re-reading roughly 650k tokens of context each round, before anyone noticed. The usual guard is a budget cap on the loop, and I can't express one —
agent get,agent listandapi snapshotcarry no usage field.What I use instead is dispatch count, which is a poor stand-in: it can't tell a 20-token reply from a 200k-token context re-read, and those differ by four orders of magnitude in cost. The gap is widest exactly when it matters most, because a runaway loop is the case where per-round cost is highest and least predictable.
The information does exist a layer down — every agent CLI Herdr wraps tracks its own turn usage and most display it. Herdr is the only component positioned to see it per pane. I don't have a view on what the right surface would be, or whether this belongs in Herdr at all versus being something orchestrators should get from the CLIs directly. Mostly I want to know whether it's on the map, since spend is the failure mode I actually worry about with unattended runs.
All reactions