Repository navigation
Dollar caps and request dedup for rig agents (BudgetHook, OrchestratedTransport) #3319
Closed
Mattbusel
started this conversation in
Show and tell
Replies: 1 comment
|
You should open a PR to https://github.com/0xPlaygrounds/awesome-rig |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Two small crates that plug into rig 0.46, in case they help anyone here.
llm-budget has a
rigfeature withBudgetHook, anAgentHookthat puts a hard dollar cap on agent runs. It reserves an estimate before each model turn, settles the real cost afterwards (rig's reported cost first, otherwise a price table with cache rates), and stops the run on the turn that would cross the cap. Agents can share one budget, and once it is spent the next agent is stopped before its first call.Same idea as
run_budget.rs, but in dollars and across models. Full example:examples/rig_budget_hook.rs.tokio-prompt-orchestrator 2.1+ has
OrchestratedTransport, which wraps a rig provider transport so identical in-flight requests share one upstream call and a circuit breaker fails fast while the provider is down. Handy when you fan out lots of agents that ask the same things.Both are MIT and I wrote them. I'd love feedback on the hook semantics, especially what should happen to a turn that is already in flight when the shared budget runs out.
https://github.com/Mattbusel/llm-budget
https://github.com/Mattbusel/tokio-prompt-orchestrator
All reactions