Repository navigation
Per-run dollar budget for ADK Go agents on openaimodel (context + RoundTripper) #1718
domondi1
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
With
openaimodeland tools, onerunner.Runcan make several model calls, and if your service handles a request by running a few agents or sessions in parallel, nothing stops that request at a dollar amount. Usage is reported after each call, so a check in a callback or plugin is a step behind, and concurrent calls can all pass it at once.Here's a setup that only uses what
openaimodel.ClientConfigalready exposes (BaseURLandHTTPClient). The run's id and budget ride on thecontext.Contextyou pass torunner.Run, a smallhttp.RoundTrippercopies them into headers, and an OpenAI-compatible gateway enforces the budget before forwarding.Run the gateway next to your service (it uses your provider key):
Then per request:
Every model call made under that context, across goroutines and sessions, draws from the same $0.25. Each call reserves its worst-case cost before it's sent, so parallel calls can't overspend together. A call that doesn't fit is refused with HTTP 402 before it reaches OpenAI; the run yields an
openai: call failed: ... 402 Payment Requirederror and stops. The headers aren't forwarded upstream.inferrail work req-4812shows what the request cost.What I tested: ADK Go v2.5.0 (
openaimodel, Chat Completions), Go 1.26, inferrail 0.4.12, against a local stub upstream. I ran six sessions in parallel on one run id with a function tool. With a tight budget, two finished, four were refused before reaching the upstream, nothing was retried, and the recorded total stayed under the cap. I haven't run it from Go against the real OpenAI API yet, so reports are welcome.Caveats: Chat Completions only (
APIChatCompletions, not Responses); setMaxOutputTokens; the budget store is a SQLite file on the gateway host, so run one gateway; the model needs a price in Inferrail (gpt-4o-mini,gpt-4.1-mini,gpt-4.1built in,inferrail modelslists others, and you can add your own prices). Gemini through the nativegeminimodel isn't covered by this.Python ADK version of the same idea: google/adk-python#7386. Setup details: guide. I maintain Inferrail (open source, Apache-2.0), so take the suggestion with that in mind.
All reactions