Replaying an ADK run offline: LiteLlm reads OPENAI_BASE_URL, so capture needs no code change

[xizhuomengcontin]

I maintain OrcaReplay (Apache-2.0), which records an agent run at the HTTP boundary to its model provider — from outside the process — and can serve that recording back so the same run happens again with no provider called and no key needed. I spent tonight checking it against ADK. Posting the result plus the one implementation detail that makes it work, since it is not obvious from the docs.

The detail that matters

LiteLlm(model="openai/gpt-4o-mini") with no base_url in code ends at the OpenAI SDK constructing a client with base_url=None, and the SDK then falls back to OPENAI_BASE_URL from the environment. orca record sets exactly that variable, and only for the child process it launches — your shell and your other agents are untouched.

So an ADK agent is captured with no changes to the agent at all:

agent = Agent(
    name="orca_demo",
    model=LiteLlm(model="openai/gpt-4o-mini"),   # model id only — no base_url
    instruction="You are terse. Answer in the fewest possible words.",
)

$ orca record generic-openai -- python agent.py
info recording run=run_135c37894350 adapter=generic-openai proxy=http://127.0.0.1:52685
info recorded run=run_135c37894350 events=6 exit=0

Then, with the origin process killed so there is nothing upstream to reach:

$ orca replay last
info replaying run=run_135c37894350 exchanges=1 egress=blocked
info replay.done reused=1/1 exact=1 divergences=0 unmatched=0 exit=0

exact=1 means the replayed request matched the recording byte for byte — instruction, message assembly, tool schemas and parameters all reproduced. InMemoryRunner, the event loop and tool dispatch execute for real; only the model's answer comes from the trace. If something had drifted it would be reported as a divergence with its size, rather than normalized into a pass.

Practical consequence: a misbehaving ADK session becomes a file a colleague can re-run without your key and without spending tokens, and the same file is a CI regression test for instruction, tool-schema or orchestration changes — the part of testing an agent that is otherwise awkward, because real calls cost money and do not answer the same way twice.

What I have not verified

The native Gemini path (model="gemini-2.0-flash" through google-genai) is a different client that does not read OPENAI_BASE_URL, and I have not tested it, so I am not claiming it. From reading the SDK it looks like it would need its own origin override rather than the OpenAI variable. If someone here knows whether google-genai honors an endpoint override cleanly, that is the gap I would most like to close — and I would rather ask than guess and post something wrong.

Limits, stated up front

• egress=blocked means model-provider egress, not network isolation. Replay calls no provider, but it re-executes the recorded tool calls for real — an ADK tool that fetches a URL or writes to a database does it again. It is not a sandbox; run it in one if that matters.
• A matching replay is not a determinism result. The model is not re-asked; its recorded answers are served back. Whether a fresh run behaves the same way is a different question this cannot answer.
• A trace holds whatever the run held — prompts, tool results, anything pasted in. Scrubbing is best-effort against known key shapes; it cannot know that a hostname or customer name is confidential. Review before sharing.

I wrote this up as an integration page for adk-docs (#2229); it is parked on the Google CLA, which I am sorting out. Happy to take corrections here on the ADK side either way — particularly if the Gemini-path question above has a known answer.

[surajksharma07]

Great writeup. The OPENAI_BASE_URL fallback through LiteLlm is exactly the kind of detail that's easy to miss and useful to have documented like this. On your open question: the native Gemini path supports a clean override too via Gemini(base_url=...) (been there since v1.25.0) so worth pointing OrcaReplay at that for the google-genai side.since it's a third-party tool demo rather than a core ADK issue, closing this out. keep this kind of thing coming.