Recording a PraisonAI agent and replaying it offline — and the env-var precedence that surprised me

[xizhuomengcontin]

I maintain OrcaReplay (Apache-2.0), which records an agent run from outside the process and can serve the recording back so the run happens again with no provider called. I ran it against PraisonAI this evening. It works with no changes to your agent code — and on the way I found one thing about PraisonAI's own plumbing that is worth knowing if you ever point it at a local endpoint.

The finding: PraisonAI resolves the origin itself, and OPENAI_API_BASE wins

The OpenAI Python SDK only reads OPENAI_BASE_URL. PraisonAI does not delegate that decision — praisonaiagents/llm/openai_client.py resolves it by hand, with its own precedence:

self.base_url = base_url or os.environ.get("OPENAI_API_BASE") or os.environ.get("OPENAI_BASE_URL")

So OPENAI_API_BASE takes precedence over OPENAI_BASE_URL here, which is the opposite way round from what someone who has only used the SDK would assume. If both are set — and they often both are, because different tools set different ones — PraisonAI silently uses OPENAI_API_BASE. I went in expecting the SDK's behaviour and would have been debugging the wrong variable if the two had disagreed.

(There is a second, smaller one right next to it: the module-level client is cached on the normalized api_key + base_url + retry triple, so changing the environment variable inside a running process does not necessarily get you a new client. Fine in practice, surprising if you are switching endpoints mid-run.)

Worth saying plainly: this is not a bug report. Explicit resolution with a documented order is a defensible choice, and the error message at line 392 already nudges people toward OPENAI_API_BASE for local servers. It is just undocumented in the README, and it is exactly the kind of thing that costs someone an hour.

Recording a PraisonAI agent

Nothing is imported into your agent. orca record launches your script as a child process with the provider origin redirected for that process only:

npm i -g orcareplay          # Node 20+
orca record generic-openai -- python my_agent.py

from praisonaiagents import Agent

agent = Agent(instructions="You are terse.", llm="gpt-4o-mini")
agent.start("Say hello in three words.")

Then kill the network and run it again:

$ orca replay last
info replaying exchanges=1 egress=blocked
praison replied: the stub answered
info replay.done reused=1/1 exact=1 divergences=0 unmatched=0 exit=0

exact=1 means the replayed request matched the recorded one byte for byte — instructions, message assembly and parameters all reproduced — and the origin process was killed before the replay, so nothing could have quietly reached the network. PraisonAI's own agent loop and tool dispatch run for real; only the model's answer comes from the trace.

Why that is useful here specifically: a PraisonAI workflow that misbehaves on the fourth step becomes something a maintainer can reproduce without your API key and without spending anything, and the same recording runs in CI as a regression test for instruction or tool changes.

Tested against praisonaiagents on Python 3.12, with a deterministic local origin standing in for a provider.

Two limits, stated rather than buried

• egress=blocked means model-provider egress, not network isolation. Replay stops the model call, but your agent's tools still run for real — a tool that fetches a URL fetches it again, and with tools_run_on= the sandbox is still doing real work. It is not a sandbox.
• A matching replay is not a determinism result. The model is not re-asked; its recorded answers are served back. Whether a fresh run would make the same choices is a different question that replay cannot answer.

Happy to open a docs PR for the OPENAI_API_BASE precedence note if that would be useful — say the word rather than me adding noise to your PR queue.