Repository navigation
OpenAIModel + any OpenAI-compatible endpoint: when to use ToolCallingAgent vs CodeAgent, and which knobs fix gateway quirks #3086
apigoto
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I work on the APIGOTO team. APIGOTO is an independent OpenAI-compatible gateway, not affiliated with OpenAI, Anthropic or Google. These notes are for anyone running smolagents through a non-OpenAI base URL (a gateway, vLLM, LM Studio, a corporate proxy). In current smolagents the class is
OpenAIModel, andOpenAIServerModelis kept as an alias, so older tutorials still run.Install and connect
pip install 'smolagents[openai]'A tool both agent types can use
The docstring's
Args:section is required: smolagents builds the tool schema from it.ToolCallingAgent: native tool calls
The model must return proper
tool_callsin the chat completions response. If the endpoint or model returns text instead of tool calls, this agent stalls or loops untilmax_steps.CodeAgent: tools called from generated Python
It needs no native tool-calling support, only a model that writes clean Python in the expected code block. So it's the safer first try for a model whose function calling you haven't verified. It can also batch several tool calls in one step. Two cautions:
executor_type="docker"(or e2b, modal, blaxel).Knobs for gateway quirks
use_structured_outputs_internally=True(CodeAgent, defaultFalse) sends aresponse_formateach step. Enable it only if the model behind yourMODEL_IDsupports JSON-schema output; otherwise leave it off.flatten_messages_as_text=True: for endpoints or models that reject list-style messagecontentand only accept plain strings.custom_role_conversions={"system": "user"}: for models that refuse thesystemrole.requests_per_minute=...throttles client-side.retry=True(default) only retries rate-limit errors (429 / "rate limit" / "too many requests").Troubleshooting
ModuleNotFoundError ... pip install 'smolagents[openai]': the extra is missing./v1missing or doubled inapi_base.model_iddoesn't match/models.max_stepswithout a final answer: try the other agent type before blaming the model.On retries: our gateway retries a failed upstream call on another route, at most 3 times, so
max_retries: 1inclient_kwargsavoids stacking a second full retry loop. Per-request logs show each step's request, which makes step-by-step agent debugging easier. Pay-as-you-go, BYOK: www.china-aiapi.comWhich agent type do you default to with models whose tool calling you haven't checked yet?
All reactions