Repository navigation
GPT Researcher on an OpenAI-compatible endpoint: OPENAI_BASE_URL, the three LLM slots, embeddings, and why custom model IDs can trip temperature/reasoning handling #2244
apigoto
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I work on the APIGOTO team. APIGOTO is an independent OpenAI-compatible gateway, not affiliated with OpenAI, Anthropic, Google or the GPT Researcher project. The docs already cover "Custom LLM" via
OPENAI_BASE_URL. These notes add the parts I had to read the source for:gpt_researcher/config/variables/default.py,llm_provider/generic/base.py,memory/embeddings.pyandutils/llm.pyonmain(v3.7.0)..envGet the IDs from
GET https://api.china-aiapi.com/v1/models. The same ID can go in all three slots while testing.What the code does with it
openai:LLMs,GenericLLMProvidercopiesOPENAI_BASE_URLintoopenai_api_basefor LangChain'sChatOpenAI, unless you already passed one inLLM_KWARGS. It also setsstream_usage=True, so cost tracking uses real usage if the endpoint returns it in the stream.EMBEDDING=openai:..., the embeddings class also picks upOPENAI_BASE_URL. So embeddings go to the same gateway, which only works if that gateway serves the embedding model you named.EMBEDDING=custom:...is the other route. It also readsOPENAI_BASE_URL, but defaults tohttp://localhost:1234/v1and setscheck_embedding_ctx_length=False, which is meant for LM Studio-style local servers.The model-name trap
utils/llm.pydecides two things by model name:reasoning_effort(SUPPORT_REASONING_EFFORT_MODELS), andtemperature(NO_SUPPORT_TEMPERATURE_MODELS).A gateway model ID that doesn't match those lists gets default treatment, so it receives
temperatureand noreasoning_effort. If the backend model is a reasoning model that rejectstemperature, you get an "unsupported parameter" error that looks like a gateway problem but isn't. Options:TEMPERATURE/LLM_KWARGSand test the slot on its own.Token limits (
FAST_TOKEN_LIMIT,SMART_TOKEN_LIMIT,STRATEGIC_TOKEN_LIMIT) default to 6000/12000/8000. Lower them if your model's output limit is smaller.Troubleshooting
OPENAI_BASE_URLnot loaded in the process that runs the server (Docker env, uvicorn started from another shell).404:/v1missing or doubled.RETRIEVER=tavily(the default) withoutTAVILY_API_KEY.DEEP_RESEARCH_CONCURRENCY(default 4) andMAX_SCRAPER_WORKERS.On the gateway side: one base URL for several model families, automatic failover on upstream errors (at most 3 retries per request on another route), a per-request log, pay-as-you-go, BYOK: www.china-aiapi.com
How do others split FAST/SMART/STRATEGIC when all three come from one endpoint: same model, or three different ones?
All reactions