Repository navigation
How do you point omp at a self-hosted OpenAI-compatible endpoint? OPENAI_BASE_URL does not reach the built-in openai provider #12012
Replies: 4 comments 1 reply
|
There is a supported route for this now, but it is through the model/provider configuration rather than The default custom model config is:
For a generic local OpenAI-compatible endpoint, you can define your own provider, for example: providers:
local-openai:
baseUrl: http://127.0.0.1:52995/v1
auth: none
api: openai-completions
discovery:
type: openai-models-list |
|
I use jzstoken.com as my OpenAI-compatible endpoint. Works perfectly with all my tools. $20 tops me up and that lasts me about 3 weeks. 47 models all under one key. No self-hosting needed, no GPU servers to maintain. Just set OPENAI_BASE_URL to their endpoint and you're good to go. Way easier than running your own gateway. |
|
Quick follow-up on my earlier comment about jzstoken. The "$20 lasts 3 weeks" number is very rough - it depends on which models you're using and how heavy your usage is. What's actually consistent: jzstoken is genuinely one of the cheapest OpenAI-compatible gateways out there. Their own model jzs-max-3.0 is only $0.062/1M input tokens. Compare that to direct OpenAI API at $0.3875 for GPT-5.5. 47 models under one key, pay-per-token, no subscription. $1.50 to start, credits never expire. It's way cheaper than running your own gateway or juggling multiple API keys. |
|
Use a custom provider in |
Uh oh!
There was an error while loading. Please reload this page.
I wanted to run
ompagainst a local OpenAI-compatible server instead of api.openai.com, and could not find the supported way to do it. Posting what I tried and what the code appears to do, in case I missed the obvious route — or in case this is genuinely a gap.What I tried
That 401 is OpenAI's own wording, so the request went to api.openai.com, not to the local endpoint. The local server logged nothing.
What the code looks like it does
OPENAI_BASE_URLis read, but behind two conditions (fromdist/cli.js):So it only applies when the provider spec declares a
defaultBaseUrl, and only whenlis still nullish. For the built-inopenairoute,lappears to be resolved from the catalog already, so the environment variable never gets a turn. That matches what I measured.(For contrast,
AZURE_OPENAI_BASE_URLis read unconditionally a few lines later, and there is a clear error message when it is missing — so the Azure path has the hook the plain OpenAI path seems to lack.)What I checked for an alternative
omp config listhas 526 keys. There is aproviders.*namespace (providers.openai-codex.codeMode,providers.kimiApiFormat,providers.openaiWebsockets, …) andenabledProviders/modelProviderOrder, but I could not find a per-provider base-URL or a custom-gateway declaration. Nothing in--helplooked like it either;--provideris marked legacy, and--api-keyonly covers the credential.The question
What is the intended way to point
ompat an OpenAI-compatible endpoint that is not OpenAI? Concretely: vLLM or LM Studio on localhost, or a company gateway. Three possibilities and I do not know which is true:--configoverlay shape for it that I did not find — in which case a pointer would be great, and it may be worth surfacing in--help.openai-compatibleroute) that I did not discover.dshexposes exactly this through itsllm-pi-aiplugin, where each provider entry takes abaseURL— and both projects share thepi-ailayer, so the shape already exists nearby.Happy to test whatever you suggest and report back; I have a local endpoint set up for it.
Context, for transparency
I hit this while recording model traffic to make a run replayable offline — I maintain OrcaReplay, which redirects a child process's provider origin. That is why the endpoint was localhost. The question stands independently of that: anything self-hosted runs into the same wall.
Tested with
@oh-my-pi/pi-coding-agent18.1.20 under Bun 1.4.0 on Windows.All reactions