Feature Request: Support custom OpenAI-compatible base URL for local AI backends

[edboco]

Summary

Add support for OPENAI_BASE_URL (and optionally OPENAI_API_MODEL) environment variables in .env.local so users running local AI servers (Ollama, LM Studio, llama.cpp, LocalAI, etc.) can point RA-H OS at their own infrastructure instead of requiring an OpenAI API key.

Motivation

I'm a researcher running a local AI server on my network (http://1.1.1.200:8080) and I want to use RA-H OS without sending any data to OpenAI. Many local AI backends expose OpenAI-compatible /v1/chat/completions and /v1/embeddings endpoints, so the code changes should be minimal.
Both npm packages already in the project support this:

• openai (v4.x) accepts a baseURL constructor option and respects the OPENAI_BASE_URL environment variable
• @ai-sdk/openai (v3.x) accepts a baseURL option via createOpenAI()

⠀But RA-H OS doesn't currently expose this to the user through .env.local or the configuration docs.

Proposed Changes

1 Add to .env.example and docs:

⠀

Optional: Point to a local OpenAI-compatible server (Ollama, LM Studio, LocalAI, etc.)

OPENAI_BASE_URL=http://localhost:8080/v1

OPENAI_API_MODEL=llama3

2 Pass baseURL through when initializing the OpenAI client(s) — wherever new OpenAI() or createOpenAI() is called in src/, read from process.env.OPENAI_BASE_URL if set.
3 Allow a dummy API key — local servers typically don't need a real key, but the OpenAI SDK requires the field to be non-empty. Accepting something like OPENAI_API_KEY=local without validation errors would help.
4 Optionally allow overriding the model name — local servers use different model identifiers than OpenAI (e.g., llama3 instead of gpt-4o-mini).

⠀Example .env.local for local AI

OPENAI_API_KEY=not-needed
OPENAI_BASE_URL=http://1.1.1.200:8080/v1
EMBEDDING_MODEL=nomic-embed-text
NODE_ENV=development
PORT=3000
NEXT_PUBLIC_APP_URL=http://localhost:3000
NEXT_PUBLIC_DEPLOYMENT_MODE=local
SQLITE_VEC_EXTENSION_PATH=./vendor/sqlite-extensions/vec0.dylib

Context

• This is a very common pattern — Ollama, LM Studio, LocalAI, and llama.cpp all expose OpenAI-compatible endpoints
• The underlying libraries (openai npm, @ai-sdk/openai) already support it natively
• Users who want fully local/air-gapped setups (researchers, privacy-focused users) are exactly the audience for a "local-first" tool like RA-H OS
• Related discussion in the broader ecosystem: vercel/ai#8564

⠀Environment

• RA-H OS (current main branch)
• Local AI server at http://1.1.1.200:8080 (OpenAI-compatible endpoint)
• macOS

[bradwmorris]

hey @edboco - working on this, will report back soon

[edboco]

Thank you. I just can't use RA-H until I can use local AI. But I'm very intrigued by your system.

[haocn-ops]

This is a good compatibility layer to add. A small implementation pattern that has worked well for OpenAI-compatible backends:

• expose OPENAI_BASE_URL, OPENAI_API_KEY, and OPENAI_MODEL in .env.local
• normalize the base URL once at startup: trim whitespace, remove trailing /, and expect the OpenAI-compatible /v1 prefix explicitly
• initialize the SDK with baseURL: process.env.OPENAI_BASE_URL instead of branching provider-specific code paths
• add a tiny startup check against /v1/models or a minimal chat completion so config errors are visible early

I put together a provider-neutral examples repo for this exact base_url pattern across Cursor, Claude Code, Codex, SDKs, and curl. Might be useful as reference when documenting the env vars here:

https://github.com/apinode-pro/openai-compatible-ai-gateway-examples

No real keys in the examples; everything uses env vars/placeholders.