feat(providers): add openai-compatible provider with custom URL and API key#1287
feat(providers): add openai-compatible provider with custom URL and API key#1287huangjunxin wants to merge 4 commits into
Conversation
…PI key Add a new "openai-compatible" provider that accepts a custom base URL and API key to access any OpenAI-compatible API endpoint. The URL is required (via --url flag, config, or interactive prompt) while the API key is optional for local servers without authentication. Supports env vars: OPENAI_COMPATIBLE_API_KEY, OPENAI_COMPATIBLE_BASE_URL, and OPENAI_COMPATIBLE_MODEL.
The openai-compatible provider supports keyless local servers. The TUI picker now shows "(configured)" when a base URL is set even without a key, and the key resolver returns an empty string instead of erroring.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for an 'OpenAI Compatible' provider across the CLI and TUI, allowing users to configure custom base URLs and optional API keys for local or third-party services. The changes include updates to configuration structures, environment variable handling, and the provider picker UI. Feedback suggests simplifying the base URL assignment logic in the CLI to reduce redundant operations and unifying validation in the prompt_base_url function to ensure empty inputs are rejected consistently across both interactive and non-interactive modes.
| if provider == ProviderKind::OpenaiCompatible { | ||
| let provider_cfg = store.config.providers.for_provider_mut(provider); | ||
| if let Some(url) = url.clone() { | ||
| provider_cfg.base_url = Some(url); | ||
| } else if provider_cfg.base_url.is_none() { | ||
| let entered = prompt_base_url(slot)?; | ||
| provider_cfg.base_url = Some(entered); | ||
| } | ||
| } else if let Some(url) = url.clone() { | ||
| let provider_cfg = store.config.providers.for_provider_mut(provider); | ||
| provider_cfg.base_url = Some(url); | ||
| } |
There was a problem hiding this comment.
The logic for handling the base URL can be simplified to avoid redundant clones and multiple calls to for_provider_mut. Since url is an owned Option<String> from the match arm, you can use it directly without cloning in an if/else if structure. This also makes the intent clearer: if a URL is provided via CLI, use it; otherwise, if it's an OpenAI-compatible provider and no URL is configured, prompt for one.
let provider_cfg = store.config.providers.for_provider_mut(provider);
if let Some(u) = url {
provider_cfg.base_url = Some(u);
} else if provider == ProviderKind::OpenaiCompatible && provider_cfg.base_url.is_none() {
let entered = prompt_base_url(slot)?;
provider_cfg.base_url = Some(entered);
}| fn prompt_base_url(slot: &str) -> Result<String> { | ||
| use std::io::{IsTerminal, Write}; | ||
| eprint!("Enter base URL for {slot}: "); | ||
| io::stderr().flush().ok(); | ||
| if !io::stdin().is_terminal() { | ||
| // Non-interactive: read directly without prompting twice. | ||
| return read_api_key_from_stdin(); | ||
| } | ||
| let mut buf = String::new(); | ||
| io::stdin() | ||
| .read_line(&mut buf) | ||
| .context("failed to read base URL from stdin")?; | ||
| let url = buf.trim().to_string(); | ||
| if url.is_empty() { | ||
| bail!("empty base URL provided — openai-compatible requires a base URL"); | ||
| } | ||
| Ok(url) | ||
| } |
There was a problem hiding this comment.
The prompt_base_url function currently skips validation when stdin is not a terminal. It's better to unify the reading and validation logic to ensure that an empty URL is never accepted, regardless of whether the input is interactive or piped. Additionally, using read_api_key_from_stdin for a URL is semantically confusing, though it works if it simply reads a line.
| fn prompt_base_url(slot: &str) -> Result<String> { | |
| use std::io::{IsTerminal, Write}; | |
| eprint!("Enter base URL for {slot}: "); | |
| io::stderr().flush().ok(); | |
| if !io::stdin().is_terminal() { | |
| // Non-interactive: read directly without prompting twice. | |
| return read_api_key_from_stdin(); | |
| } | |
| let mut buf = String::new(); | |
| io::stdin() | |
| .read_line(&mut buf) | |
| .context("failed to read base URL from stdin")?; | |
| let url = buf.trim().to_string(); | |
| if url.is_empty() { | |
| bail!("empty base URL provided — openai-compatible requires a base URL"); | |
| } | |
| Ok(url) | |
| } | |
| fn prompt_base_url(slot: &str) -> Result<String> { | |
| use std::io::{IsTerminal, Write}; | |
| eprint!("Enter base URL for {slot}: "); | |
| io::stderr().flush().ok(); | |
| let url = if !io::stdin().is_terminal() { | |
| read_api_key_from_stdin()? | |
| } else { | |
| let mut buf = String::new(); | |
| io::stdin() | |
| .read_line(&mut buf) | |
| .context("failed to read base URL from stdin")?; | |
| buf.trim().to_string() | |
| }; | |
| if url.is_empty() { | |
| bail!("empty base URL provided — openai-compatible requires a base URL"); | |
| } | |
| Ok(url) | |
| } |
Security review (deferred from v0.8.26 hotfix queue)Doing a careful pass before merge per the v0.8.26 handoff's note that this PR needs security review. Overall the integration is well-structured and consistent across the 10 files; flagging the following before this lands: URL-shape validation
A small Userinfo / credential leakage in logsIf a user pastes Empty API key behavior
|
…tests - Add validate_openai_compatible_url() that rejects empty, whitespace-only, and non-http/https URLs at config time instead of failing at first request - Strip embedded userinfo (user:password@) from base URLs before persisting - Simplify auth-set URL-handling to avoid redundant clones and borrows - Fix prompt_base_url to validate in both TTY and non-TTY paths - Add read_line_from_stdin() helper for single-line stdin reads - Add unit tests for URL validation (empty, scheme rejection, userinfo, whitespace trimming) - Add TUI config test for OPENAI_COMPATIBLE_BASE_URL/API_KEY/MODEL env vars
|
This PR was opened before the v0.8.41 rebrand and is now stale. Feel free to rebase onto current |
2 participants
Summary
Add a new
openai-compatibleprovider type that allows users to connect to anyOpenAI-compatible API endpoint by providing a custom base URL and optional API key.
This covers self-hosted LLM servers (e.g. vLLM, SGLang), third-party API proxies, and the official OpenAI API.
deepseek auth set --provider openai-compatible --url <BASE_URL> [--api-key <KEY>]OPENAI_COMPATIBLE_API_KEY,OPENAI_COMPATIBLE_BASE_URL,OPENAI_COMPATIBLE_MODEL[providers.openai_compatible]withapi_key,base_url,model/providerpicker shows "OpenAI Compatible" with "(configured)" when URL is setChanges
10 files, +238 / -16 lines:
crates/config/src/lib.rsOpenaiCompatibleinProviderKind,ProvidersToml, env forwarding, skip model normalizationcrates/cli/src/lib.rsOpenaiCompatibleinProviderArg,--urlonauth set, optional-key prompt, env varscrates/secrets/src/lib.rsOPENAI_COMPATIBLE_API_KEYenv mappingcrates/tui/src/config.rsOpenaiCompatibleinApiProvider, picker, env forwarding, empty key allowedcrates/tui/src/client.rsOpenaiCompatiblein reasoning-effort match arms (no-op, like Ollama)crates/tui/src/commands/provider.rscrates/tui/src/tui/provider_picker.rscrates/tui/src/tui/ui.rscrates/tui/src/core/engine.rscrates/tui/src/main.rsTest plan
cargo fmt --check— cleancargo clippy --workspace --all-targets --all-features— cleancargo test --workspace— all tests pass