fix(llm): gate OpenAI/Compatible supports_vision on model capability#6423
Merged
Conversation
OpenAiProvider::supports_vision() unconditionally returned true regardless of the configured model, so a text-only model (e.g. gpt-3.5-turbo, or an arbitrary text-only compatible endpoint) would have images attached and sent, typically failing with a 400. CompatibleProvider inherited the same bug via delegation. supports_vision() now resolves via an explicit vision override (OpenAiConfig::vision / CompatibleConfig::vision, or the with_vision builder) when set, otherwise a built-in model-name prefix table for known OpenAI vision-capable families (gpt-4o, gpt-4-turbo, gpt-4-vision, gpt-4.1, gpt-5, and vision-capable o-series models, explicitly excluding text-only o-series reasoning variants like o1-mini/o1-preview/o3-mini). Unrecognised models fail safe to false, which is also the default for arbitrary OpenAI-compatible endpoints unless vision is explicitly configured. ProviderEntry.vision is wired through provider_factory.rs so operators can override via config.toml. Closes #6411
bug-ops
enabled auto-merge (squash)
July 17, 2026 21:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OpenAiProvider::supports_vision()unconditionally returnedtrueregardless of the configured model, so a text-only model (e.g.gpt-3.5-turbo, or an arbitrary text-onlytype = "compatible"endpoint) would have images attached and sent, typically failing with a 400.CompatibleProviderinherited the same bug via delegation.supports_vision()now resolves via an explicitvisionoverride (newOpenAiConfig/CompatibleConfigfield,with_visionbuilders, and a newvisionfield on[[llm.providers]]entries) falling back to a built-in model-name prefix table for well-known OpenAI vision families (gpt-4o,gpt-4-turbo,gpt-4-vision,gpt-4.1,gpt-5, vision-capableo-series), explicitly excluding text-only reasoning variants (o1-mini,o1-preview,o3-mini). Unrecognised model names fail safe tofalse— this is also the default for arbitrarycompatibleendpoints unlessvision = trueis set explicitly.Closes #6411
Test plan
cargo +nightly fmt --check,cargo clippy --profile ci --workspace --all-targets --features "desktop,ide,server,chat,pdf,scheduler,testing" -- -D warnings,cargo nextest run --config-file .github/nextest.toml --workspace --features "desktop,ide,server,chat,pdf,scheduler" --lib --bins(14217 passed, 0 failed, 35 skipped; one unrelated concurrency test flake confirmed non-reproducing on re-run), rustdoc gate withRUSTFLAGS="-D warnings" RUSTDOCFLAGS="--deny rustdoc::broken_intra_doc_links"— all clean.cargo test --doc --workspace— all 32 crates pass, including newwith_visiondoctests onOpenAiProvider/CompatibleProvider.zeph-llm,zeph-config) — clean.gpt-3.5*false, unrecognised-model fail-safe false, explicit override in both directions, text-onlyo-series exclusion (o1-mini/o1-preview/o3-mini),CompatibleConfig.visionconfig-field forwarding (distinct from the.with_vision()builder path).gpt-4o-miniOpenAI provider confirmed the request reaches the server and is rejected only with429(rate-limited, a known pre-existing test-account constraint, unrelated to this change) — never a400/422, confirming the request payload is well-formed. Verified in source that400responses are routed through a distinct error path (map_error_response→LlmError::ApiError) never conflated with the429/503retry loop that was actually hit.