[Bug] Image upload rejected with "current model does not support images" even after switching to a non-DeepSeek model — custom pi-ai provider models silently default to text-only input

[wizardcypress]

Summary

dsh Web UI (v0.1.0-rc.3) rejects image uploads with "当前模型不支持图片，请切换支持图片的模型" (the current model does not support images; switch to a model that does). The error persists even after switching to a model on a custom OpenAI-compatible provider (added via ~/.dsh/settings.yaml under llm-pi-ai) that actually serves vision models. The root cause is that hand-declared models not present in the installed pi-ai catalog silently fall back to inputModalities: ['text'], and there is no UI surface to declare image support.

Environment

• dsh version: 0.1.0-rc.3 (npm global install, dsh web), macOS
• Configuration: custom provider under llm-pi-ai.providers in settings.yaml, with hand-declared models entries (id + name only)

Repro steps

1. In ~/.dsh/settings.yaml, add a custom provider to llm-pi-ai.providers with an OpenAI-compatible baseURL/api, and hand-declare one or more models without an input field:

   llm-pi-ai:
     providers:
       my-gateway:
         apiKeyEnv: MY_GATEWAY_API_KEY
         api: openai-responses
         baseURL: https://gateway.example.com/v1
         models:
           - id: vision-model
             name: Vision Model

2. Start dsh web, open a session, and switch the model picker to vision-model.

3. Upload an image and send.

4. The composer shows: 当前模型不支持图片，请切换支持图片的模型 (or in English: The current model does not support images; switch to a model that does).

Expected behavior

After switching to a model that supports images, the image should be accepted and sent.

Root cause

1. The prompt admission check in the host API (packages/host/apiproxy/src/api-proxy.ts) calls ctx.llm.resolveModelInfo(provider, model) and rejects the prompt when inputModalities is defined and does not include image (error code MODEL_DOES_NOT_SUPPORT_IMAGES). The same check also blocks model switching into a session that already contains images.

2. dsh-llm-pi-ai resolves a model's input modalities as: entry input → installed pi-ai catalog entry → route defaultInput, and the route defaultInput itself defaults to ['text'] (packages/llm/llm-pi-ai/src/config.ts, DEFAULT_INPUT). Every model added through the web UI's "add a custom provider" card is hand-declared and unknown to the catalog, so it always resolves to text-only — no matter what the gateway actually serves.

3. This conservative default is deliberate (see Agent Note .agents/notes/implemented/architecture/2026-08-12-pi-ai-route-default-input-modalities.md): pi-ai silently downgrades images to a "(image omitted)" placeholder for models whose input lacks image, so refusing early is safer than claiming a capability the endpoint may not have. However, the remedy — writing input: [text, image] by hand in settings.yaml — is undiscoverable for web-only users, and the model-settings editor exposes neither the per-model input field nor the route-level defaultInput.

4. The error message itself ("switch to a model that does [support images]") offers no reachable fix, because no configuration surface could make a hand-declared model image-capable at the time.

Fix

Immediate workaround (no upgrade needed, hot-reloads):

Declare image support at the route (applies to all undeclared models on the route) or per model:

llm-pi-ai:
  providers:
    my-gateway:
      # option A: route-level default for every undeclared model
      defaultInput: [text, image]
      models:
        - id: vision-model
          # option B: per-model declaration
          input: [text, image]

settings.yaml is watched by the settings service (chokidar, default on), so the change applies without restarting dsh web.

Code fix (implemented locally, ready for upstream adoption):

1. packages/client/ui-settings-models: the pi-ai model-list editor's advanced fold now has an "Image input" checkbox per model — checking it writes input: ['text', 'image'], unchecking drops the declaration. This closes the discoverability gap at the surface that creates hand-declared models. Tests added; ModelListEditor.tsx at 100% coverage.
2. packages/client/ui-conversation: the MODEL_DOES_NOT_SUPPORT_IMAGES message now points to the fix ("…or enable image input for it under Models settings" / 或在"设置→模型"中为该模型开启图片输入).
3. The Agent Note above was updated to reflect the editor change.

Validation: test:gui (3756 tests), typecheck, lint, doc-sync (28 gates), and per-file coverage all pass.

Suggestion for maintainers

• Consider whether the "undeclared = text-only" default should distinguish declared text-only from undeclared (unknown), admitting images for unknown models and letting the provider decide — or keep the conservative default and ship the UI toggle (described above) so the remedy is reachable.
• The change also leaves the error message accurate while making the fix discoverable.

[weijiafu14]

原帖给出的 input: [text, image] 是“让这个自定义模型直接吃图片”的正确修法。另一个现成路线是：保持主路由的能力声明不变，把该 OpenAI-compatible 视觉端点作为辅助眼睛，让原来的 DeepSeek 主模型继续推理。

dsh plugin --profile web add pi2dsh@0.11.0
dsh plugin --profile web add @kassing/pi-vision

export VISION_BRIDGE_BASE_URL=https://gateway.example.com/v1
export VISION_BRIDGE_MODEL=vision-model
export VISION_BRIDGE_API_KEY=$MY_GATEWAY_API_KEY

重启 Web 后选择 DeepSeek + Vision Bridge 伴生分组并直接贴图。伴生 route 负责图片准入；@kassing/pi-vision 调用上面配置的视觉模型，把分析作为 DSH 上下文注入；再从发给文本主模型的请求中移除图片块。因此无需把不吃图的主路由谎报成 image，也不会触发 MODEL_DOES_NOT_SUPPORT_IMAGES。

这条链已用真实 DSH Web 图片会话完成端到端验证。复现步骤和测试图：pi2dsh vision-bridge example。