Bug: adopted vision models lose input modalities during discovery (MODEL_DOES_NOT_SUPPORT_IMAGES)

[Ricardo-M-L]

Summary

When a model is adopted through the settings page's fetch action, it loses its input-modality declaration. The discovery reply and wire view carry only id/name/contextWindow/maxTokens, and adoption writes no input field — so a vision model (e.g. Kimi K3 behind a custom route) materializes with the route fallback ['text'], and every image prompt is refused with MODEL_DOES_NOT_SUPPORT_IMAGES.

Root cause

• LlmDiscoveredModel and the llm.discoverModels wire view don't expose inputModalities.
• The settings-page adoption path doesn't write the input field.

Proposed fix

I have a working fix on a fork branch (9 files, +65/−3):

https://github.com/Ricardo-M-L/deepseek-harness/tree/fix/discovered-model-modality

• Threads inputModalities through model discovery and the wire view.
• The pi-ai catalog short-circuit now answers with the installed entry's modalities.
• The settings-page fetch path keeps the vision capability on adopted models.

I understand external PRs aren't accepted right now, so I'm reporting here for visibility. Happy to open a PR whenever the repo accepts them.

[zoahdev]

核验通过（对照 main/master HEAD 99f6f02 rc.7）——现象与影响面成立，根因描述需要一处细分，补丁范围可以更精确。

1. 现象链路成立。 packages/client/ui-settings-models/src/client/ModelListEditor.tsx:146-154 的 adopt() 只拷贝 id/name/contextWindow/maxTokens，不写 input——被采纳的模型草稿没有模态声明，落库后按 route fallback ['text'] 处理，图片输入被 MODEL_DOES_NOT_SUPPORT_IMAGES 拒绝。✅

2. 根因细分（一处修正）：不是类型不支持，而是两处实现没填充 + 客户端采纳丢弃。 LlmDiscoveredModel 类型已经声明 inputModalities?: readonly ModelModality[]（packages/llm/llm/src/types.ts:243），且 LlmRuntime.discover 在 adapter 提供时会附上（llm/src/index.ts:599-605）。真正的缺口是 pi-ai 的实现：

• readListing（llm-pi-ai/src/discovery.ts:138-162）解析端点 /models 回复时只读 id/name/context_window/context_length/max_output_tokens/max_tokens，不读模态字段；
• 目录短路（discovery.ts:190-197）更典型：已安装 catalog 条目明明带 input（如 Kimi K3 经 pi-provider），短路却只返回 id/name/contextWindow/maxTokens——模态在这里被显式剥离；
• 客户端侧即使 wire 带上字段，adopt()（ModelListEditor.tsx:146-154）也会丢。

所以修复 = 四处贯通：① readListing 读模态；② 目录短路透传 installed entry 的 input；③ wire view 增加字段；④ adopt() 拷贝 inputModalities。你的分支 9 个文件 +65/−3 正好覆盖这四处的形态，方向正确。

3. 补丁范围建议： 若 endpoints 的 /models 回复没有标准模态字段，readListing 侧可保守（有则透传、无则缺省）；目录短路是必改项（数据已在手边，零猜测）。建议至少包含目录短路 + adopt 两处，README 注明"发现链路仅在目录/端点声明模态时保留"。

4. 回归测试形状： 参照 catalog.spec.ts:247-257 已有的 resolveModelInfo 模态断言，补 discovery 层断言（目录短路返回含 input 的条目）+ ModelListEditor 采纳单测（adopt 保留 inputModalities）。

需要的话，我可以按之前的惯例拉你的分支做独立验证 + 跑相关包测试，然后贴结果。

[zoahdev]

更正我上一条回复中的一处错误（核验时把两个接口混淆了，向作者和读者致歉）：

LlmDiscoveredModel 在 main 99f6f02 上并没有 inputModalities。 我之前引用的 llm/src/types.ts:243 字段属于 LlmModelInfo（resolved model info，inputModalities?: readonly ModelModality[]），不是 LlmDiscoveredModel。OP 的根因描述"LlmDiscoveredModel 不暴露 inputModalities"是正确的；缺口不仅在两处实现没填充，类型层本身也没有——需要类型、discovery 实现、wire view、客户端采纳四处贯通，与你分支的 9 文件改动完全对应。我的"类型已有、只需填充"的说法作废。

独立验证已完成（拉取 Ricardo-M-L/fix/discovered-model-modality 631eecc，rebase 到 main 99f6f02 后逐处核对）：

• rebase 分支：fix/discovered-model-modality-rebased（zoahdev fork，两提交 eac57cc + ee3659b）
• 验证：vitest run llm-pi-ai + llm + ui-settings-models → 41 files / 854 tests 全绿；host 全树 tsc -b typecheck exit 0
• 修正后分支包含全部 9 个文件（类型 + runtime discover 映射 + discovery 目录短路 + wire view + api-proxy + ModelListEditor adopt + 复选框 UI + CSS + 双语文案）

已登记 zoahdev upstream patch queue（dsh-docs #46）。PR 通道开放时可 cherry-pick。

[nhype]

I reproduced the same user-visible failure with https://api.modelrouter.org/v1. Its OpenAI-compatible /models response exposes OpenRouter-style modality metadata such as architecture.input_modalities and capabilities.vision, rather than the top-level inputModalities field that the current parser understands. As a result, discovered models remain modality-unknown and are later reported as not supporting images.

I preserved @Ricardo-M-L's propagation fix as the first, separately attributed commit and added a follow-up for this gateway response shape:

• Branch: https://github.com/nhype/deepseek-harness/tree/fix/openrouter-discovered-modalities
• OpenRouter metadata commit: nhype@8c5b55e
• Ricardo's original branch: https://github.com/Ricardo-M-L/deepseek-harness/tree/fix/discovered-model-modality

The follow-up:

• recognizes supported values from architecture.input_modalities and returns them in canonical catalog order;
• treats capabilities.vision: true as a fallback when architecture metadata provides no recognized modality;
• keeps explicit architecture metadata authoritative (for example, ["text"] is not upgraded merely because vision is true);
• leaves malformed or unsupported-only metadata unknown instead of guessing;
• documents the behavior in both English and Chinese package READMEs.

Validation passed:

• focused discovery regression test with per-file coverage;
• shared LLM service tests;
• full GUI tests;
• built-browser snapshot replay (DSH_SNAPSHOT=replay pnpm run test:web);
• documentation synchronization and lint;
• the repository's pre-push incremental type checks.

I am posting a fork branch rather than opening a PR because the current CONTRIBUTING.md says external pull requests are not accepted. The follow-up commit is ready to cherry-pick on top of Ricardo's propagation patch.