Custom pi-ai routes lose catalog-known modalities — inheritance is keyed by provider route, not model id

[aka-danielZhang]

中文 TL;DR：自定义 OpenAI 兼容路由（如 sub2api 反代 OpenAI 官方模型 id）下，即使已安装的 pi-ai catalog 明确记录了该 model id 支持 image 输入，模态继承也只按 provider 路由键查找；查不到就静默回退 ['text']，导致带图会话无法切换到该模型。建议为手写路由增加一层按 model id 的 catalog 继承。

Setup

A custom OpenAI-compatible route fronted by a translation proxy (sub2api) that serves genuine upstream model ids. The route key is a private one, so every model entry is hand-declared with just an id:

llm-pi-ai:
  providers:
    my-gateway:
      apiKeyEnv: MY_GATEWAY_API_KEY
      api: openai-completions
      baseURL: https://sub2api.example.com/v1
      models:
        - id: gpt-5.2
        - id: gpt-5.6-sol
        # ...

Symptom

• A session that already contains images cannot switch its model to any model on this route — selectModel refuses with "Model ... does not accept image input, but this session already contains images; select an image-capable model."
• Sending a new image is refused by prompt with MODEL_DOES_NOT_SUPPORT_IMAGES → "当前模型不支持图片，请切换支持图片的模型".

Both gates live in packages/host/apiproxy/src/api-proxy.ts and call ctx.llm.resolveModelInfo(provider, model).

Root cause

resolveRouteModels in packages/llm/llm-pi-ai/src/catalog.ts resolves input modalities as:

input: declaredInput(entry.input) ?? base?.input ?? [...request.defaultInput]

where base = catalogModels(provider).get(entry.id) and catalogModels(provider) indexes the installed catalog by the provider route key. A hand-declared route key that pi-ai does not ship yields an empty defaults map, so even model ids the installed catalog fully describe inherit nothing and fall through to DEFAULT_INPUT = ['text'] (packages/llm/llm-pi-ai/src/config.ts).

The part that stings: with pi-ai 0.82.1, the generated catalog already knows these ids — MODELS['openai']['gpt-5.6-sol'] (and every other gpt-5.x) records input: [text, image] with contextWindow: 272000. The information is present in the very package the adapter imports; only the route-key indirection loses it. Nothing interrogates the gateway, which is a deliberate posture — but here the answer is already local.

For contrast: routes whose key happens to match an installed catalog provider inherit modalities per model id, which is why one settings file can work for those routes and break only for the hand-declared key.

Workaround (works today, hot-reloads)

• route-level defaultInput: [text, image] covering every undeclared model on the route, or
• per-model input: [text, image], or
• naming the route key openai so it becomes a catalog route (keeps the custom baseURL) — at the cost of the custom display name, and non-catalog entries (e.g. gpt-image-*) still fall back.

Suggestion

For hand-declared routes (route key not shipped by the installed catalog), add one model-id-keyed inheritance layer between base?.input and route.defaultInput:

• if the entry's id exactly matches a single model in the installed catalog across all providers (or models all from one provider), inherit its input — and possibly contextWindow / maxTokens / reasoning capability — as the next resolution layer;
• ambiguous ids (the same id under multiple providers) keep today's behavior.

This keeps the conservative ['text'] floor for genuinely unknown models — under-claiming refuses an image up front and names the model, while over-claiming gets rejected mid-turn after the message is durable — while proxies serving genuine upstream model ids keep the correct modalities for free.

Related: #356 covers the same admission gate for hand-declared models in general; this discussion is the narrower case where the installed catalog already knows the exact model id and the route-key indirection is the only thing dropping it.

[zoahdev]

Confirmed at source level and implemented as a cherry-pick-ready branch.

Source audit (master 47f9438)

packages/llm/llm-pi-ai/src/catalog.ts (resolveRouteModels) resolves base = catalogModels(provider).get(entry.id) - keyed by the provider route only. A private route key (e.g. my-gateway in front of a translation proxy) never matches the installed catalog, so base is undefined and input modalities fall back to [...request.defaultInput]: image-capable ids are reported as text-only, exactly as you described.

Patch: fix/pi-ai-catalog-model-id-inheritance

https://github.com/zoahdev/deepseek-harness/tree/fix/pi-ai-catalog-model-id-inheritance

• Adds an id-level catalog fallback for input modalities: declared entry input > provider-catalog input > global id-catalog input > route default.
• The id lookup is used ONLY for capability facts - api/baseUrl still come from the route, so a foreign catalog entry can never leak its wire protocol.
• The global id index is cached per model id (stable catalog, O(1) repeat lookups).

Verification

• tsc -b tsconfig.host.json - clean
• catalog.spec 53/53, including a new regression test: a custom route declaring an anthropic vision model id keeps vision.input while api stays openai-completions

This also unblocks the #1986 family (image sessions switching to custom routes) for pi-ai-backed gateways. Thanks for the precise root cause - the "inheritance keyed by provider route, not model id" framing made the fix one line plus a cache.