[Bug]: post-#956 vision sidecar gaps on nemotron/mimo/kimi probes (controls deterministic)

[brunoflma]

Client or integration

Direct HTTP/API client through OpenCodex proxy (/v1/responses, /v1/models).

Area

Proxy and routing · Catalog / models · Vision sidecar

Summary

Following #956 (PR #980 merged into dev as 7343f0ba5914a96aa9eab0a1397bfa5454c286bc), deterministic image test still fails for four additional models despite noVisionModels configuration: nvidia/nvidia/nemotron-3-ultra-550b-a55b, opencode-zen/mimo-v2.5-free, opencode-zen/nemotron-3-ultra-free, and TR/moonshotai/kimi-k3-free. Each returns HTTP 200 but never answers the red-image probe correctly, whereas controls on other providers succeed with the same payload.

For clarity of labels below, TR means TokenRouter (https://api.tokenrouter[.]com/v1), NIM means NVIDIA Inference Microservices (NVIDIA NIM), and mi/mimo refers to Xiaomi MiMo-hosted items offered through the opencode-zen free route list. These are catalog adapter families, not unrelated headers.

Reproduction

Uses a 16×16 opaque red PNG converted to data:image/png;base64,<red> via a single shell helper and submitted as the only image asset.

1. ocx start --port 10100.
2. POST to /v1/responses with model=<one of the four>, a text input asking for the dominant color, and one input_image data: URL.
3. Observe whether reply is RED, GREEN, or BLUE.
4. Repeat five trials for each. Success means exactly RED; empty, refusal, or “cannot process” responses count as miss even when status is 200.
5. Run the identical probe once against anthropic/claude-sonnet-5, anthropic/claude-opus-5, DW/devworld/deepseek-v4-pro-r, TR/qwen/qwen3.8-max, google-antigravity/gemini-3.6-flash, AGR-OAI/claude-opus-5 as controls.

Version

2.10.0 (npm @bitkyc08/opencodex); this state includes #956 results yet requires cross-provider catalog + route evaluation rather than only NIM classification.

Operating system

Windows 11, Codex runtime 0.146.0 under the OpenCodex proxy.

Provider and model

• NVIDIA NIM: nvidia/nvidia/nemotron-3-ultra-550b-a55b.
• OpenCode (free group): opencode-zen/mimo-v2.5-free (MiMo= Xiaomi MiMo), opencode-zen/nemotron-3-ultra-free.
• TokenRouter: TR/moonshotai/kimi-k3-free.
• Healthy controls listed above include upstreams belonging to Moonshot (kimi), GLM lineages, DevWorld resellers, Google Antigravity/Vortex, and Anthropic Claude.
• Each failing provider exposes a local registry noVisionModels entry and the surfaced model catalog marks them as expected sidecar-capable by configuration, though actual outcomes contradict that.

Cross-references

• [Bug]: Vision sidecar never triggers for NVIDIA NIM text-only models (registry nvidia entry missing noVisionModels) #956 — correct bounded fix for NVIDIA/Kimi supports, defines the expected canonical pattern and signals that modality truth is registry + route + catalog state together.
• [Bug]: No management-plane support for provider headers — custom providers needing client fingerprints (Agent Router) 401 after headers are lost #959 — parallel management-plane header fix (merged in feat: manage provider custom headers via PATCH and ocx provider edit --headers #961) covers provider-level overrides and secrets, showing how one feature tunnel can undermine another if metadata is not broadcast consistently.

Checks

• I searched existing issues and documentation.
• I removed secrets, tokens, account details, request credentials, and personal data.

[github-actions]

_{Automated translation bookkeeping — detected language: unknown.}

[github-actions]

Issue reopened

The report now contains the information required by the automated check. Thanks for updating it.

[brunoflma]

Client or integration

Codex runtime via the local OpenCodex proxy: /v1/responses for execution and /v1/models for catalog visibility.

Area

Proxy and routing · Catalog / models · Vision sidecar · Provider adapter

Summary

Following the NVIDIA-specific fix recorded in #956 (PR #980 merged into dev as 7343f0ba5914a96aa9eab0a1397bfa5454c286bc), a similar class of regression persists for other text-only or capability-uncertain models. These models do not advertise reliable modality coverage, yet they occasionally attempt to answer image requests without any sidecar guidance and therefore deliver either empty responses, hallucinated confidence, or explicit inability statements.

Four models in my environment are confirmed to produce HTTP 200 responses without recognizing the deterministic image:

Model	Failure mode
nvidia/nvidia/nemotron-3-ultra-550b-a55b	Explicit cannot-see answer after image payload
opencode-zen/mimo-v2.5-free	Empty response body
opencode-zen/nemotron-3-ultra-free	Same cannot-see pattern as NVIDIA
TR/moonshotai/kimi-k3-free	Empty response body

Controls on the same payload and agents answer "RED" correctly: anthropic/claude-sonnet-5, anthropic/claude-opus-5, DW/devworld/deepseek-v4-pro-r, TR/qwen/qwen3.8-max, google-antigravity/gemini-3.6-flash, plus both Agent Router transports AGR-OAI and AGR. The general request-build, authentication, and proxy routing paths work; the divergence is isolated to capability attribution and sidecar invocation for the emerging providers.

This sits alongside #956 rather than duplicating it: the NVIDIA path was repaired, but the corresponding decision surface now depends on per-provider hints that are not visibly synchronized with catalog state, registry confidence, or the exported configuration consumed by the Codex client.

Reproduction

1. Install @bitkyc08/opencodex 2.10.0 and start the proxy (ocx start).
2. Feed any flailing model this red 16×16 PNG via data:image/png;base64, inside a /v1/responses call and ask for the dominant color.
3. Observe the four failure cases above return readable or blank text with no vision sidecar acknowledgement.
4. Repeat against the controls above and note the correct RED response under the same conditions.

Version and runtime

• OpenCodex: 2.10.0.
• Local proxy confirmed healthy on port 10100 with upstreams reachable.
• Reference points to [Bug]: Vision sidecar never triggers for NVIDIA NIM text-only models (registry nvidia entry missing noVisionModels) #956 merged and tested on the dev branch — this issue therefore tracks what remains uncovered after that stack.

Evidence supporting root-cause hypothesis

• /v1/models rows for the four failing models show inputModalities: ["text"] or omit the field, so Codex grips only text-side affordances before any sidecar can intervene.
• Exported custom rows also lack inputModalities unless explicitly written, while the server-side registry keeps hint fields private to its import path — causing an off-by-one between runtime behavior and what users see in dashboards or clients.
• Logs during the failing probes contain no [vision]/sidecar traces, so no arbitration agent is invoked before the upstream rejection or hallucination.

Impact

• Failed models misbehave silently or confidently wrong without operator visibility.
• Automations that assume “image input supported” route around or ignore the sidecar but hit text-only ceilings.
• [Bug]: No management-plane support for provider headers — custom providers needing client fingerprints (Agent Router) 401 after headers are lost #959’s header work resolves transport for secrets, but does not address this capability-versus-catalog shadow state.

Suggested fix

1. Introduce a canonical per-model capability source of truth that merges registry hints, manual noVisionModels, and catalog export outputs, and make the runtime and export interfaces both read from it.
2. Extend persistent e2e coverage beyond NVIDIA: at least one text-only, one native-vision, and one unknown-provider probe per adapter, asserting non-empty, accurate answer or explicit 4xx.
3. Add an integration smoke test that walks /v1/models and /v1/responses for every known provider family in the local catalog build, so missing modality flags are caught before release.

Cross-references

• [Bug]: Vision sidecar never triggers for NVIDIA NIM text-only models (registry nvidia entry missing noVisionModels) #956 — NVIDIA/NIM vision classification fix (26 text-only, 16 vision-capable audited models).
• [Bug]: No management-plane support for provider headers — custom providers needing client fingerprints (Agent Router) 401 after headers are lost #959 — Provider headers management gap merged under feat: manage provider custom headers via PATCH and ocx provider edit --headers #961; excludes vision capability semantics discussed here.

Checks

• I searched existing issues and documentation.
• I removed secrets, tokens, headers, and prompt content.