docs: modalities field for custom provider models is undocumented

[f-trycua]

Summary

The modalities config field for custom provider model entries is not documented anywhere — not on opencode.ai/docs, not in the README, not in the public config.json schema reference. Users hit confusing errors and silent failures with no way to fix them without reading source.

The problem in practice

When using a vision-capable model via a custom provider (e.g. gemma4:26b via Ollama / @ai-sdk/openai-compatible), images are silently stripped and the model receives:

ERROR: Cannot read image (this model does not support image input). Inform the user.

The fix is to declare modalities in the model config — but this is impossible to discover without reading provider.ts.

The fix (what should be documented)

Add modalities to a custom model entry to declare its input/output capabilities:

{
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "options": { "baseURL": "http://localhost:11434/v1" },
      "models": {
        "gemma4:26b": {
          "modalities": {
            "input": ["text", "image"],
            "output": ["text"]
          }
        }
      }
    }
  }
}

Allowed modality values (from provider.ts): "text", "audio", "image", "video", "pdf".

Without this, capabilities.input.image defaults to false for all custom provider models, so vision never works regardless of what the underlying model supports.

Also worth noting

supportsMediaInToolResults in message-v2.ts is a hardcoded allowlist (@ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/amazon-bedrock, @ai-sdk/google). Custom providers using @ai-sdk/openai-compatible (the standard path for Ollama) are excluded, so MCP tool results containing images are never passed natively in the tool result — they get re-injected as synthetic user messages instead. This is a separate issue but compounds the confusion.

Suggested fix

Document modalities in the configuration reference, ideally with a "custom Ollama vision model" example.

[github-actions]

This issue doesn't fully meet our contributing guidelines.

What needs to be fixed:

• The issue does not use one of the required templates (Bug Report, Feature Request, or Question). Please resubmit using the appropriate template — this looks like either a Bug Report (undocumented behavior causing silent failures) or a Feature Request (requesting documentation improvements).

Please edit this issue to address the above within 2 hours, or it will be automatically closed.

---

This issue might be a duplicate of existing issues. Please check:

• Custom OpenAI-compatible providers: image file attachments do not reach vision-capable models correctly #20802: Same root cause — custom OpenAI-compatible providers have image input silently stripped because modalities is undocumented; the comments on that issue already contain the identical root-cause analysis and the modalities workaround you describe.
• Custom OpenAI-compatible provider via LiteLLM is incorrectly classified as text-only / no-reasoning and blocks image input #22673: Same symptom — custom provider models via OpenAI-compatible gateway are classified as text-only, blocking image input.

If your issue is distinct (e.g. specifically focused on the docs gap rather than the bug itself), please clarify the difference and consider whether a docs PR would be more appropriate.

If you believe this was flagged incorrectly, please let a maintainer know.

[f-trycua]

Closing as duplicate of #20802. The specific workaround for Ollama vision models (e.g. gemma4:26b via @ai-sdk/openai-compatible) is to declare modalities explicitly in config.json — see comment on #20802.