bug: vision_analyze first call returns 400 when main model lacks vision support

[webtecnica]

Description

When vision_analyze is called and the main model does not support image inputs, the first invocation fails with a 400 error instead of automatically falling back to the configured auxiliary.vision provider. The second call succeeds without any configuration changes.

Expected Behavior

The system prompt indicates that when 'the active model has native vision, the image is attached to your context directly; otherwise Hermes falls back to an auxiliary vision model'. This fallback should happen automatically on the first attempt.

Actual Behavior

First call:

Error analyzing image: Error code: 400 - {'error': {'message': 'The provided messages input is invalid. The error info is [Unexpected item type in content].'}}

Second call (same image, no config change): Success.

Configuration

auxiliary:
  vision:
    provider: openrouter
    model: nvidia/nemotron-nano-12b-v2-vl:free
    timeout: 120
    download_timeout: 30

Main model: qwen3.7-max (custom/Alibaba provider) — does NOT support vision.

Reproduction Steps

1. Configure auxiliary.vision with OpenRouter + a free vision model
2. Set a non-vision model as primary (e.g., qwen3.7-max)
3. Attach an image via WebUI or CLI
4. Call vision_analyze(image_url='/path/to/image.jpg', question='...')
5. Observe error 400 on first attempt
6. Retry same call → succeeds

Environment

• Hermes Agent v0.18.0 (2026.7.1)
• WebUI session
• Primary provider: custom (Alibaba MAAS, ap-southeast-1)
• Fallback chain: deepseek-v4-flash, deepseek-v4-pro, qwen3.6-plus, qwen-plus (all custom/Alibaba)

Hypothesis

The vision tool likely attempts to inject the image into the primary model's message payload first, and only falls back to auxiliary.vision after the primary call fails. The fallback logic should check model capabilities upfront rather than retrying after a 400 error.

[webtecnica]

Opened PR #58543 to fix this.

Root cause: When resolve_vision_provider_client(provider="auto") falls through to the main provider branch, _main_model_supports_vision() returns True for unknown capability models (optimistic default). For custom/text-only providers (DeepSeek, Qwen via Alibaba), this sends an image to a model that rejects it with 400.

Fix: When the user has explicitly configured auxiliary.vision, skip the unknown-capability main provider and fall through to the aggregator chain (OpenRouter → Nous) which uses known-vision-capable models.

[teknium1]

This is an automated hermes-sweeper review. Current main already honors the reported explicit auxiliary-vision configuration path.

• agent/auxiliary_client.py:6070 returns a non-auto auxiliary.vision.provider from config before auto-detection.
• agent/auxiliary_client.py:7301-7307 passes that resolved provider and configured model into the vision client resolver; the main-provider auto chain is not selected for the configuration in this report.
• tools/vision_tools.py:1498-1510 reads auxiliary.vision.model from config.yaml and forwards it to the vision request path (commit 149641485c7f5bcb33f6acf11544f0a816d8b054).

The linked proposals in #58543 and #58600 addressed an auto-routing branch, but the issue's explicit auxiliary.vision.provider: openrouter configuration now resolves before that branch. Closing as implemented on main.