2.2.1 - Multi modality is here!

[2.2.1] - 2026-03-16

Highlights

Added modality-aware routing to intelligently route requests based on input type (vision, tool-calling, text, embeddings). Enhanced changelog organization and documentation.

New Features

Modality-Aware Routing

• Modality detection module (router/modality.py) - Automatic detection of request modalities from request shape:
  • Vision: Image URL content parts in messages
  • Tool Calling: Presence of tools in request
  • Text: Default text-based chat
  • Embedding: Embeddings endpoint requests
• Model filtering by modality - Filters available models based on modality capabilities using profile flags and name heuristics.
• Safe fallback - When modality filtering removes all candidates, falls back to all available models.
• Name-based heuristics for models without profile data:
  • Vision: llava, pixtral, gpt-4o, claude-3, gemini, etc.
  • Tool calling: gpt-4, claude-3, mistral-large, qwen2.5, etc.
  • Embeddings: embed, nomic, mxbai, text-embedding, etc.

Integration

• Chat endpoint - Modality detected from request and applied during model selection.
• Embeddings endpoint - Added modality validation to warn when non-embedding models are requested.
• Router engine - Modality-based filtering integrated into model selection pipeline.

Documentation

• Reorganized 2.2.0 changelog for better readability with logical grouping.
• Removed (Item #XX) references from 2.2.0 changelog.

Testing

• Added comprehensive modality detection tests (tests/test_modality.py).
• Coverage for all modality types, edge cases, and fallback behavior.