v0.4.0

Release Notes

Overview

A major release that introduces extensive multimodal capabilities across multiple providers, OpenAI API compatibility enhancements, and improved SDK testing infrastructure.

Major Features

🎨 Enhanced Provider Multimodal Support

Azure OpenAI Provider

• Embeddings: Full support for text embedding models
• Audio: Speech-to-text and text-to-speech capabilities
• Image Generation: DALL-E model support

Fireworks Provider

• Embeddings: Support for Nomic Embed and other embedding models
• Audio: Whisper v3 transcription and translation
• Image Generation: Limited support (async workflow API)

Mistral Provider

• Embeddings: Native embedding model support
• Moderation: Content moderation capabilities

Together AI Provider

• Embeddings: Text embedding support
• Image Generation: FLUX.1 model support
• Text-to-Speech: Cartesia Sonic with 100+ voices

xAI Provider

• Image Generation: grok-2-image model support

🔌 OpenAI API Compatibility

New Endpoints

• Batch API (/v1/batches): Asynchronous batch processing
• Images API (/v1/images/generations): Image generation
• Audio API (/v1/audio/*): Speech-to-text, translation, and TTS
• Moderation API (/v1/moderations): Content moderation
• Responses API (/v1/responses): Next-generation conversational API
• Realtime API (/v1/realtime/*): Session management for real-time interactions

Enhanced Chat Completions

• Additional parameters for full OpenAI compatibility
• Improved parameter handling and validation

🧪 Testing & Infrastructure

• SDK Integration Tests: Comprehensive testing for OpenAI, Anthropic, and Azure SDKs
• Provider Testing: Enhanced test coverage for all providers
• CI/CD Improvements: Better performance testing and migration handling

🔧 Other Improvements

• Optional Authentication: Gateway authentication can now be disabled via configuration
• Migration System: Improved ClickHouse migration handling and rollback support
• Performance: Various optimizations and bug fixes

Breaking Changes

• Some internal APIs have been refactored for better multimodal support

Migration Guide

Configuration Changes

If using new multimodal features, update your model configurations:

# Example: Azure embeddings
[models."text-embedding-ada-002"]
routing = ["azure"]
endpoints = ["embedding"]

# Example: Together TTS
[models."together-tts"]
routing = ["together"]
endpoints = ["text_to_speech"]

Known Issues

• Fireworks image generation uses an async workflow API that is not fully compatible with the synchronous interface
• Some provider-specific parameters may not be fully exposed through the unified API

What's Next

• Enhanced streaming support for audio endpoints
• WebSocket support for bidirectional communication
• Additional provider integrations
• Performance improvements for large-scale deployments