Add Together.ai provider support

[moncapitaine]

Overview

Add support for Together.ai as an AI provider, enabling users to access 100+ open-source models and deploy custom fine-tuned models at scale.

Why Together.ai?

• 100+ open models: Largest collection of open-source models
• Fine-tuning platform: Custom model training and deployment
• Cost-effective: Competitive pricing for open models
• OpenAI-compatible: Easy integration
• Research-friendly: Academic pricing, latest research models
• Bleeding edge: Newest open models available quickly

Capabilities

• ✅ Chat Completion: Llama, Mixtral, Qwen, Yi, DeepSeek, and 100+ more
• ✅ Embedding: bge, e5, instructor models
• ✅ Vision: LLaVA, CogVLM, Qwen-VL
• ✅ Code models: CodeLlama, DeepSeek-Coder, StarCoder
• ✅ Function Calling: Model-dependent
• ✅ Custom models: Deploy your own fine-tuned models

Implementation Checklist

Backend

• Create Together.ai provider client in ai-service-client package
• Implement chat completion with streaming support
• Implement embedding generation
• Implement vision support
• Handle model-specific parameters
• Add model capability detection
• Add to provider cache system
• Add connection testing endpoint
• Support custom/fine-tuned model deployment

Database

• Add together to provider enum (if needed)
• Update provider configuration schema
• Support custom model endpoints

GraphQL API

• Add Together.ai provider to AiServiceProvider mutations
• Model discovery from Together.ai API (100+ models)
• Support filtering by model type (chat, embedding, vision, code)
• Support custom model IDs

Frontend

• Add Together.ai provider UI in /admin/ai-services
• API key configuration form
• Connection testing
• Model selection with filtering (chat/embed/vision/code)
• Support for custom model IDs
• Display model metadata (parameters, context length)

Documentation

• Update /docs/admin/ai-models with Together.ai
• Update /docs/admin/ai-services with Together.ai configuration
• Add Together.ai setup guide
• Document popular model recommendations
• Explain custom model deployment

Testing

• Unit tests for Together.ai client
• Integration tests for chat completion (multiple models)
• Integration tests for embeddings
• Integration tests for vision models
• E2E tests for provider configuration
• Test custom model endpoints

API Details

Base URL: https://api.together.xyz/v1
Authentication: Bearer token via Authorization header
API: OpenAI-compatible
Models Endpoint: /v1/models
Chat Endpoint: /v1/chat/completions
Embeddings Endpoint: /v1/embeddings

Model Examples (100+ available):

• meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo (largest Llama)
• Qwen/Qwen2.5-72B-Instruct-Turbo (high quality chat)
• mistralai/Mixtral-8x22B-Instruct-v0.1 (MoE model)
• deepseek-ai/deepseek-coder-33b-instruct (code generation)
• BAAI/bge-large-en-v1.5 (embeddings)

Unique Features

Fine-tuning Platform

Together.ai allows users to fine-tune models and deploy them:

1. Upload training data
2. Fine-tune base model (Llama, Mistral, etc.)
3. Deploy as custom endpoint
4. Use with same API as pre-trained models

Model Diversity

Access to cutting-edge research models:

• Latest Llama releases
• Chinese models (Qwen, Yi, DeepSeek)
• Specialized models (legal, medical, math)
• Multimodal models (vision, audio)

Resources

• Together.ai Documentation
• API Reference
• Available Models
• Fine-tuning Guide

Related

• Part of multi-provider support initiative
• Enables open model experimentation
• Supports custom fine-tuned model deployment
• Complements Hugging Face for open model access

Priority

P3-low - Overlap with Hugging Face for open models, but fine-tuning platform is unique. Consider after Hugging Face implementation.