Skip to content

v0.2.0 - OpenRouter Provider Support

Choose a tag to compare

View all tags
@TianyiPeng TianyiPeng released this 23 Aug 15:35
· 14 commits to main since this release
v0.2.0
ace0494

🚀 Major Feature Release: OpenRouter Provider Support

🎯 What's New

OpenRouter Integration

  • 100+ AI Models: Access to OpenAI, Anthropic, Meta, Google, Mistral, and more through one unified API
  • DeepSeek Support: Full support for deepseek/deepseek-v3.1-base and deepseek/deepseek-chat
  • Cost Effective: Often cheaper than going directly to providers
  • Unified API: Single integration for multiple model providers

Enhanced Configuration

  • Added **kwargs support to LLMConfig for provider-specific parameters
  • Maintains full backward compatibility with existing code
  • Enhanced error handling and retry logic

📖 Quick Start with OpenRouter

# Install the latest version
pip install llm-batch-helper==0.2.0

# Set up your API key
export OPENROUTER_API_KEY="your-openrouter-api-key"
from llm_batch_helper import LLMConfig, process_prompts_batch

# Use any OpenRouter model
config = LLMConfig(
    model_name="deepseek/deepseek-v3.1-base",  # or openai/gpt-4o, anthropic/claude-3-5-sonnet, etc.
    temperature=0.7,
    max_completion_tokens=500
)

results = await process_prompts_batch(
    config=config,
    provider="openrouter",  # New provider!
    prompts=your_prompts
)

🌟 Supported Models (Examples)

DeepSeek Models

  • deepseek/deepseek-v3.1-base - Latest base model
  • deepseek/deepseek-chat - Chat-optimized version

OpenAI Models via OpenRouter

  • openai/gpt-4o
  • openai/gpt-4o-mini
  • openai/gpt-4-turbo

Anthropic Models

  • anthropic/claude-3-5-sonnet
  • anthropic/claude-3-haiku

Meta Models

  • meta-llama/llama-3.1-405b-instruct
  • meta-llama/llama-3.1-70b-instruct

Google Models

  • google/gemini-pro-1.5
  • google/gemini-flash-1.5

🔧 Technical Improvements

  • Full integration with existing caching and verification systems
  • Proper retry logic with exponential backoff
  • Token usage tracking and monitoring
  • Credit-based rate limiting support
  • Enhanced provider comparison documentation

📚 Documentation Updates

  • Comprehensive OpenRouter setup guide
  • Updated provider comparison table
  • Model selection examples and best practices
  • Enhanced API reference

🔄 Migration

No breaking changes! Existing code continues to work unchanged. Simply add OpenRouter as a new option when you need access to additional models.

🙏 Get Started

  1. Get your OpenRouter API key: https://openrouter.ai/settings/keys
  2. Install: pip install llm-batch-helper==0.2.0
  3. Follow the documentation: Provider Guide

🚦 Supported Providers

Your package now supports:

  • ✅ OpenAI (original)
  • ✅ Together.ai (existing)
  • ✅ OpenRouter (new!) - 100+ models including DeepSeek v3.1

📊 Performance

OpenRouter integration has been tested and verified with:

  • Fast response times (~1.8s per request for DeepSeek)
  • Reliable error handling and retry logic
  • Proper token usage tracking
  • Full compatibility with existing batch processing features

Full Changelog: https://github.com/TianyiPeng/LLM_batch_helper/blob/main/CHANGELOG.md

PyPI Package: https://pypi.org/project/llm-batch-helper/0.2.0/

Assets 2
Loading