Skip to content

Add LiteLLM, Cerebras, and OpenRouter embedding provider support #108

@bashandbone

Description

@bashandbone

Description

Add support for LiteLLM, Cerebras, and OpenRouter providers to expand embedding model options.

Current State

  • File: openai_factory.py:315
  • Limited provider support
  • Missing popular alternatives

Providers to Add

LiteLLM

  • Unified API for 100+ LLM providers
  • Simplifies multi-provider support
  • Built-in fallbacks and load balancing
  • Cost tracking

Benefits:

  • Access to many providers through one interface
  • Easier provider switching
  • Better resilience

Cerebras

  • High-performance inference
  • Fast embedding generation
  • Cost-effective at scale

Benefits:

  • Performance improvements
  • Additional provider option
  • Competitive pricing

OpenRouter (NEW)

  • Recently added embedding model support
  • OpenAI-compatible API
  • Access to multiple models through one endpoint

Benefits:

  • Single API for multiple providers
  • Simplified integration (OpenAI-compatible)
  • Cost optimization options

Implementation Considerations

All three are OpenAI-compatible providers, which should simplify implementation:

  • Reuse OpenAI client patterns
  • Provider-specific authentication
  • API compatibility verification
  • Rate limiting differences
  • Error handling per provider
  • Configuration management

Testing

  • Test with each provider's API
  • Validate embedding quality
  • Performance benchmarking
  • Error scenario testing

Source

  • File: openai_factory.py:315
  • Branch: 003-our-aim-to

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions