Summary
Wire the LiteLLM adapter (from #103) into the CLI and handle embeddings provider dispatch.
Scope (Part 2 of 2)
- Add
litellm to --judge-provider CLI choices
- Handle embeddings dispatch — keep VertexAIEmbeddings for Vertex, add provider-aware dispatch for LiteLLM models
- Update
--judge-provider help text with examples
- Add CLI integration tests
- Update docs (README, getting-started) with per-provider examples
Usage
# Gemini on Vertex AI
raki run --judge --judge-provider litellm --judge-model vertex_ai/gemini-2.5-pro
# OpenAI
raki run --judge --judge-provider litellm --judge-model gpt-4o
# Any LiteLLM-supported model
raki run --judge --judge-provider litellm --judge-model groq/llama-3.1-70b
Files
Depends on
#103 (LiteLLM adapter) must ship first.
Summary
Wire the LiteLLM adapter (from #103) into the CLI and handle embeddings provider dispatch.
Scope (Part 2 of 2)
litellmto--judge-providerCLI choices--judge-providerhelp text with examplesUsage
Files
src/raki/cli.py— add litellm to provider choices, help textsrc/raki/metrics/ragas/llm_setup.py— embeddings dispatch (if not in feat: add LiteLLM provider adapter for judge metrics #103)tests/test_cli.py— integration tests for new providerdocs/— usage examplesDepends on
#103 (LiteLLM adapter) must ship first.