Skip to content

feat: add LiteLLM provider adapter for judge metrics (#103)#210

Merged
decko merged 5 commits into
mainfrom
soda/103
Apr 25, 2026
Merged

feat: add LiteLLM provider adapter for judge metrics (#103)#210
decko merged 5 commits into
mainfrom
soda/103

Conversation

@decko
Copy link
Copy Markdown
Owner

@decko decko commented Apr 25, 2026

Summary

  • Adds litellm to LLMProvider Literal type
  • Adds litellm optional dependency in pyproject.toml
  • New dispatch branch in create_ragas_llm() for LiteLLM provider
  • patch_litellm_for_token_tracking() captures token usage from LiteLLM responses
  • 255 lines of mock-based tests (no real API calls)

Test plan

  • uv run pytest tests/test_metrics_ragas.py -v — 168 passed
  • LiteLLM dispatch creates correct client
  • Token tracking works with LiteLLM response format
  • Existing anthropic/vertex-anthropic paths unchanged

Refs #103

🤖 Generated with SODA + Claude Code

@decko decko merged commit 69ce966 into main Apr 25, 2026
4 checks passed
@decko decko deleted the soda/103 branch April 25, 2026 09:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant