Skip to content

LiteLLM: image_generation(), transcription(), speech(), and rerank() not instrumented #165

@braintrust-bot

Description

@braintrust-bot

Summary

The Braintrust LiteLLM wrapper (wrap_litellm() and patch_litellm()) instruments completion, acompletion, responses, aresponses, embedding, and moderation. LiteLLM also exposes several other documented API functions that are not instrumented at all:

  • litellm.image_generation() / litellm.aimage_generation() — image generation (DALL-E, etc.)
  • litellm.transcription() / litellm.atranscription() — audio transcription (Whisper, etc.)
  • litellm.speech() / litellm.aspeech() — text-to-speech
  • litellm.rerank() / litellm.arerank() — reranking for RAG pipelines (Cohere, etc.)

Additionally, litellm.aembedding() (async embedding) is not wrapped — the EmbeddingWrapper class only accepts a sync function.

What is missing

No tracing spans are created when users call the above functions through either wrap_litellm() or patch_litellm(). The LiteLLMWrapper class in py/src/braintrust/wrappers/litellm.py (lines 484-518) only exposes: completion, acompletion, responses, aresponses, embedding, moderation. The patch_litellm() function (lines 630-667) only patches: completion, acompletion, responses, aresponses.

Braintrust docs status

not_found — The Braintrust LiteLLM integration docs at https://www.braintrust.dev/docs do not mention image generation, transcription, speech, or rerank support.

Upstream sources

Local repo files inspected

  • py/src/braintrust/wrappers/litellm.py — full wrapper implementation (668 lines)
  • py/src/braintrust/wrappers/test_litellm.py — tests (no tests for the missing functions)
  • py/noxfile.py — LiteLLM test sessions

Relationship to existing issues

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions