Skip to content

[bot] Cohere: Embed Jobs API (client.embed_jobs.create()) not instrumented #623

Description

@braintrust-bot

Summary

The Cohere Python SDK's Embed Jobs API — client.embed_jobs.create() / client.embed_jobs.get() / client.embed_jobs.list() (and the AsyncClient equivalents) — is not instrumented. This is Cohere's bulk/batch embedding execution surface: it launches an async job that reads a Dataset of type embed-input, runs the configured embedding model (model, input_type, embedding_types) over every record, and writes the resulting vectors to a new embed-output Dataset. It is the async, large-scale counterpart to the synchronous client.embed() call, which is instrumented.

Calls to client.embed_jobs.create() fall through uninstrumented today — the Cohere integration only patches BaseCohere/AsyncBaseCohere (chat, chat_stream, embed, rerank) and the v2 client and audio transcription client. No patcher touches the embed_jobs resource, so job creation produces zero Braintrust tracing (no span for the submitted job, its model/config, or its terminal status).

What is missing

Cohere method Instrumented?
client.chat() / client.chat_stream() (v1 + v2) Yes
client.embed() (v1 + v2) Yes
client.rerank() (v1 + v2) Yes
client.audio.transcriptions.create() Yes
client.embed_jobs.create() No
client.embed_jobs.get() No
AsyncClient.embed_jobs.create() No

At minimum, instrumentation should create a span for embed_jobs.create() capturing:

  • Input: dataset ID, model, input_type, embedding_types, job name/truncate settings
  • Output: job ID and initial status (processing/complete/failed)
  • Metadata: model name, dataset reference

This mirrors the pattern already accepted for the analogous async-batch-execution gaps in sibling integrations: OpenAI Batch API (#295), Google GenAI Batch API (#332), and Mistral Batch Jobs API (#272) — this is the same class of gap (an async, job-based generative-execution API that bypasses the wrapper) applied to Cohere's bulk embeddings surface.

Braintrust docs status

not_found — The Cohere integration page documents only: chat completion spans (cohere.chat / cohere.chat_stream), tool call spans, embedding spans (cohere.embed), rerank spans (cohere.rerank), and audio transcription spans (cohere.audio.transcriptions.create). Embed Jobs, batch/bulk embeddings, the Classify API, and the v1 generate() endpoint are not mentioned anywhere on the page.

Upstream sources

Local files inspected

  • py/src/braintrust/integrations/cohere/patchers.py — defines ChatPatcher, ChatStreamPatcher, EmbedPatcher, RerankPatcher (+ async/v2 variants) and TranscriptionsCreatePatcher/AsyncTranscriptionsCreatePatcher; no embed_jobs patcher of any kind
  • py/src/braintrust/integrations/cohere/integration.pyCohereIntegration.patchers registers only the five patchers above; embed_jobs is absent from the registry
  • py/src/braintrust/integrations/cohere/tracing.py — no embed_job wrapper functions exist
  • py/src/braintrust/integrations/cohere/test_cohere.py — no test cases reference embed_jobs
  • Repo-wide search for embed_job/embedjob (case-insensitive) returns zero matches outside this issue

Relationship to existing issues

Distinct from #488 (Cohere v1 generate()/generate_stream()) and #343 (Cohere Classify API) — both already open and unrelated to Embed Jobs. Same class of gap as the already-filed provider Batch API issues (#295, #332, #272), applied to a surface (Cohere Embed Jobs) none of those cover.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions