Skip to content

[bot] Mistral: Beta Conversations API (client.beta.conversations) not instrumented #273

@braintrust-bot

Description

@braintrust-bot

Summary

The Mistral Beta Conversations API (/v1/conversations) is not instrumented. Calls to client.beta.conversations.start(), client.beta.conversations.append(), and their streaming variants produce zero Braintrust tracing. This is a documented beta API on the Mistral platform for multi-turn agentic conversations where the model autonomously executes server-side tools.

The Braintrust Mistral integration instruments client.agents.complete() (agent-configured chat completions), but the Conversations API is a distinct, higher-level agentic execution surface where Mistral manages conversation state server-side and autonomously executes tools including code interpreter, web search, image generation, and document library.

This is the Mistral equivalent of the Anthropic Managed Agents API tracked in #259 — both are beta, server-side agentic execution surfaces from major providers that produce zero tracing today.

What is missing

Mistral Resource Method Instrumented?
client.chat complete(), stream() Yes
client.agents complete(), stream() Yes
client.beta.conversations start(), start_stream() No
client.beta.conversations append(), append_stream() No
client.beta.conversations restart(), restart_stream() No
client.beta.conversations get(), list(), delete(), get_history(), get_messages() No (CRUD — lower priority)

Why this matters

Conversations is an agentic execution surface, not a simple chat API. When a conversation runs:

  • The model autonomously decides which tools to invoke (code interpreter, web search, image generation, document library, custom functions)
  • Tool execution happens server-side — results are returned as part of the conversation outputs
  • Responses include outputs arrays with message entries, tool execution entries, function calls, and agent handoffs
  • Token usage is reported via ConversationUsageInfo
  • Streaming is supported for real-time output delivery

This is functionally equivalent to the agentic frameworks already instrumented in this repo (Anthropic Managed Agents, Pydantic AI agents, Agno agents, Google ADK, OpenAI Agents SDK, Claude Agent SDK).

Minimum instrumentation

At minimum, start() and append() (and their streaming variants) should create spans capturing:

  • Conversation-level span (type task): conversation ID, model, total duration, aggregate token usage
  • Output detail: message content, tool execution results, function call results
  • Tool spans: child spans for server-side tool executions (code interpreter, web search, image generation, document library)
  • Metrics: token usage from ConversationUsageInfo, time-to-first-token for streaming
  • Metadata: model, tools configuration, conversation ID, guardrails config

Braintrust docs status

not_found — The Mistral integration page documents chat completions, embeddings, FIM, and agents. No mention of the Conversations API.

Upstream sources

  • Mistral Conversations API endpoints: https://docs.mistral.ai/api/endpoint/beta/conversations
  • Mistral Agents & Conversations guide: https://docs.mistral.ai/agents/agents
  • Supported tools: web search, web search premium, code interpreter, image generation, document library, custom function tools
  • Python SDK: client.beta.conversations.start(), .append(), .restart(), and streaming variants
  • Beta status: accessible to all API users, documented in the official API reference under the Beta section

Local files inspected

  • py/src/braintrust/integrations/mistral/patchers.py — defines patchers for Chat, Embeddings, Fim, Agents; zero references to conversation, conversations, or beta
  • py/src/braintrust/integrations/mistral/tracing.py — wrapper functions for chat, embeddings, FIM, agents only; no conversation wrappers
  • py/src/braintrust/integrations/mistral/integration.py — integration class registers 4 composite patchers; no ConversationsPatcher
  • py/src/braintrust/integrations/mistral/test_mistral.py — no conversation test cases
  • py/noxfile.pytest_mistral session tests against LATEST and 1.12.4; no conversations coverage

Relationship to existing issues

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions