[bot] Mistral streaming drops `thinking` content blocks from Magistral and Mistral Small 4 reasoning models

[braintrust-bot]

Summary

The Mistral chat API returns structured thinking content blocks when using reasoning-capable models (Magistral family, Mistral Small 4 with reasoning_effort: "high"), but the Mistral plugin's streaming aggregation silently drops all thinking content. The extractDeltaText function only processes type: "text" chunks from content arrays; type: "thinking" chunks fall through and are lost. The reasoning_effort request parameter is also excluded from span metadata. This is a direct parity gap with Anthropic, Google GenAI, OpenAI, and Cohere reasoning instrumentation in this repo.

What instrumentation is missing

1. Streaming: thinking content silently dropped

In js/src/instrumentation/plugins/mistral-plugin.ts, the extractDeltaText function (lines 358–378) only extracts type: "text" parts from content arrays:

function extractDeltaText(content: unknown): string | undefined {
  if (typeof content === "string") {
    return content;
  }
  if (!Array.isArray(content)) {
    return undefined;
  }
  const textParts = content
    .map((part) => {
      if (!isObject(part) || part.type !== "text") {
        return "";  // thinking chunks silently dropped here
      }
      return typeof part.text === "string" ? part.text : "";
    })
    .filter((part) => part.length > 0);
  return textParts.length > 0 ? textParts.join("") : undefined;
}

When Mistral sends delta.content as an array containing { type: "thinking", thinking: [...] } chunks, the function returns empty string for them. Only { type: "text", text: "..." } chunks are captured.

2. Accumulator has no field for reasoning content

The MistralChoiceAccumulator type (lines 523–530) only tracks content (text), toolCalls, role, finishReason, and index. There is no field for reasoning/thinking content.

3. Request metadata: reasoning_effort not captured

The MISTRAL_REQUEST_METADATA_ALLOWLIST (lines 154–180) does not include reasoning_effort. When users set reasoning_effort: "high" or reasoning_effort: "none", this configuration is excluded from span metadata.

4. Vendor types: content typed as string only

The vendor type MistralChatMessageDelta.content is typed as string | null (line 13 of js/src/vendor-sdk-types/mistral.ts), but Mistral's reasoning models return content as Array<ContentChunk> with ThinkChunk elements. The runtime code in extractDeltaText does handle arrays, but only for type: "text" parts.

Upstream API format

Mistral reasoning models return structured content in streaming deltas:

{
  "choices": [{
    "delta": {
      "content": [{
        "type": "thinking",
        "thinking": [{ "type": "text", "text": "Let me analyze..." }]
      }]
    }
  }]
}

The final answer arrives as either a plain string or { type: "text", text: "..." } chunks. Non-streaming responses use a similar array with mixed thinking and text chunks in message.content.

Comparison with other providers in this repo

Provider	Thinking content captured	Request config captured
Anthropic	thinking_delta aggregated in streaming	N/A
Google GenAI	thought parts handled, thoughtsTokenCount metric	N/A
OpenAI	Reasoning tokens tracked via completion_tokens_details	reasoning_effort in metadata
Cohere	Not yet (open issue #1845)	Not yet (open issue #1845)
Mistral	Silently dropped	reasoning_effort not in allowlist

Note: Token-level reasoning metrics (e.g. completionTokensDetails.reasoningTokens) ARE correctly captured by parseMistralMetricsFromUsage through its generic nested detail handling. The gap is specifically in content capture and request metadata.

Braintrust docs status

not_found — The Braintrust Mistral integration page at https://www.braintrust.dev/docs/integrations/ai-providers/mistral does not mention reasoning or thinking content.

Upstream references

• Mistral Native Reasoning docs: https://docs.mistral.ai/capabilities/reasoning/native
• Mistral Adjustable Reasoning docs: https://docs.mistral.ai/capabilities/reasoning/adjustable
• Mistral Chat API endpoint: https://docs.mistral.ai/api/
• Mistral TypeScript SDK: https://github.com/mistralai/client-ts
• reasoning_effort parameter: accepts "high" or "none"
• Affected models: Magistral Medium, Magistral Small, Mistral Small 4 (with reasoning_effort: "high")
• Content format: ThinkChunk with type: "thinking" in structured content arrays

Local files inspected

• js/src/instrumentation/plugins/mistral-plugin.ts (lines 358–378: extractDeltaText only handles type: "text"; lines 523–530: MistralChoiceAccumulator has no thinking field; lines 154–180: MISTRAL_REQUEST_METADATA_ALLOWLIST missing reasoning_effort)
• js/src/vendor-sdk-types/mistral.ts (line 13: content typed as string | null)
• js/src/instrumentation/plugins/mistral-channels.ts
• js/src/wrappers/mistral.ts
• e2e/scenarios/mistral-instrumentation/ (no reasoning test scenarios)