[Feature]: Native reasoning_effort support for Mistral AI (api.mistral.ai/v1) endpoint

[wyan-r]

Problem or Use Case

Summary

Add support for Mistral AI's adjustable reasoning capability when api.mistral.ai/v1 is
configured as a custom provider in Hermes Agent. This requires injecting reasoning_effort
as a root-level request field and guarding injection against Mistral model families that do
not support the parameter.

Note: This feature request is specific to Mistral AI's reasoning_effort contract.
The general streaming and field-leakage bugs that were exposed while implementing this
have been filed separately as they affect all strict OpenAI-compatible providers.

---

Background

Mistral AI offers two approaches to reasoning (see Mistral Reasoning docs):

Approach	Models	How it works
Adjustable	mistral-small-*	Controlled by a reasoning_effort request parameter (low, medium, high)
Native	magistral-small-*, magistral-medium-*	Model always reasons; no parameter needed

When mistral-small-latest is used with reasoning_effort: high in Hermes, the parameter
is currently never sent to the API. Mistral receives no reasoning instruction and returns
a standard (non-reasoning) response.

The parameter must be sent as a root-level field in the Chat Completions request:

{
  "model": "mistral-small-latest",
  "reasoning_effort": "high",
  "messages": [...]
}

This differs from the OpenRouter convention (extra_body.reasoning.effort) and the native
OpenAI o-series convention (reasoning_effort directly on the client kwargs).

---

Problem

reasoning_effort is never injected for Mistral

File: run_agent.py — _build_api_kwargs()

The method injects reasoning parameters through two existing paths:

• OpenRouter path: extra_body["reasoning"] = {...} (via _supports_reasoning_extra_body()).
• Ollama path: extra_body["think"] = false (via Ollama URL detection).

Neither path handles Mistral-style reasoning_effort. The setting is read from config but
silently discarded. Mistral returns a non-reasoning response with no error or warning.

Sending reasoning_effort to unsupported Mistral models causes HTTP 422

Only mistral-small-* accepts reasoning_effort. If the parameter is injected blindly for
all Mistral models:

• magistral-small-* / magistral-medium-* — reject it with HTTP 422 (they reason natively,
  no parameter is needed or accepted).
• All other Mistral models (mistral-large-*, mistral-medium-*, codestral, etc.) — reject
  it with HTTP 422 (no reasoning support at all).

---

Proposed Solution

Proposed Implementation

Inject reasoning_effort via extra_body for Mistral, guarded by model family

run_agent.py — _build_api_kwargs(), after the Ollama think=False block:

# Inject reasoning_effort for custom endpoints that use the root-level parameter
# (e.g. Mistral AI). Only send when the model is known to support it.
if (self.provider == "custom" or not self._supports_reasoning_extra_body()) \
        and self.reasoning_config and isinstance(self.reasoning_config, dict):
    _effort = (self.reasoning_config.get("effort") or "").strip().lower()
    _enabled = self.reasoning_config.get("enabled", True)
    if _enabled and _effort and _effort != "none":
        _model_lower = (self.model or "").lower()
        _is_mistral_endpoint = "api.mistral.ai" in self._base_url_lower
        if _is_mistral_endpoint:
            # Only mistral-small-* supports adjustable reasoning_effort.
            # Magistral models reason natively and do not accept this parameter.
            # All other Mistral models have no reasoning support.
            # See: https://docs.mistral.ai/capabilities/reasoning/
            _mistral_supports_effort = _model_lower.startswith("mistral-small")
            if not _mistral_supports_effort:
                logger.debug(
                    "Skipping reasoning_effort for Mistral model %r: "
                    "only mistral-small-* supports adjustable reasoning_effort. "
                    "Magistral models reason natively; other models will 422.",
                    self.model,
                )
        else:
            _mistral_supports_effort = True  # Non-Mistral custom endpoints: always send
        if _mistral_supports_effort:
            extra_body["reasoning_effort"] = _effort

extra_body is already merged into api_kwargs["extra_body"] at the end of
_build_api_kwargs(). The OpenAI Python SDK serializes extra_body as root-level JSON
fields — exactly the format Mistral expects.

The check is scoped to api.mistral.ai so non-Mistral custom endpoints (NVIDIA, vLLM,
LiteLLM, etc.) are unaffected.

Extend _supports_reasoning_extra_body() with a Mistral guard

To prevent reasoning_content / reasoning_details from leaking into Mistral message
history (which already causes HTTP 422, see companion bug report), _supports_reasoning_extra_body()
must return False for api.mistral.ai. This is already the case since the method only
returns True for known OpenRouter/Nous Portal/GitHub Models routes — no change needed here,
but it should be explicitly documented:

def _supports_reasoning_extra_body(self) -> bool:
    """Return True when OpenRouter-style reasoning extra_body is safe to send.

    Returns False for api.mistral.ai — Mistral uses root-level reasoning_effort
    (handled separately) and rejects extra_body.reasoning with extra_forbidden.
    """
    if "api.mistral.ai" in self._base_url_lower:
        return False  # explicit: Mistral uses reasoning_effort, not extra_body.reasoning
    ...

---

Mistral Reasoning Model Table

Model slug	Reasoning support	reasoning_effort accepted
mistral-small-latest (mistral-small-3.1)	Adjustable	✅ Yes
mistral-small-3-1-* versioned aliases	Adjustable	✅ Yes
magistral-small-latest	Native (always on)	❌ No (422)
magistral-medium-latest	Native (always on)	❌ No (422)
mistral-large-latest	None	❌ No (422)
mistral-medium-latest	None	❌ No (422)
codestral-latest	None	❌ No (422)
All other Mistral models	None	❌ No (422)

Source: https://docs.mistral.ai/capabilities/reasoning/

---

Verification

Tested locally against mistral-small-latest with reasoning_effort: high:

import run_agent

agent = run_agent.AIAgent(
    model="mistral-small-latest",
    api_key="<MISTRAL_API_KEY>",
    provider="custom",
    base_url="https://api.mistral.ai/v1",
    reasoning_config={"enabled": True, "effort": "high"},
    max_iterations=2,
)
agent.reasoning_callback = lambda x: print("REASONING:", x[:80])

result = agent.run_conversation("Explain the halting problem.")
print("RESPONSE:", result.get("final_response", "")[:120])

Expected output:

REASONING: <Mistral thinking trace, streamed in real time>
RESPONSE: The halting problem is a fundamental result in computability theory...

Also verified that mistral-large-latest and codestral-latest do not receive
reasoning_effort and respond normally.

---

Files Changed

File	Change
run_agent.py	Inject reasoning_effort into extra_body for supported Mistral models
run_agent.py	Document Mistral exclusion in _supports_reasoning_extra_body()

---

Acceptance Criteria

• reasoning_effort is present as a root-level field in the outgoing JSON when
  provider=custom, base_url is api.mistral.ai/v1, model is mistral-small-*,
  and reasoning_effort is configured and not none.
• reasoning_effort is not sent to magistral-*, mistral-large-*,
  mistral-medium-*, codestral, or any other non-supporting Mistral model family.
• Non-Mistral custom endpoints (NVIDIA, vLLM, etc.) are unaffected by this change.
• python -m pytest tests/ -q passes with no regressions.

---

References

• Mistral reasoning overview: https://docs.mistral.ai/capabilities/reasoning/
• Mistral adjustable reasoning (reasoning_effort): https://docs.mistral.ai/capabilities/reasoning/adjustable
• Mistral native reasoning (Magistral): https://docs.mistral.ai/capabilities/reasoning/native
• Mistral Chat Completions API: https://docs.mistral.ai/api/#tag/chat/operation/chat_completion_v1_chat_completions_post

Labels

feat provider-compat reasoning mistral

Alternatives Considered

No response

Feature Type

Configuration option

Scope

Small (single file, < 50 lines)

Contribution

• I'd like to implement this myself and submit a PR

Debug Report (optional)

[Tipiweb]

Adding a real-world datapoint for this feature request.

Running hermes on a Raspberry Pi with api.mistral.ai/v1 as custom provider, model mistral-small-latest (= Small 4 / 2603, which unifies Magistral reasoning + Pixtral multimodal + Devstral coding into one model). Without reasoning_effort active, the model hallucinates tool success on non-trivial agentic tasks — e.g. in one case it confidently reported "5 items deleted" with fabricated IDs after the underlying CLI had silently returned a Mailbox does not exist stderr and nothing had actually changed server-side. The pattern isn't specific to the tool in question; it's the classic "didn't re-read stderr, didn't verify post-action" failure mode that a pre-action reasoning pass tends to catch.

To get reasoning_effort working end-to-end with tool-use on Mistral, I needed four pieces stacked:

1. Top-level reasoning_effort for direct api.mistral.ai — covered by fix(agent): add Mistral reasoning_effort support #11390. Scope is currently mistral-small-*; worth extending to magistral-* and devstral-*.
2. Magistral structured content blocks (content returned as [{type: thinking}, {type: text}] instead of a plain string) — needs flattening in both streaming deltas and the non-streaming message builder. This was proposed in fix: handle Mistral Magistral structured content blocks #4748 but closed by the maintainer with "not pursuing this approach", so I assume a different design is planned.
3. Strip reasoning_content / reasoning_details / reasoning from outgoing messages on replay — Mistral rejects them with HTTP 422 extra_forbidden (same issue reported on fix(agent): add Mistral reasoning_effort support #11390).
4. Mistral API quirk: reasoning_effort accepts only "none" or "high" (422 on medium/low/xhigh). Any clamping logic should map intermediate values to high or skip emission.

With all four in place locally, reasoning+tool-use is reliable: visible reasoning before the tool decision, correct tool execution, second reasoning pass interpreting the result, clean final answer. ~25s round-trip on a Pi, usable for production agentic flows.

Happy to help validate whatever official approach the team lands on for the structured content blocks piece.

[Adolanium]

Confirming this is still the state on main (checked 2026-06-21) while wiring Mistral as a first-class provider, with fresh detail on the point-2 blocker.

The wire-level part (point 1) is the easy half: a provider profile can emit a root-level reasoning_effort and Mistral accepts it. The hard blocker is the structured content, and it reproduces end-to-end today. With reasoning_effort set on mistral-small-latest, every response fails with Response remained truncated after 3 continuation attempts, even on a trivial prompt in a fresh session.

Root cause, captured from the raw stream (stream=True):

delta.content = [{'type': 'thinking', 'thinking': [{'type': 'text', 'text': 'The'}]}]
...
delta.content = [{'type': 'text', 'text': 'The sum ...'}]
finish_reason = stop

The API returns a complete response (finish_reason=stop), but delta.content is a list of blocks, not a string. The stream consumer expects a string, accumulates nothing, and calls the turn truncated. That is exactly the #4748 territory (flatten the thinking / text blocks in both the streaming deltas and the non-streaming message builder).

Two updates to the four-point list:

• Point 4 (accepted values): on the current API the rejection is HTTP 400, not 422. reasoning_effort: medium on mistral-small-latest, and any value on a non-reasoning model (mistral-large-latest, codestral-latest), both return 400 invalid_request. So high / none only, and the family gate still matters.
• Downstream effect of the truncation: when the stream is cut before any content lands, the turn persists as {"role": "assistant", "content": ""}, and the next request replays it, which Mistral rejects with 400 "Assistant message must have either content or tool_calls, but not none". Mistral does not require role alternation, so dropping the empty turn from the outgoing payload clears that 400, but the real fix is not producing the empty turn in the first place (point 2).

One more in the same family as point 3: the max-iterations summary path hand-builds messages and bypasses the transport sanitization, so resumed sessions leak the DB timestamp field and hit 422 extra_forbidden on messages[N].timestamp.

Net: non-reasoning chat and tool calling on Mistral are solid; reasoning needs the structured-content flattening (#4748) before any of the wire-level work is usable.

[luluthehungrycat]

This is now addressed by PR #65450 — a bundled Mistral AI model provider plugin that handles reasoning_effort with a version-threshold gate.

Covered in the PR:

• reasoning_effort sent as root-level field for models above per-family version thresholds (small >= 2603, medium >= 2604, large >= 2600 speculative, plus exact matches for -latest aliases and 3.x family)
• Automatically covers future date-stamped models (e.g. mistral-small-2703, mistral-medium-2609, mistral-large-2601)
• Correctly blocks reasoning_effort for magistral-*-latest (native reasoning, rejects the parameter), codestral, ministral, pixtral, and older date-stamped variants
• 48 unit tests, 86/86 live API checks passed

Not covered (upstream Hermes issues, not provider-level):

• Structured content blocks in streaming responses (thinking chunks) — this is a Hermes stream consumer issue tracked separately. The provider correctly emits reasoning_effort and Mistral returns the thinking blocks, but Hermes needs to flatten [{type: thinking}, {type: text}] into strings.

#65450

[alt-glitch]

This was generated by AI during triage.

Related: #11390 is the focused direct-Mistral reasoning_effort implementation, but its current diff targets the pre-refactor path and needs porting to the live transport/summary paths. #65450 adds a provider profile and model gate but does not cover the reported structured-content stream normalization.

[GottZ]

This was generated by AI during triage.

Summary

One PR directly addresses #11243. #11390 implements the required root-level Mistral reasoning_effort field with a mistral-small-* model gate and prevents Ollama think=false leakage, but its diff modifies the pre-refactor request path rather than the current transport and summary paths.

Related pull requests

• fix(agent): add Mistral reasoning_effort support #11390 best fix — (+75/-1) — partial fix / best consolidation target: the diff correctly adds direct-Mistral host detection, limits reasoning_effort to mistral-small-*, omits it for unsupported families, and adds focused regression tests. Consistent with the keep_open review on fix(agent): add Mistral reasoning_effort support #11390, it must be ported from run_agent.py to the live chat-completions transport and the max-iterations summary path before merge; the separately reported structured-content normalization issue is not covered by this diff.

Suggested consolidation

Consolidate on #11390 and merge it after porting its host/model gate and root-level reasoning_effort handling to the current transport and summary request paths, with equivalent regression coverage. This follows the visible keep_open review rather than merging the stale diff as-is; there are no listed duplicate PRs to close.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I11243(["issue #11243 (open)"])
    P11390["PR #11390 (open)"]
    P11390 -->|best fix| I11243
    class I11243 open
    class I11243 target
    class P11390 open
    class P11390 best
    click I11243 "https://github.com/NousResearch/hermes-agent/issues/11243"
    click P11390 "https://github.com/NousResearch/hermes-agent/pull/11390"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed or no verify verdict yet (state tag in the node label).

Cross-PR triage: Reviewed 1 pull request and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 5 kB of PR diffs, 10 kB of issue/PR text, 11 kB of discussion (8 comments), 2 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.