Ollama models (Mistral, Gemma) fail during MCP tool calling due to message formatting incompatibilities

[jackccrawford]

Ollama models (Mistral, Gemma) fail during MCP tool calling due to message formatting incompatibilities

Before submitting your bug report

• ✅ I believe this is a bug. I'll try to join the Continue Discord for questions
• ✅ I'm not able to find an open issue that reports the same bug
• ✅ I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: macOS
- Continue version: v1.5.26 (CLI)
- IDE version: N/A (using CLI)
- Models: 
  - mistral-large-3:675b-cloud (via Ollama)
  - ministral-3:14b-cloud (via Ollama)
  - gemma3:27b-cloud (via Ollama)
- Config:
  name: Ship-IDE Config
  version: 1.0.0
  schema: v1
  models:
    - name: Mistral Large 3 (675B Cloud)
      provider: openai
      model: mistral-large-3:675b-cloud
      apiBase: http://localhost:11434/v1
      apiKey: ollama
    - name: Ministral 3 (14B Cloud)
      provider: openai
      model: ministral-3:14b-cloud
      apiBase: http://localhost:11434/v1
      apiKey: ollama
    - name: Gemma3 (27B Cloud)
      provider: openai
      model: gemma3:27b-cloud
      apiBase: http://localhost:11434/v1
      apiKey: ollama
  mcpServers:
    - name: reachy-mini
      command: /Users/mars/.local/bin/poetry
      args: ["-C", "/Users/mars/Dev/reachy-mini-mcp", "run", "python", "src/server.py"]
      # MCP server repo: https://github.com/mvara-ai/reachy-mini-mcp

Description

Certain Ollama cloud models (Mistral Large 3, Ministral 3, Gemma3 27B) fail during MCP tool calling when Continue sends the conversation back to the model after tool execution (Turn 2 of the tool calling flow).

Tool Calling Flow

1. Turn 1: User prompt → Continue → Ollama → Model generates tool_call → Continue executes tool via MCP ✅ Works
2. Turn 2: Continue sends conversation + tool result back to Ollama → ❌ Fails with errors

Errors

Mistral Large 3 / Ministral 3:

Error: 400 Bad Request: Unexpected role 'system' after role 'tool'

Gemma3 27B:

Error: 400 Bad Request: Invalid 'tool_calls': unknown variant 'index', expected one of `function`, `id`, `type`

Working Models (for comparison)

These Ollama cloud models successfully complete the full tool calling flow with the same MCP setup:

• Qwen3 Coder (480B Cloud) - Excellent tool calling support, recommended for MCP workflows
• Qwen3 VL (235B Cloud) - Vision model with working tool calling
• Qwen3 Next (80B Cloud) - Good performance with tools
• Cogito 2.1 (671B Cloud) - Uses custom <|tool▁candidate▁N|> format, which Continue parses correctly
• GLM 4.6 (Cloud) - Working tool support
• Minimax M2 (Cloud) - Working tool support
• Kimi K2 (1T Cloud) - Large model with working tool calling

Note: All working models handle the Turn 2 message structure correctly, suggesting the issue is specific to how Mistral/Gemma models parse the conversation history after tool execution.

To Reproduce

1. Configure Continue with Ollama cloud models (Mistral Large 3, Ministral 3, or Gemma3 27B)
2. Set up any MCP server with tools (e.g., reachy-mini, filesystem)
3. Start Continue CLI: cn --config ~/.continue/config.yaml
4. Ask the model to use an MCP tool (e.g., "use reachy-mini to say hello")
5. Observe:
   • Turn 1 works: Model generates tool_call, Continue executes it
   • Turn 2 fails: Error occurs when sending conversation back to model

Expected Behavior

Models should successfully complete the tool calling flow:

1. Generate tool_call
2. Continue executes tool
3. Model receives tool result and generates final response

Actual Behavior

Mistral/Ministral: Fail with "Unexpected role 'system' after role 'tool'" - these models don't accept system messages after tool messages in the conversation history.

Gemma3: Fails with "Invalid 'tool_calls': unknown variant 'index'" - this model doesn't recognize the 'index' field in tool_calls structure.

Root Cause Analysis

The issue appears to be in how Continue formats messages before sending to Ollama's OpenAI-compatible endpoint (/v1/chat/completions). After tool execution, the message structure includes:

• System message (for tool documentation)
• Assistant message with tool_calls
• Tool result message (role: "tool")

Mistral/Ministral don't accept this ordering (system after tool).
Gemma3 doesn't accept the 'index' field in tool_calls.

Proposed Solution

Add message normalization in streamChatResponse.ts before sending to the model API (around line 259, after convertFromUnifiedHistoryWithSystemMessage):

1. For Mistral/Ministral: Move system messages before any tool interactions
2. For Gemma3: Remove 'index' field from tool_calls structure

This would be a model-specific compatibility layer that ensures message formatting matches what each model expects.

Additional Context

• These same models work correctly in other contexts (model-gateway using Ollama's native Python client)
• The issue is specific to Ollama's OpenAI compatibility layer via Continue
• MCP server communication works perfectly (stdio-based)
• The models can generate tool_calls successfully in Turn 1
• Only Turn 2 (after tool execution) fails

Logs

Available upon request - can provide full Continue CLI logs showing the exact message structure being sent and error responses.

[continue]

Related Issues

Found the following related issues about MCP tool calling and message formatting:

• Bug: DeepSeek R1 (Reasoner) fails with MCP tools - 400 Missing reasoning_content #9245 - DeepSeek R1 (Reasoner) fails with MCP tools - 400 Missing reasoning_content
• When use deepseek v3.2 as agent, throw "400 Missing reasoning_content field in the assistant message at message index 2" #8989 - When use deepseek v3.2 as agent, throw "400 Missing reasoning_content field in the assistant message at message index 2"
• continue tool calling with supported models not working #9157 - continue tool calling with supported models not working

These issues involve similar message formatting/compatibility problems during tool calling, though with different models (DeepSeek vs Ollama).