Skip to content

Conversation roles must alternate" error with vLLM/Mistral after aborted responses #11460

Description

@jaydaftari

When using LibreChat with vLLM-served models (Mistral, Mixtral, Llama, etc.), the error "Conversation roles must alternate user/assistant/user/assistant/..." occurs after aborting a response or when there are empty/partial assistant messages in the conversation history.

Steps to Reproduce

  1. Connect LibreChat to a vLLM backend serving Mistral/Mixtral
  2. Send a message to start a conversation
  3. While the assistant is responding, abort/cancel the response
  4. Send another message
  5. Error occurs: "Conversation roles must alternate user/assistant/user/assistant/..."

Alternative reproduction:

  • Any scenario where an assistant message is saved with empty content or unfinished: true

Expected Behavior

LibreChat should sanitize the message history before sending to the LLM by:

  • Merging consecutive messages of the same role
  • Filtering out empty/unfinished assistant messages
  • Ensuring proper role alternation for strict chat template models

Actual Behavior

The raw message history is sent to the LLM, resulting in sequences like:

  • user → assistant (empty/partial) → user (treated as user → user)
  • user → assistant → assistant → user (consecutive same roles)

This violates the strict chat template requirements of Mistral/Mixtral and other models.

Technical Analysis

Where the issue originates:

Location Issue
api/server/controllers/agents/request.js:114 Saves partial messages with unfinished: true that may be empty
@librechat/agents formatAgentMessages Doesn't merge consecutive same-role messages
client.js buildMessages Doesn't validate role alternation before sending

Evidence of awareness:

This could be implemented in:
formatAgentMessages in @librechat/agents
Or as a pre-processing step in buildMessages
Environment
LibreChat version: [your version]
LLM Backend: vLLM
Model: Mistral/Mixtral (or other strict chat template model)
Deployment: Docker/Local
Related Issues
#4089 - Artifacts UI causes issues with vLLM (same root cause)
#11344 - Memory Agent fails with Bedrock (similar role ordering issue, fixed in #11353)
vllm-project/vllm#2112 - Conversation roles must alternate
Additional Context
This issue affects all users connecting LibreChat to:
vLLM with Mistral/Mixtral models
Any LLM backend enforcing strict role alternation
Potentially Bedrock, Azure, and other providers with strict message formatting
The fix would benefit the entire community using non-OpenAI backends that enforce stricter API contracts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions