When using LibreChat with vLLM-served models (Mistral, Mixtral, Llama, etc.), the error "Conversation roles must alternate user/assistant/user/assistant/..." occurs after aborting a response or when there are empty/partial assistant messages in the conversation history.
Steps to Reproduce
- Connect LibreChat to a vLLM backend serving Mistral/Mixtral
- Send a message to start a conversation
- While the assistant is responding, abort/cancel the response
- Send another message
- Error occurs:
"Conversation roles must alternate user/assistant/user/assistant/..."
Alternative reproduction:
- Any scenario where an assistant message is saved with empty content or
unfinished: true
Expected Behavior
LibreChat should sanitize the message history before sending to the LLM by:
- Merging consecutive messages of the same role
- Filtering out empty/unfinished assistant messages
- Ensuring proper role alternation for strict chat template models
Actual Behavior
The raw message history is sent to the LLM, resulting in sequences like:
user → assistant (empty/partial) → user (treated as user → user)
user → assistant → assistant → user (consecutive same roles)
This violates the strict chat template requirements of Mistral/Mixtral and other models.
Technical Analysis
Where the issue originates:
| Location |
Issue |
api/server/controllers/agents/request.js:114 |
Saves partial messages with unfinished: true that may be empty |
@librechat/agents formatAgentMessages |
Doesn't merge consecutive same-role messages |
client.js buildMessages |
Doesn't validate role alternation before sending |
Evidence of awareness:
This could be implemented in:
formatAgentMessages in @librechat/agents
Or as a pre-processing step in buildMessages
Environment
LibreChat version: [your version]
LLM Backend: vLLM
Model: Mistral/Mixtral (or other strict chat template model)
Deployment: Docker/Local
Related Issues
#4089 - Artifacts UI causes issues with vLLM (same root cause)
#11344 - Memory Agent fails with Bedrock (similar role ordering issue, fixed in #11353)
vllm-project/vllm#2112 - Conversation roles must alternate
Additional Context
This issue affects all users connecting LibreChat to:
vLLM with Mistral/Mixtral models
Any LLM backend enforcing strict role alternation
Potentially Bedrock, Azure, and other providers with strict message formatting
The fix would benefit the entire community using non-OpenAI backends that enforce stricter API contracts.
When using LibreChat with vLLM-served models (Mistral, Mixtral, Llama, etc.), the error
"Conversation roles must alternate user/assistant/user/assistant/..."occurs after aborting a response or when there are empty/partial assistant messages in the conversation history.Steps to Reproduce
"Conversation roles must alternate user/assistant/user/assistant/..."Alternative reproduction:
unfinished: trueExpected Behavior
LibreChat should sanitize the message history before sending to the LLM by:
Actual Behavior
The raw message history is sent to the LLM, resulting in sequences like:
user → assistant (empty/partial) → user(treated asuser → user)user → assistant → assistant → user(consecutive same roles)This violates the strict chat template requirements of Mistral/Mixtral and other models.
Technical Analysis
Where the issue originates:
api/server/controllers/agents/request.js:114unfinished: truethat may be empty@librechat/agentsformatAgentMessagesclient.jsbuildMessagesEvidence of awareness:
This could be implemented in:
formatAgentMessages in @librechat/agents
Or as a pre-processing step in buildMessages
Environment
LibreChat version: [your version]
LLM Backend: vLLM
Model: Mistral/Mixtral (or other strict chat template model)
Deployment: Docker/Local
Related Issues
#4089 - Artifacts UI causes issues with vLLM (same root cause)
#11344 - Memory Agent fails with Bedrock (similar role ordering issue, fixed in #11353)
vllm-project/vllm#2112 - Conversation roles must alternate
Additional Context
This issue affects all users connecting LibreChat to:
vLLM with Mistral/Mixtral models
Any LLM backend enforcing strict role alternation
Potentially Bedrock, Azure, and other providers with strict message formatting
The fix would benefit the entire community using non-OpenAI backends that enforce stricter API contracts.