fix(mistral): forward stopSequences as Mistral native stop parameter#15200
Conversation
|
@cristiandrei1234 can you attach the repro script you used to verify the fix? ideally it produced the warning before and then worked after |
|
I'd already deleted the original ad-hoc script after the verification run, so I rewrote a fresh one against the local build of @ai-sdk/mistral and ran it in both states. Same prompt, same stopSequences: ['Pacific'], the only thing that changes between runs is whether the fix is applied to mistral-chat-language-model.ts (toggled via git checkout HEAD~1 -- ... + pnpm -F @ai-sdk/mistral build). Gist: https://gist.github.com/cristiandrei1234/ebf0b79e757c3aa470492774a8a34505 Without the fix --- request.body (raw payload sent to api.mistral.ai) --- --- warnings --- --- finishReason --- --- content text ---
No stop field in the request, the warning is emitted, and the model runs to
No stop field in the request, the warning is emitted, and the model runs to completion — Pacific appears twice in the output, untruncated. The stop finishReason here is just the natural end-of-message, not a stop-sequence hit. With the fix applied --- request.body (raw payload sent to api.mistral.ai) --- --- warnings --- --- finishReason --- --- content text --- stop: ['Pacific'] is on the wire, no warning, and Mistral truncates mid-token immediately before the stop word with finish_reason: "stop". Happy to fold this into the PR description if you'd like it on the body rather than buried in a comment. |
|
|
…parameter (#15293) This is an automated backport of #15200 to the release-v6.0 branch. FYI @cristiandrei1234 This backport has conflicts that need to be resolved manually. ### `git cherry-pick` output ``` Auto-merging packages/mistral/src/mistral-chat-language-model.test.ts Auto-merging packages/mistral/src/mistral-chat-language-model.ts CONFLICT (content): Merge conflict in packages/mistral/src/mistral-chat-language-model.ts error: could not apply 7acf358... fix(mistral): forward stopSequences as Mistral native stop parameter (#15200) hint: After resolving the conflicts, mark them with hint: "git add/rm <pathspec>", then run hint: "git cherry-pick --continue". hint: You can instead skip this commit with "git cherry-pick --skip". hint: To abort and get back to the state before "git cherry-pick", hint: run "git cherry-pick --abort". hint: Disable this message with "git config set advice.mergeConflict false" ``` --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Aayush Kapoor <aayushkapoor34@gmail.com> Co-authored-by: Aayush Kapoor <83492835+aayush-kapoor@users.noreply.github.com>
Background
@ai-sdk/mistralacceptsstopSequenceson the publicgenerateText/streamTextAPI but silently drops the value, emitting only a warning:[{ "type": "unsupported", "feature": "stopSequences" }]The Mistral REST API itself supports stop sequences natively via the
stopfield, which accepts a string or array of strings (Mistral API reference →stop: string | array<string>). The adapter's job is to translatestopSequences→stopin the outgoing request, but that translation is missing.Result: a documented, standard
generateTextparameter does nothing for Mistral users, with no error surfaced beyond the warning array.This is an isolated regression specific to
@ai-sdk/mistral. Five sibling providers already implement the exact same translation:@ai-sdk/openaipackages/openai/src/chat/openai-chat-language-model.ts:185@ai-sdk/openai-compatiblepackages/openai-compatible/src/chat/openai-compatible-chat-language-model.ts:295@ai-sdk/groqpackages/groq/src/groq-chat-language-model.ts:146@ai-sdk/deepseekpackages/deepseek/src/chat/deepseek-chat-language-model.ts:176@ai-sdk/alibabapackages/alibaba/src/alibaba-chat-language-model.ts:128All of them do literally
stop: stopSequences,in their requestbaseArgs. Mistral is the odd one out.Summary
packages/mistral/src/mistral-chat-language-model.ts:if (stopSequences != null) warnings.push({ type: 'unsupported', feature: 'stopSequences' })branch.stop: stopSequences,tobaseArgs, in the same position used by sibling providers (betweentop_pandrandom_seed).The change is minimal and matches an established intra-repo pattern. Vercel's
stopSequences?: string[]is a valid subtype of Mistral's documentedstop: string | array<string>, so direct assignment is correct.Manual Verification
Unit tests (deterministic, in-process)
Added a regression test in
mistral-chat-language-model.test.tsthat:doGeneratewithstopSequences: ['foo', 'bar']{ model, messages, stop: ['foo', 'bar'] }result.warningsdoes NOT contain{ type: 'unsupported', feature: 'stopSequences' }Validated against the unfixed branch (
git stashthe source change): the new test fails as expected, plus 7 inline-snapshot tests fail because the snapshots are updated to reflect the newstopfield inbaseArgs. With the fix applied: all 72 mistral package tests pass.Live API against
api.mistral.aiRan a small script that imports the local built
@ai-sdk/mistral(with the fix applied) and callsdoGenerateonmistral-small-latestwithstopSequences: ['Pacific'], prompt asking for facts about the Pacific Ocean:Mistral truncates the output mid-token immediately before "Pacific" and returns
finish_reason: "stop", confirming both the field name and the end-to-end behavior. This matches the curl-only verification the reporter already did in #15194 againstmistral-large-latest.Checklist
stopSequencesdocumentation incontent/docs/.../generate-text#stop-sequencespnpm changesetin the project root)Future Work
The reporter flagged that other adapters may emit
{ type: 'unsupported', feature: 'stopSequences' }while their underlying API actually supports it under a different field name. A repo-wide grep finds 7 other call sites with the same dropping pattern:Each requires individual verification against the respective provider's API spec — left out of scope here to keep this PR narrow and easy to review. Happy to follow up with separate PRs for providers whose REST APIs actually honor stop sequences (xAI and Perplexity look like obvious candidates given their OpenAI-compatible designs).
Related Issues
Fixes #15194.