Feat(mistral) use actual response format #6327
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…nge (vercel#5531) Co-authored-by: Carl Brugger <cebrugg@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…#5541) Co-authored-by: Bram Meerten <bram.meerten@acagroup.be>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Summary Announce v5 Alpha ## Tasks - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root)
Collaborator
|
|
…spans (vercel#6357) ## Background `generateObject`, `generateText`, `streamText`, and `streamObject` currently call `JSON.stringify` on the input messages. If the input messages contain an image, it is most likely normalized into a `Uint8Array`. `JSON.stringify` does not the most obvious things to TypedArrays including `Uint8Array`. ```javascript // this returns '{"0": 1,"1": 2,"2": 3}', where I'd expect this to be '[1,2,3]' JSON.stringify(new Uint8array([1, 2, 3])) ``` In practice, this results in bloating images by about 5-15x depending on the original image size. For Laminar, for example, a span with 3 avg sized images will not be able to be sent as it is larger than the (reasonably high) gRPC payload size for our traces endpoint. From [MDN docs](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#examples): ```javascript // TypedArray JSON.stringify([new Int8Array([1]), new Int16Array([1]), new Int32Array([1])]); // '[{"0":1},{"0":1},{"0":1}]' JSON.stringify([ new Uint8Array([1]), new Uint8ClampedArray([1]), new Uint16Array([1]), new Uint32Array([1]), ]); // '[{"0":1},{"0":1},{"0":1},{"0":1}]' JSON.stringify([new Float32Array([1]), new Float64Array([1])]); // '[{"0":1},{"0":1}]' ``` ## Summary Added a function that maps over messages in a `LanguageModelV1Prompt` and maps over content parts in each message, replacing `UInt8Array`s with raw base64 strings instead. Call this function when calling `recordSpan` for the inner (doStream/doGenerate) span in `generateObject`, `generateText`, `streamText`, and `streamObject`. ## Verification Ran this small script against a local instance of Laminar and logged the Telemetry payloads (span attributes) on the backend to verify that they are indeed base64. ```javascript import { Laminar, getTracer } from '@lmnr-ai/lmnr' Laminar.initialize(); import { openai } from '@ai-sdk/openai' import { generateText, generateObject, streamText, streamObject, tool } from "ai"; import { z } from "zod"; import dotenv from "dotenv"; dotenv.config(); const handle = async () => { const imageUrl = "https://upload.wikimedia.org/wikipedia/commons/b/bc/CoinEx.png" const imageData = await fetch(imageUrl) .then(response => response.arrayBuffer()) .then(buffer => Buffer.from(buffer).toString('base64')); const o = streamObject({ schema: z.object({ text: z.string(), companyName: z.string().optional().nullable(), }), messages: [ { role: "user", content: [ { type: "text", text: "Describe this image briefly" }, { type: "image", image: imageData, mimeType: "image/png" } ] } ], model: openai("gpt-4.1-nano"), experimental_telemetry: { isEnabled: true, tracer: getTracer() } }); for await (const chunk of o.fullStream) { console.log(chunk); } await Laminar.shutdown(); }; handle().then((r) => { console.log(r); }); ``` ## Related Issues Fixes vercel#6210
## Background The Codex VM uses pnpm 10. I am running into build issues with Codex and want to rule out that they are caused by pnpm version differences. ## Summary Upgrade pnpm to version 10.
Author
|
@lgrammel Fixed a bug and added a test case. Good to merge now. |
## Background Vercel is adding an API for the v0 model per https://vercel.com/docs/v0/api and an AI SDK provider would make it easier for developers to interact with it. ## Summary Added an initial provider for the Vercel API. ## Verification Added examples across several features and executed end to end manually. Co-authored-by: Walter Korman <shaper@vercel.com>
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @ai-sdk/vercel@0.0.1 ### Patch Changes - 42e37fb: feat (provider/vercel): initial vercel provider Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Background This pull request adds support for Anthropic's new Claude v4 models. ## Summary Updated the model ids to include `claude-4-opus-20250514` and `claude-4-sonnet-20250514`. ## Tasks - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root)
# Releases ## @ai-sdk/anthropic@1.2.12 ### Patch Changes - f64f4f0: feat (providers/anthropic): add claude v4 models ## @ai-sdk/google-vertex@2.2.23 ### Patch Changes - Updated dependencies [f64f4f0] - @ai-sdk/anthropic@1.2.12 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Summary Adds guide for Claude 4 models. ## Tasks - [ ] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [ ] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root)
lgrammel
reviewed
May 23, 2025
Comment on lines
+2
to
+3
| '@ai-sdk/mistral': minor | ||
| 'ai': minor |
lgrammel
reviewed
May 23, 2025
Comment on lines
+540
to
+543
| responseFormat: { | ||
| schema: outputStrategy.jsonSchema, | ||
| type: 'json', | ||
| }, |
Collaborator
There was a problem hiding this comment.
in the object json mode mode.schema should be used. pls remove
Author
There was a problem hiding this comment.
But is the mode not depricated?
The reason why I added responseFormat there was that I've seen that mistral (and I assume others) handle it with a warning, but responseFormat was from what Ive seen never set if you passed on an schema. if I remove this I have to modify my code to handle the schema by the depricated mode field.
If you are still set on removal of this I can do that.
## Background Fal added support for new Flux Kontext models. ## Summary Adds model ids, example, and updates docs. ## Tasks - [ ] Tests have been added / updated (for bug fixes / features) - [x] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root) ## Future Work We might want to add typed provider options here. Currently just pass through the entire provider options into the body.
## Background Backport of vercel#6567 The assign-team-pr-to-author CI action has been failing reliably with an error as: ``` Run gh pr edit $PULL_REQUEST_URL --add-assignee $AUTHOR_LOGIN failed to run git: fatal: not a git repository (or any of the parent directories): .git ``` ## Summary Claude claims best practices are to check out the repo before running `gh`. While it may have worked without a local checkout in the past, `gh` is typically expected to require the repository to be present locally ahead of time. ## Verification Will need to see how subsequent PRs go. Co-authored-by: Walter Korman <shaper@vercel.com>
# Releases ## @ai-sdk/fal@0.1.12 ### Patch Changes - 2e6e5d3: feat (@ai-sdk/fal): support new Flux Kontext models Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Description Add support for new Claude 4 models in Amazon Bedrock provider: - `anthropic.claude-sonnet-4-20250514-v1:0` - `anthropic.claude-opus-4-20250514-v1:0` ## Changes - Added new model IDs to `BedrockChatModelId` type in `bedrock-chat-settings.ts` - Added changeset for version bump ## Notes These are the latest Claude 4 models available in Amazon Bedrock as of May 2025.
…ercel#6597) ## Background The Google API for reasoning was changed in their latest model, leading to Zod errors. ## Summary Make text optional in thinking chunks. Ignore thinking chunks without text. ## Verification Tested example against google api. ## Future work Expose thinking signature using provider metadata, and explore sending it to google in follow-up requests. ## Related Issues Fixes vercel#6589
# Releases ## @ai-sdk/amazon-bedrock@2.2.10 ### Patch Changes - 05b8324: feat (provider/amazon-bedrock): add Claude 4 model ids (claude-sonnet-4-20250514-v1:0, claude-opus-4-20250514-v1:0) ## @ai-sdk/google@1.2.19 ### Patch Changes - f262012: fix (provider/google): prevent error when thinking signature is used ## @ai-sdk/google-vertex@2.2.24 ### Patch Changes - Updated dependencies [f262012] - @ai-sdk/google@1.2.19 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Background Some APIs changed and v5 site is launched. ## Summary Updated continueUntil to stopWhen and added reference to new v5 site. ## Tasks - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root)
## Background Google 2.5 model family now supports implicit caching. ## Summary Update docs. ## Tasks - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root)
**Background** Added documentation for the Requesty AI SDK provider to help developers integrate with Requesty's unified LLM gateway. Summary Created comprehensive documentation for the Requesty provider (content/providers/03-community-providers/5-requesty.mdx) including setup instructions, API key configuration, usage examples, and advanced features. Also added Requesty to the main providers list in the foundations documentation. **Tasks** - [x] Documentation has been added (for new provider) - [x] Formatting issues have been fixed --------- Co-authored-by: nicoalbanese <gcalbanese96@gmail.com>
Collaborator
|
Sorry, we didn't get to this in time. Please re-create the PR if the change is still needed and we'll take a look! This comment is automated as part of our triage process. Truly sorry for the inconvenieince, we are doing our best 🖤 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
Currently when calling the mistral AI we are passing on the response schema as an message, but do not actually make use of the structured output endpoint. This commit changes that.
Summary
Verification
I manually ran the mistral file inside of the ai core generate-object endpoint. And it worked now as expected. (I would prefer if someone could look over it as well to be certain and test if my ResponseFormat fix actually impacted other packages. (Many packages have error handling for ResponseFormat / warnings for it, but since it always returned undefined those we're never invoked (from what I know, I did not check other packages besides mistral and a bit openai)
Tasks
pnpm changesetin the project root)pnpm prettier-fixin the project root)Future Work
People would now need to go around and update all other packages to move away from the depricated mode field. This should lay the first step to make ResponseFormat usuable.
Related Issues
Did not check.