docs: added clarifai provider support #6730
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…nge (vercel#5531) Co-authored-by: Carl Brugger <cebrugg@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…#5541) Co-authored-by: Bram Meerten <bram.meerten@acagroup.be>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…spans (vercel#6357) ## Background `generateObject`, `generateText`, `streamText`, and `streamObject` currently call `JSON.stringify` on the input messages. If the input messages contain an image, it is most likely normalized into a `Uint8Array`. `JSON.stringify` does not the most obvious things to TypedArrays including `Uint8Array`. ```javascript // this returns '{"0": 1,"1": 2,"2": 3}', where I'd expect this to be '[1,2,3]' JSON.stringify(new Uint8array([1, 2, 3])) ``` In practice, this results in bloating images by about 5-15x depending on the original image size. For Laminar, for example, a span with 3 avg sized images will not be able to be sent as it is larger than the (reasonably high) gRPC payload size for our traces endpoint. From [MDN docs](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#examples): ```javascript // TypedArray JSON.stringify([new Int8Array([1]), new Int16Array([1]), new Int32Array([1])]); // '[{"0":1},{"0":1},{"0":1}]' JSON.stringify([ new Uint8Array([1]), new Uint8ClampedArray([1]), new Uint16Array([1]), new Uint32Array([1]), ]); // '[{"0":1},{"0":1},{"0":1},{"0":1}]' JSON.stringify([new Float32Array([1]), new Float64Array([1])]); // '[{"0":1},{"0":1}]' ``` ## Summary Added a function that maps over messages in a `LanguageModelV1Prompt` and maps over content parts in each message, replacing `UInt8Array`s with raw base64 strings instead. Call this function when calling `recordSpan` for the inner (doStream/doGenerate) span in `generateObject`, `generateText`, `streamText`, and `streamObject`. ## Verification Ran this small script against a local instance of Laminar and logged the Telemetry payloads (span attributes) on the backend to verify that they are indeed base64. ```javascript import { Laminar, getTracer } from '@lmnr-ai/lmnr' Laminar.initialize(); import { openai } from '@ai-sdk/openai' import { generateText, generateObject, streamText, streamObject, tool } from "ai"; import { z } from "zod"; import dotenv from "dotenv"; dotenv.config(); const handle = async () => { const imageUrl = "https://upload.wikimedia.org/wikipedia/commons/b/bc/CoinEx.png" const imageData = await fetch(imageUrl) .then(response => response.arrayBuffer()) .then(buffer => Buffer.from(buffer).toString('base64')); const o = streamObject({ schema: z.object({ text: z.string(), companyName: z.string().optional().nullable(), }), messages: [ { role: "user", content: [ { type: "text", text: "Describe this image briefly" }, { type: "image", image: imageData, mimeType: "image/png" } ] } ], model: openai("gpt-4.1-nano"), experimental_telemetry: { isEnabled: true, tracer: getTracer() } }); for await (const chunk of o.fullStream) { console.log(chunk); } await Laminar.shutdown(); }; handle().then((r) => { console.log(r); }); ``` ## Related Issues Fixes vercel#6210
## Background The Codex VM uses pnpm 10. I am running into build issues with Codex and want to rule out that they are caused by pnpm version differences. ## Summary Upgrade pnpm to version 10.
## Background Vercel is adding an API for the v0 model per https://vercel.com/docs/v0/api and an AI SDK provider would make it easier for developers to interact with it. ## Summary Added an initial provider for the Vercel API. ## Verification Added examples across several features and executed end to end manually. Co-authored-by: Walter Korman <shaper@vercel.com>
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @ai-sdk/vercel@0.0.1 ### Patch Changes - 42e37fb: feat (provider/vercel): initial vercel provider Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Background This pull request adds support for Anthropic's new Claude v4 models. ## Summary Updated the model ids to include `claude-4-opus-20250514` and `claude-4-sonnet-20250514`. ## Tasks - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root)
# Releases ## @ai-sdk/anthropic@1.2.12 ### Patch Changes - f64f4f0: feat (providers/anthropic): add claude v4 models ## @ai-sdk/google-vertex@2.2.23 ### Patch Changes - Updated dependencies [f64f4f0] - @ai-sdk/anthropic@1.2.12 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Summary Adds guide for Claude 4 models. ## Tasks - [ ] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [ ] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root)
## Background Fal added support for new Flux Kontext models. ## Summary Adds model ids, example, and updates docs. ## Tasks - [ ] Tests have been added / updated (for bug fixes / features) - [x] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root) ## Future Work We might want to add typed provider options here. Currently just pass through the entire provider options into the body.
## Background Backport of vercel#6567 The assign-team-pr-to-author CI action has been failing reliably with an error as: ``` Run gh pr edit $PULL_REQUEST_URL --add-assignee $AUTHOR_LOGIN failed to run git: fatal: not a git repository (or any of the parent directories): .git ``` ## Summary Claude claims best practices are to check out the repo before running `gh`. While it may have worked without a local checkout in the past, `gh` is typically expected to require the repository to be present locally ahead of time. ## Verification Will need to see how subsequent PRs go. Co-authored-by: Walter Korman <shaper@vercel.com>
# Releases ## @ai-sdk/fal@0.1.12 ### Patch Changes - 2e6e5d3: feat (@ai-sdk/fal): support new Flux Kontext models Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Description Add support for new Claude 4 models in Amazon Bedrock provider: - `anthropic.claude-sonnet-4-20250514-v1:0` - `anthropic.claude-opus-4-20250514-v1:0` ## Changes - Added new model IDs to `BedrockChatModelId` type in `bedrock-chat-settings.ts` - Added changeset for version bump ## Notes These are the latest Claude 4 models available in Amazon Bedrock as of May 2025.
…ercel#6597) ## Background The Google API for reasoning was changed in their latest model, leading to Zod errors. ## Summary Make text optional in thinking chunks. Ignore thinking chunks without text. ## Verification Tested example against google api. ## Future work Expose thinking signature using provider metadata, and explore sending it to google in follow-up requests. ## Related Issues Fixes vercel#6589
# Releases ## @ai-sdk/amazon-bedrock@2.2.10 ### Patch Changes - 05b8324: feat (provider/amazon-bedrock): add Claude 4 model ids (claude-sonnet-4-20250514-v1:0, claude-opus-4-20250514-v1:0) ## @ai-sdk/google@1.2.19 ### Patch Changes - f262012: fix (provider/google): prevent error when thinking signature is used ## @ai-sdk/google-vertex@2.2.24 ### Patch Changes - Updated dependencies [f262012] - @ai-sdk/google@1.2.19 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Background Some APIs changed and v5 site is launched. ## Summary Updated continueUntil to stopWhen and added reference to new v5 site. ## Tasks - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root)
## Background Google 2.5 model family now supports implicit caching. ## Summary Update docs. ## Tasks - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root)
**Background** Added documentation for the Requesty AI SDK provider to help developers integrate with Requesty's unified LLM gateway. Summary Created comprehensive documentation for the Requesty provider (content/providers/03-community-providers/5-requesty.mdx) including setup instructions, API key configuration, usage examples, and advanced features. Also added Requesty to the main providers list in the foundations documentation. **Tasks** - [x] Documentation has been added (for new provider) - [x] Formatting issues have been fixed --------- Co-authored-by: nicoalbanese <gcalbanese96@gmail.com>
## Background Adding Smithery to the docs as an additional toolkit. ## Verification Only the documentation was changed. No tests required. Co-authored-by: Nico Albanese <49612682+nicoalbanese@users.noreply.github.com>
|
@lgrammel not sure what happened but I'm unable to resolve all the conflicts in this PR, will cherry-pick my commits and create a new PR |
2 tasks
|
Reopened without merge conflicts in #6990 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
This change adds Clarifai as an openai compatible provider for Vercel AI SDK
Summary
This change adds a page to the openai-compatible-providers path that explains how to use the Vercel AI SDK with Clarifai
Have also included examples for
generate-text,stream-text&generate-objectmethods.Verification
Clarifai provides a OpenAI-compatible API endpoint to ensure users can easily access models using their existing tools like
openainpm packageWe also ensure model compatibility with Vercel AI SDK and keep a comprehensive list of model capabilities in our docs
I have also setup E2E tests for the examples included in this PR in the Repo: https://github.com/DaniAkash/clarifai-vercel-sdk
Test Run: https://github.com/DaniAkash/clarifai-vercel-sdk/actions/runs/15616667559/job/43991126428
Tasks
pnpm prettier-fixin the project root)Future Work
We plan to include more examples on using Clarifai with Vercel AI SDK