feat(openai-compatible): passing config to chat models #6849
Conversation
samurai00
force-pushed
the
feat-passing-config-to-chat-models
branch
from
1d638ae to
f4f171d
Compare
samurai00
changed the title
Refine the `stream_options` handling in `OpenAICompatibleChatLanguageModel` to correctly omit the `stream_options` field when `includeUsage` is false/undefined, preventing `undefined` from being sent in the request body.
samurai00
force-pushed
the
feat-passing-config-to-chat-models
branch
from
c70af4b to
fb2c151
Compare
| stream_options: this.config.includeUsage | ||
| ? { include_usage: true } | ||
| : undefined, |
There was a problem hiding this comment.
we use json stringify, which omits undefined values from objects, so how is this any different in terms of the body that is sent?
There was a problem hiding this comment.
Thanks for the feedback!
With the original code, if includeUsage is false, stream_options is always set to undefined, wiping out any value that providerOptions might have had. The new code only applies the override when includeUsage is true, leaving the original stream_options untouched otherwise.
There was a problem hiding this comment.
when would stream_options be included in args?
.changeset/hot-eggs-obey.md
Outdated
| '@ai-sdk/openai-compatible': patch | ||
| --- | ||
|
|
||
| Allow passing config to chat models; Refine the `stream_options` handling in `OpenAICompatibleChatLanguageModel` to correctly omit the `stream_options` field when `includeUsage` is false/undefined, preventing `undefined` from being sent in the request body. |
There was a problem hiding this comment.
So to confirm: should stream_options only be determined by config.includeUsage, and not by what's in providerOptions?
There was a problem hiding this comment.
we can change that in v5 but for v4 this is sufficient imo
3 participants
Background
Cannot pass the
stream_optionsto the OpenAICompatible Providers.Summary
stream_optionshandling inOpenAICompatibleChatLanguageModelto correctly omit thestream_optionsfield whenincludeUsageis false/undefined, preventingundefinedfrom being sent in the request body.Verification
Tasks
pnpm changesetin the project root)pnpm prettier-fixin the project root)Future Work
Related Issues
Fixes #6774