feat: support images in mcp tool responses#4100
Conversation
d8cedfb to
c450a57
Compare
|
One thing to point out: this only supports images in mcp tools, not tools from the registry. However, I realize that extracting base64 data from the url might get a bit messy and don't want to block this PR bc of that so leaving it out. |
772b621 to
eb855e1
Compare
c450a57 to
27c57b2
Compare
|
gonna merge today sorry for delays |
|
/oc revert the toModelOutput changes but keep the rest as is. If you don't know what im saying read the previous commits. DO NOT commit your changes |
|
Perfect! I've reverted the |
Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
|
Im gonna merge it like this, I mentioned on discord I dont think the toModelOutput change was needed? And tbh I think we can delete that block altogether but maybe im missing something |
| output, | ||
| output: textParts.join("\n\n"), | ||
| attachments, | ||
| content: result.content, // directly return content to preserve ordering when outputting to model |
There was a problem hiding this comment.
remember to remove this when removing toModelOutput
took a quick skim, looks like toModelOutput isn't used as it was replaced with MessageV2.toModelMessage() just left a note to not forget to remove this as it was directly used for toModelOutput when I thought this was the function that sent messages to the model |
Co-authored-by: opencode-agent[bot] <opencode-agent[bot]@users.noreply.github.com> Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
Co-authored-by: opencode-agent[bot] <opencode-agent[bot]@users.noreply.github.com> Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
Co-authored-by: opencode-agent[bot] <opencode-agent[bot]@users.noreply.github.com> Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
Co-authored-by: opencode-agent[bot] <opencode-agent[bot]@users.noreply.github.com> Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
Co-authored-by: opencode-agent[bot] <opencode-agent[bot]@users.noreply.github.com> Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
Co-authored-by: opencode-agent[bot] <opencode-agent[bot]@users.noreply.github.com> Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>

Vision-capable models can now consume images from mcp tool responses.
https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#multi-modal-tool-results
This resolves the merge conflict in the original PR: #1589
Closes: #1563