Skip to content

feat(google): support agentic video processing in the Interactions API - #20244

Merged
aayush-kapoor merged 2 commits into
mainfrom
aayush/40
Sep 2, 2026
Merged

feat(google): support agentic video processing in the Interactions API#20244
aayush-kapoor merged 2 commits into
mainfrom
aayush/40

Conversation

@aayush-kapoor

Copy link
Copy Markdown
Collaborator

Background

Gemini interactions API now has the capability to agentically understand videos: https://aistudio.google.com/docs/video-understanding#agentic-video-understanding

we previously only supported static understanding. but now agentic is also available

Summary

  • video file parts now accept processing: agentic along with static - which are translated into the Gemini Interactions request format
  • gemini’s processing_call and processing_result steps are exposed as observable custom parts and preserved across stateless multi-turn requests

End-to-End Verification

verified by running the example http://localhost:3000/chat/google-interactions-agentic-video with a multiturn convo

Checklist

  • All commits are signed (PRs with unsigned commits cannot be merged)
  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • I have reviewed this pull request (self-review)

@aayush-kapoor
aayush-kapoor merged commit 18ad19c into main Sep 2, 2026
53 checks passed
@aayush-kapoor
aayush-kapoor deleted the aayush/40 branch September 2, 2026 19:42
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

🚀 Published in:

Package Version
ai 7.0.91 github npm
@ai-sdk/amazon-bedrock 5.0.73 github npm
@ai-sdk/angular 3.0.91 github npm
@ai-sdk/anthropic 4.0.49 github npm
@ai-sdk/anthropic-aws 2.0.41 github npm
@ai-sdk/azure 4.0.60 github npm
@ai-sdk/code-mode 1.0.48 github npm
@ai-sdk/gateway 4.0.73 github npm
@ai-sdk/google 4.0.63 github npm
@ai-sdk/google-vertex 5.0.75 github npm
@ai-sdk/harness 1.0.100 github npm
@ai-sdk/harness-acp 1.0.38 github npm
@ai-sdk/harness-claude-code 1.0.104 github npm
@ai-sdk/harness-cline 1.0.27 github npm
@ai-sdk/harness-codex 1.0.102 github npm
@ai-sdk/harness-cursor 1.0.13 github npm
@ai-sdk/harness-deepagents 1.0.100 github npm
@ai-sdk/harness-fx 1.0.13 github npm
@ai-sdk/harness-grok-build 1.0.37 github npm
@ai-sdk/harness-opencode 1.0.102 github npm
@ai-sdk/harness-pi 1.0.102 github npm
@ai-sdk/langchain 3.0.91 github npm
@ai-sdk/llamaindex 3.0.91 github npm
@ai-sdk/minimax 3.0.25 github npm
@ai-sdk/openai 4.0.57 github npm
@ai-sdk/otel 1.0.91 github npm
@ai-sdk/perplexity 4.0.39 github npm
@ai-sdk/policy-opa 1.0.91 github npm
@ai-sdk/react 4.0.94 github npm
@ai-sdk/rsc 3.0.91 github npm
@ai-sdk/sandbox-just-bash 1.0.100 github npm
@ai-sdk/sandbox-vercel 1.0.100 github npm
@ai-sdk/svelte 5.0.91 github npm
@ai-sdk/tui 1.0.92 github npm
@ai-sdk/vue 4.0.91 github npm
@ai-sdk/workflow 2.0.22 github npm
@ai-sdk/workflow-harness 1.0.100 github npm
@ai-sdk/xai 4.0.54 github npm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants