Repository navigation
Which event should carry a widget that a server tool produces: a tool call, ACTIVITY_SNAPSHOT or CUSTOM?
#3402
Unanswered
epignosisx
asked this question in
Q&A
Replies: 1 comment
|
Not a maintainer, just someone who read the 1.0 spec closely ( 1. Server-produced widgets → activity (Option 2)
2. UI-only tool calls (Option 1) → I'd move away from them
3. Interactive widgets → split it in two
Short version
|
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hi all! We're building a chat client on AG-UI 1.0, with a web app and a React Native app talking to one agent backend. We'd like the community's view on which event should carry a widget to the client.
The setup
show_product_card, and the tool builds a UI payload.MESSAGES_SNAPSHOT.So the payload is produced by the server, not proposed by the model for the client to execute. The question is which event to deliver it with. These are the three options we've looked at.
Option 1: a tool call (
TOOL_CALL_START/ARGS/END)What we do today: the server emits a separate tool call named after the widget type (
card-carousel,guided-question), with the payload as its arguments. The model's own tool call stays on the server.Pros
toolCalls, so it keeps its place in the sequence and comes back inMESSAGES_SNAPSHOTlike any other message. Reload works without extra machinery.toolCallIdgives interactive widgets a stable key for their stored state and for linking the user's answer to the widget.Cons
TOOL_CALL_RESULTfor it. The 1.0 spec says a consumer derives pending calls as "every tool call the run started that received no result" and "MUST NOT read absence as 'nothing pending'" (tool calls → Frontend tools; see also Spec clarification: what should `RUN_FINISHED.outcome` be when a run ends with unresolved client-side tool calls? #2564). A strictly conforming client would treat every carousel as an unanswered frontend tool call. Ours only acts on tool names it advertised intools, but that's our own convention.TOOL_CALL_RESULTright after it would stop it reading as pending. But resultcontentis meant for the model (a string or parts), and nothing would be reading it.Option 2: an activity message (
ACTIVITY_SNAPSHOT/ACTIVITY_DELTA)The server answers the model's tool call normally with
TOOL_CALL_RESULT, then emits anACTIVITY_SNAPSHOTwithactivityType: "card-carousel"and the payload ascontent.Pros
TOOL_CALL_RESULT, thenACTIVITY_SNAPSHOTwithactivityType: "mcp-apps"). The A2UI middleware does the same, keying surfaces per tool call.agent.ts), so widget payloads stay in the Agent↔Client conversation. They're stored in the client transcript like any other message. ([Feature]: Restore A2UI surfaces from a MESSAGES_SNAPSHOT of durable tool results #2708 shows what goes wrong when a backend builds snapshots from the model's history instead.)Cons
contentshape for eachactivityTypeisn't specified (also noted in A2UI with a Python (Strands) agent and non-CopilotKit clients: is a TypeScript `A2UIMiddleware` tier required? #2845).userandassistantmessages only, so activity messages aren't rendered (CopilotChat.tsx). That matters to us because we ship a React Native app.Option 3: a
CUSTOMeventThe server emits
CUSTOM { name: "widget.card-carousel", value: payload }.Pros
Cons
MESSAGES_SNAPSHOT, so we'd need our own storage, ordering and merging.Our questions
TOOL_CALL_RESULTthe right way to keep conforming consumers from treating it as pending?messagesas a user message, or as something tied to the widget's id?Thanks! We're happy to share more detail or contribute docs once there's a direction.
All reactions