Repository navigation
A2UI with a Python (Strands) agent and non-CopilotKit clients: is a TypeScript A2UIMiddleware tier required?
#2845
Replies: 3 comments 1 reply
|
Any help/insights here @contextablemark @ranst91 ? Thanks |
|
No, the protocol doesn't need a TypeScript tier. There are three places the bridge can live:
This is the version I tested: import json
from ag_ui.core import ActivitySnapshotEvent, EventType, UserMessage
from ag_ui_a2ui_toolkit import A2UI_OPERATIONS_KEY
from ag_ui_strands import StrandsAgent
_OPS = ("createSurface", "updateComponents", "updateDataModel", "deleteSurface")
def _a2ui_ops(content):
if isinstance(content, list): # multimodal parts -> text
content = "".join(getattr(p, "text", "") or "" for p in content)
try:
data = json.loads(content)
if isinstance(data, str): # double-encoded, as the TS parser allows
data = json.loads(data)
except (TypeError, ValueError):
return None
ops = data.get(A2UI_OPERATIONS_KEY) if isinstance(data, dict) else None
return [op for op in ops if isinstance(op, dict)] if isinstance(ops, list) else None
def _surface_id(op):
return next((op[k]["surfaceId"] for k in _OPS if isinstance(op.get(k), dict) and "surfaceId" in op[k]), "default")
def describe_action(a):
text = f'User performed action "{a.get("name", "unknown_action")}" on surface "{a.get("surfaceId", "unknown_surface")}"'
if a.get("sourceComponentId"):
text += f' (component: {a["sourceComponentId"]})'
return text + f". Context: {json.dumps(a.get('context') or {})}"
class A2UIStrandsAgent(StrandsAgent):
async def run(self, input_data, **kwargs):
action = ((input_data.forwarded_props or {}).get("a2uiAction") or {}).get("userAction")
if action: # an action run carries no new user message; give the model one
msg = UserMessage(id=f"a2ui-action-{input_data.run_id}", role="user", content=describe_action(action))
input_data = input_data.model_copy(update={"messages": [*input_data.messages, msg]})
async for event in super().run(input_data, **kwargs):
yield event
if getattr(event, "type", None) != EventType.TOOL_CALL_RESULT:
continue
ops = _a2ui_ops(event.content)
if not ops:
continue
by_surface = {}
for op in ops:
by_surface.setdefault(_surface_id(op), []).append(op)
for sid, surface_ops in by_surface.items():
# same messageId scheme as A2UIMiddleware, so the two are idempotent
suffix = event.tool_call_id if len(by_surface) == 1 else f"{sid}-{event.tool_call_id}"
yield ActivitySnapshotEvent(
type=EventType.ACTIVITY_SNAPSHOT,
message_id=f"a2ui-surface-{suffix}",
activity_type="a2ui-surface",
content={A2UI_OPERATIONS_KEY: surface_ops},
replace=True,
)Use
Native clients need two things: render Two caveats. First, activities aren't durable. The client strips them from run input (agent.ts), so a reconnect drops surfaces (#2708). A Python bridge could re-emit them from stored tool results. Second, the content shape isn't in the spec. The A2A integration emits |
|
Thanks for your reply. I think my question was not so much technical feasibility (I could have let Claude re-implement the typescript middleware), but more a lack of clarity, documentation and vision. Because to me it seems like the entire concept of middleware which is explained in the documentation, is not possible to use in a python context, but this is not mentioned anywhere. Even the Kotlin ChatController you link to, refers to typescript files and copilotkit in the comments. So there just seem to be a lot of hidden implications in the ag-ui sdks that imply the user is using CopilotKit somewhere, which I don't think is the intent of a protocol like this. So that's why I'd like the maintainers to chip in on this. @contextablemark @ranst91 @mme |
Uh oh!
There was an error while loading. Please reload this page.
We're working out how to add A2UI surfaces to an existing AG-UI deployment, and we're stuck on something the docs seem to take for granted. I'd like to check whether we've misunderstood the intended architecture, and if we haven't, suggest what the docs could say.
Our setup
ag-ui-strands0.4.x. The AG-UI endpoint is served directly from the agent (AWS AgentCore). There's no Node tier in between.@ag-ui/clientHttpAgentin the browser (rendered with assistant-ui'sreact-ag-ui), plus native iOS and Android apps.generate_a2ui), with the button-action round trip working.What we found
The A2UI integration skill describes a server-side bridge that doesn't exist for Python.
a2ui-runtime-and-renderer.mdsays A2UI needs "a server-side AG-UI bridge". Every example appliesA2UIMiddlewareinside CopilotRuntime or withagent.use()in TypeScript.middlewares/a2ui-middlewareis TypeScript only, andsdks/pythonhas no middleware stage at all. The draft processing model says the same about Python and .NET in its enforcement note.No Python integration emits the
a2ui-surfaceactivity.ag_ui_strandsreturns thea2ui_operationsenvelope as an ordinaryTOOL_CALL_RESULT, and it re-emits the render sub-agent's progress as syntheticTOOL_CALL_*events. It never constructs anActivitySnapshotEvent. The LangGraph Python adapter readsinjectA2UITool"set by the A2UI middleware via forwardedProps". The Strands Python fixed-schema example says the result is what "the client's A2UIMiddleware scans". Each piece assumes a TypeScript consumer sits between the Python agent and the UI, but no page states that assumption or says where that consumer should live.The action round trip doesn't work on a Python agent without that middleware. The client sends
forwardedProps.a2uiAction.userAction, and only the TS middleware turns it into something the model sees: the syntheticlog_a2ui_eventpair.ag_ui_strandshas no handling fora2uiAction. An action run carries no new user message. Withreplay_history_into_strands=False, the adapter builds the prompt by walking back to the last user message, so the model gets the previous question again. As far as we can tell, the fix would be something likestate_context_builderreadingforwardedProps, but we found that by reading the adapter source rather than any documentation."Middleware" means three different things in the repo. It's the
agent.use()stream middleware (middlewares/*), the "middleware implementation" inquickstart/middleware.mdx(really an integration that produces events), andadk-middleware(the ADK integration). It took us a while to work out that the A2UI docs mean the first.Small contradiction on fixed-schema tool results.
framework-adapters.mdsays "return a plain object envelope" for Strands. The Python example insists on a JSON string, because a dict "may land in ajsonblock the adapter skips".Why this matters with more than one client
The spec calls middleware something a consumer installs. That puts
A2UIMiddlewarewherever the receivingAbstractAgentlives. With a browserHttpAgentit could run in the browser, but native clients have no equivalent: the community Kotlin, Dart and other SDKs have no middleware stage. Any A2UI logic kept in client middleware would have to be rewritten once per platform, so a Python producer serving several kinds of client has two options:A2UIMiddleware, orACTIVITY_SNAPSHOTand handlea2uiActionon the producer side, in the Python agent.The first option isn't attractive to us. It adds a service to deploy, scale and secure, and an extra hop on every streamed token, all to run what is essentially an event transform. The agent already speaks AG-UI directly to every client. Adding a TypeScript runtime to a Python stack just for A2UI feels like the protocol leaking an implementation detail.
We're leaning towards the second option. It looks like about 40 lines of Python, and it's the same approach we used in an earlier MCP Apps proof of concept. But it's undocumented, and we don't want to build on it if it isn't considered supported.
The same question applies to MCP Apps
This isn't specific to A2UI.
middlewares/mcp-apps-middlewareis also TypeScript only. In an earlier proof of concept we hand-ported its client-facing part to Python to get MCP Apps widgets rendering. That worked, but only by copying the payload from the TypeScript source, and it would break silently if either side changed. So the broader question is what the intended way is for a Python agent to provide what themiddlewares/*packages provide.What we'd like to see
ag_ui_a2ui_toolkitorag_ui_strands. It would detecta2ui_operationsin tool results and emitACTIVITY_SNAPSHOT { activityType: "a2ui-surface" }, handleforwardedProps.a2uiActionso action runs reach the model, and ideally cover the building/retrying lifecycle for dynamic mode. That's the fixed-schema and action half ofA2UIMiddleware, and it's the same pattern we used in an earlier MCP Apps experiment. Is this direction something you'd accept? We're happy to contribute it.mcp-apps-middlewaretoo, either as a Python equivalent or as a documented contract a Python producer can rely on.Thanks. The toolkit and adapters are clearly well built, and most of the pieces already exist in Python. What's missing is the last step that turns them into a surface on screen without a TypeScript runtime in the middle.
All reactions