fix(chat): synthesize human-readable label for A2UI action user bubbles#464
Merged
Conversation
Today, when a user interacts with a rendered A2UI surface (clicks
Submit on a booking form, clicks Select on a flight card, etc.), the
A2uiActionMessage flows through agent.submit() as a HumanMessage whose
content is the JSON-serialized protocol payload. The chat-message-list
renders that JSON as a user bubble, leaking ~300 chars of
{"version":"v1","action":{"name":"bookingSubmit","surfaceId":"booking",
...}} into the visible transcript as if the end-user typed it.
Per the A2UI v0.9 spec, action messages flow on the client → agent
return channel and are framed as typed events resembling tool call
results, NOT user utterances. Google's "A2UI in Practice" article and
the Stream Chat reference both warn against modeling actions as
chat-history user turns. No reference client renders the raw payload
as a user bubble.
Fix: new `a2uiActionLabel(content)` helper detects v1 action messages
and returns a short human-readable label ("Search flights" /
"Selected flight UA123" / "Modify search" for known action names;
"Camel case" humanization for unknown ones). Returns null for any
non-action content so regular typed prompts pass through unchanged.
The chat composition's `<ng-template chatMessageTemplate="human">` now
routes through `humanContent(message)` which prefers the synthesized
label and falls back to the raw content. No backend or message-stream
change — the action still flows through state and back to the agent;
only the visible bubble is rewritten.
Sources cited inline in libs/chat/src/lib/a2ui/action-label.ts:
- https://a2ui.org/specification/v0.9-a2ui/
- "A2UI in Practice" (Google Cloud Medium)
- Stream Chat A2UI integration (synthesized text + metadata pattern)
Verified locally via chrome MCP on c-a2ui: clicking 'Search flights'
on the booking form now produces a 'Search flights' user bubble
instead of the JSON dump. 11/11 unit cases pass (known actions,
unknown actions, malformed JSON, plain prose, etc.).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
6 tasks
blove
added a commit
that referenced
this pull request
May 19, 2026
…N_LABELS (#474) * docs: spec LLM-generated labels (thread titles + action labels) Two related fixes in one design pass: - Thread titles via inline per-cap node + LangGraph SDK metadata write (Pattern D from the design discussion — fully inline, no shared helper, matches per-cap pedagogical purpose). - Action message labels derived from the authored Button's child Text at emit time, killing the hardcoded KNOWN_LABELS map in libs/chat/src/lib/a2ui/action-label.ts (PR #464). Both backed by deep cross-library research (Open Canvas, Vercel, assistant-ui, CopilotKit, ChatGPT/Claude reference UX) showing universal consensus: titles are LLM-generated post-first-turn from thread metadata; action labels come from the authored UI element, never a centralized map in the rendering primitive. Scope: c-a2ui first (proves the pattern), document for other caps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: plan LLM-generated labels implementation 10-task plan covering: - libs/a2ui type extension (A2uiActionMessage.action.label?: string) - libs/chat/a2ui/build-action-message.ts label derivation + 4 new spec cases - libs/chat/a2ui/action-label.ts: drop KNOWN_LABELS, prefer authored label, fall back to camelCase humanizer; 11-case new spec - cockpit/chat/a2ui/python/src/graph.py: inline generate_title node (Pattern D from spec), wired between terminal nodes and END Implementer = Tasks 1-9 (code, tests, smoke). Orchestrator = Task 10 (push, PR, CI watch, merge). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(a2ui): add optional A2uiActionMessage.action.label field * feat(chat): derive action.label from source Button's Text child * test(chat): add cases for action.label derivation * refactor(chat): drop KNOWN_LABELS; derive action label from authored UI * feat(c-a2ui): inline generate_title node writes LangGraph thread metadata * fix(chat): action.label derivation also accepts raw-string Text.text shorthand * chore(docs): regenerate api docs --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Today, when a user interacts with a rendered A2UI surface (clicks Submit on a form, Select on a flight, Modify search, etc.), the protocol's
A2uiActionMessageflows throughagent.submit()as a HumanMessage whose content is the JSON-serialized protocol payload. The chat-message-list renders that JSON as a user bubble — the end-user sees a ~300-char dump like:…as if they typed the protocol themselves.
Research basis
Per the A2UI v0.9 spec, action messages flow on the client → agent return channel and are framed as typed events resembling tool call results — NOT user utterances. Google Cloud's "A2UI in Practice" article and the Stream Chat reference integration both warn against modeling actions as chat-history user turns. No reference client surfaces the raw payload as a bubble; Stream Chat synthesizes a human-readable string and tucks the JSON into
extraData; CopilotKit transforms into a natural-language query string. The protocol spec itself is silent on UI presentation.Fix
New helper
a2uiActionLabel(content)inlibs/chat/src/lib/a2ui/action-label.ts:bookingSubmit→'Search flights'flightSelect→'Selected flight UA123'(or'Selected flight'if no flightId)modifySearch→'Modify search'addToCart→'Add to cart')nullfor any non-action content (regular prose, markdown, malformed JSON) so typed prompts pass through unchangedThe chat composition's human-message template now routes through a new
humanContent(message)method that prefers the synthesized label and falls back to the raw content. No backend or message-stream change — the action still flows through state and reaches the agent; only the visible bubble is rewritten.Files
libs/chat/src/lib/a2ui/action-label.ts— new helper with full inline citation of the research sourceslibs/chat/src/lib/compositions/chat/chat.component.ts— wire the helper into the human-message template viahumanContent()Test plan
npx nx run chat:build— greennpx nx run chat:test— greennpx nx run cockpit-chat-a2ui-angular:build— green'Search flights'user bubble instead of the JSON dumpOut of scope (worth a follow-up)
KNOWN_LABELSover time as new surfaces are introduced.🤖 Generated with Claude Code