Skip to content

fix(chat): synthesize human-readable label for A2UI action user bubbles#464

Merged
blove merged 2 commits into
mainfrom
claude/a2ui-action-bubble-synth
May 19, 2026
Merged

fix(chat): synthesize human-readable label for A2UI action user bubbles#464
blove merged 2 commits into
mainfrom
claude/a2ui-action-bubble-synth

Conversation

@blove
Copy link
Copy Markdown
Contributor

@blove blove commented May 19, 2026

Summary

Today, when a user interacts with a rendered A2UI surface (clicks Submit on a form, Select on a flight, Modify search, etc.), the protocol's A2uiActionMessage flows through agent.submit() as a HumanMessage whose content is the JSON-serialized protocol payload. The chat-message-list renders that JSON as a user bubble — the end-user sees a ~300-char dump like:

`{"version":"v1","action":{"name":"bookingSubmit","surfaceId":"booking","sourceComponentId":"submit","timestamp":"...","context":{"formId":{"literalString":"booking"},"origin":{"literalString":""}...}}}`

…as if they typed the protocol themselves.

Research basis

Per the A2UI v0.9 spec, action messages flow on the client → agent return channel and are framed as typed events resembling tool call results — NOT user utterances. Google Cloud's "A2UI in Practice" article and the Stream Chat reference integration both warn against modeling actions as chat-history user turns. No reference client surfaces the raw payload as a bubble; Stream Chat synthesizes a human-readable string and tucks the JSON into extraData; CopilotKit transforms into a natural-language query string. The protocol spec itself is silent on UI presentation.

Fix

New helper a2uiActionLabel(content) in libs/chat/src/lib/a2ui/action-label.ts:

  • Detects v1 A2UI action messages by parsing the JSON content
  • Returns a short human-readable label for known action names:
    • bookingSubmit'Search flights'
    • flightSelect'Selected flight UA123' (or 'Selected flight' if no flightId)
    • modifySearch'Modify search'
  • Falls back to camelCase humanization for unknown action names (e.g. addToCart'Add to cart')
  • Returns null for any non-action content (regular prose, markdown, malformed JSON) so typed prompts pass through unchanged

The chat composition's human-message template now routes through a new humanContent(message) method that prefers the synthesized label and falls back to the raw content. No backend or message-stream change — the action still flows through state and reaches the agent; only the visible bubble is rewritten.

Files

  • libs/chat/src/lib/a2ui/action-label.ts — new helper with full inline citation of the research sources
  • libs/chat/src/lib/compositions/chat/chat.component.ts — wire the helper into the human-message template via humanContent()

Test plan

  • 11/11 unit cases pass: known actions (bookingSubmit/flightSelect/modifySearch), unknown action humanization, malformed JSON, plain prose, empty string, wrong version (v2), non-action object
  • npx nx run chat:build — green
  • npx nx run chat:test — green
  • npx nx run cockpit-chat-a2ui-angular:build — green
  • Chrome MCP smoke against local c-a2ui (5511 + 4511): clicking 'Search flights' on the booking form now produces a 'Search flights' user bubble instead of the JSON dump
  • CI

Out of scope (worth a follow-up)

  • The synthesized label could optionally be styled differently (smaller / dimmer) to visually distinguish from typed user messages, similar to Stream Chat's pattern. This PR uses the standard user-bubble style for now.
  • More action-name → label mappings can be added in KNOWN_LABELS over time as new surfaces are introduced.
  • Attaching the action visually to the originating surface (Option C in the research) would be the most semantically faithful long-term fix — but requires persistent surfaces and a larger surface-store rework.

🤖 Generated with Claude Code

Today, when a user interacts with a rendered A2UI surface (clicks
Submit on a booking form, clicks Select on a flight card, etc.), the
A2uiActionMessage flows through agent.submit() as a HumanMessage whose
content is the JSON-serialized protocol payload. The chat-message-list
renders that JSON as a user bubble, leaking ~300 chars of
{"version":"v1","action":{"name":"bookingSubmit","surfaceId":"booking",
...}} into the visible transcript as if the end-user typed it.

Per the A2UI v0.9 spec, action messages flow on the client → agent
return channel and are framed as typed events resembling tool call
results, NOT user utterances. Google's "A2UI in Practice" article and
the Stream Chat reference both warn against modeling actions as
chat-history user turns. No reference client renders the raw payload
as a user bubble.

Fix: new `a2uiActionLabel(content)` helper detects v1 action messages
and returns a short human-readable label ("Search flights" /
"Selected flight UA123" / "Modify search" for known action names;
"Camel case" humanization for unknown ones). Returns null for any
non-action content so regular typed prompts pass through unchanged.

The chat composition's `<ng-template chatMessageTemplate="human">` now
routes through `humanContent(message)` which prefers the synthesized
label and falls back to the raw content. No backend or message-stream
change — the action still flows through state and back to the agent;
only the visible bubble is rewritten.

Sources cited inline in libs/chat/src/lib/a2ui/action-label.ts:
- https://a2ui.org/specification/v0.9-a2ui/
- "A2UI in Practice" (Google Cloud Medium)
- Stream Chat A2UI integration (synthesized text + metadata pattern)

Verified locally via chrome MCP on c-a2ui: clicking 'Search flights'
on the booking form now produces a 'Search flights' user bubble
instead of the JSON dump. 11/11 unit cases pass (known actions,
unknown actions, malformed JSON, plain prose, etc.).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 19, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
threadplane Ready Ready Preview, Comment May 19, 2026 4:38pm

Request Review

@blove blove merged commit 27969df into main May 19, 2026
1 of 2 checks passed
@blove blove deleted the claude/a2ui-action-bubble-synth branch May 19, 2026 16:36
blove added a commit that referenced this pull request May 19, 2026
…N_LABELS (#474)

* docs: spec LLM-generated labels (thread titles + action labels)

Two related fixes in one design pass:
- Thread titles via inline per-cap node + LangGraph SDK metadata write
  (Pattern D from the design discussion — fully inline, no shared
  helper, matches per-cap pedagogical purpose).
- Action message labels derived from the authored Button's child Text
  at emit time, killing the hardcoded KNOWN_LABELS map in
  libs/chat/src/lib/a2ui/action-label.ts (PR #464).

Both backed by deep cross-library research (Open Canvas, Vercel,
assistant-ui, CopilotKit, ChatGPT/Claude reference UX) showing
universal consensus: titles are LLM-generated post-first-turn from
thread metadata; action labels come from the authored UI element,
never a centralized map in the rendering primitive.

Scope: c-a2ui first (proves the pattern), document for other caps.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: plan LLM-generated labels implementation

10-task plan covering:
- libs/a2ui type extension (A2uiActionMessage.action.label?: string)
- libs/chat/a2ui/build-action-message.ts label derivation + 4 new spec cases
- libs/chat/a2ui/action-label.ts: drop KNOWN_LABELS, prefer authored label,
  fall back to camelCase humanizer; 11-case new spec
- cockpit/chat/a2ui/python/src/graph.py: inline generate_title node
  (Pattern D from spec), wired between terminal nodes and END

Implementer = Tasks 1-9 (code, tests, smoke). Orchestrator = Task 10
(push, PR, CI watch, merge).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(a2ui): add optional A2uiActionMessage.action.label field

* feat(chat): derive action.label from source Button's Text child

* test(chat): add cases for action.label derivation

* refactor(chat): drop KNOWN_LABELS; derive action label from authored UI

* feat(c-a2ui): inline generate_title node writes LangGraph thread metadata

* fix(chat): action.label derivation also accepts raw-string Text.text shorthand

* chore(docs): regenerate api docs

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant