[Proposal] Answer Artifacts: Make final agent replies readable HTML instead of plain text

[Thinker-Joe]

Summary

Long Markdown text from agents creates high cognitive load for users.
Reading long text is now the primary bottleneck in human-agent workflows.

We propose a first-class feature: Answer Artifacts.
When an agent completes a task, the platform renders its final response as a clean, visual HTML card (tables, sequence diagrams, key metrics, and callouts).
Users see structured information immediately instead of reading walls of plain text.

This proposal builds directly on Multica's existing render engine. It solves four missing requirements:

1. Low token cost: The LLM writes only a concise Markdown draft (~600 tokens). A client/server renderer generates the HTML/SVG layout.
2. Strict scope: This feature applies only to final agent outputs, not to human comments or issue descriptions.
3. Closed feedback loop: Users can select options or submit feedback directly on the artifact card to start the next turn.
4. Safe sandbox: The system renders markup in a sandboxed iframe with strict Content Security Policy (CSP).

---

Why This Matters

Andrej Karpathy highlighted the core problem (reference link):

As LLMs do more work, human review becomes the hard part. A diagram or a web page is much easier to understand than a wall of text.

His "understanding ladder" shows the ideal progression of output formats:
Plain text → Controlled English (ASD-STE100) → Diagrams → Visual HTML pages → Explainer videos.

Karpathy describes these as "custom, discardable software artifacts."
For short answers, plain text is sufficient. But for multi-step task summaries, system architectures, and data comparisons, visual artifacts save massive human review time.

This matches the core vision in Multica's VISION.md:

"The team does not review status theatre. It reviews the work itself: the plan, the document, the implementation, the diff, the preview..."
"Humans spend less time carrying information between tools and chasing handoffs."

---

Current State in Multica

Multica already contains strong visualization building blocks:

• [Feature] In-app HTML visualization/rendering #2428 (Closed as Completed): Added in-app HTML visualization support.
• MUL-7649: issue deliverables and dynamic blocks — sidebar section, overview grid, viewer info panel, html/mermaid block frame #8779 & Release v0.6.0: Added dynamic blocks in packages/views/editor/dynamic-block.tsx. When an agent outputs html or mermaid code blocks, the frontend renders them inline.
• MUL-7650: attachment viewer — CSV tables, JSON/YAML tree, numbered code, HTML viewports, every kind in a new tab #8868 & MUL-7744: feat(views): address bar for HTML attachment previews (MUL-7737) #8873: Added HTML viewport previews, address bars, and multi-format viewers in the attachment viewer.
• Chat runtime: no file/artifact creation support in chat sessions #5119 / feat(views): implement frontend UI artifact viewer and markdown parsing [MYL-41] #5122 / MYL-40: Add chat inline artifact wrapper instructions #5129 (Open track): Community requests for <multica-artifact> tag parsing and chat artifact cards.
• [Bug]: No in-page text search in HTML maximized preview #5259 (Open): Users report that reading and searching large HTML agent outputs is difficult, proving that agents already generate complex deliverables.

Why PR #7188 was Reverted (#7584)

PR #7188 introduced live editable HTML previews in chat, but maintainers reverted it in #7584.
The revert reason was product scope, not code quality or security:

1. Unclosed loop: Edits were copy-only. Users could not save them back to the message or return them to the agent.
2. Uncontrolled scope: The edit button appeared on all HTML blocks across human messages, issue descriptions, and comments.

This proposal directly resolves both concerns:

• Scope is constrained: Applied strictly to agent final deliverables.
• Loop is closed: Artifact state and user form clicks pass directly into the next agent turn.

---

The Economics: Draft vs. Raw HTML

Direct HTML generation by LLMs is inefficient:

• A benchmark by answer-me-with-html shows direct HTML averages 4,893 tokens per answer. 47% of tokens are SVG coordinates, 15% are CSS classes, and 17% are repetitive HTML tags.
• When the LLM outputs a structured Markdown draft and a local renderer builds the page, the model uses only 612 tokens (an 8x reduction, 2.6x faster generation).

With this architecture, rich visual answers remain affordable for daily use.

---

Proposed Solution

1. User Interface

• The final agent comment renders an Artifact Card above the Markdown text.
• The card displays clean visual components: key-value metrics, flowcharts, status tables, and callouts.
• Users can click controls on the card: Expand Fullscreen, Open in New Tab, Copy Source, and Export HTML/PDF.
• Mobile and external channels (Telegram, Lark, DingTalk) receive clean Markdown text plus a signed link to the artifact page.

2. Draft Protocol

The agent returns structured draft blocks instead of raw CSS/HTML coordinates:

:::artifact{type="summary" title="Database Migration Status"}
- status: completed
- duration: 4m 12s
- affected_rows: 14,280

[flow]
Backup DB -> Run Migrations (566-572) -> Health Check -> Active
:::

The client renderer converts this draft into an interactive SVG/HTML view in ~50 ms.

3. Execution Policy

• Workspace setting: Off | On Request | Always On (for complex tasks).
• Skip artifacts for simple chat, one-line clarifications, or raw CLI logs.

---

Security Model

Agent outputs are untrusted inputs. We maintain the strict security posture established in #7188 and #5272:

• Isolated iframe: Render inside <iframe sandbox="allow-scripts"> without allow-same-origin.
• Strict CSP: Enforce default-src 'none'. Inline styles only. Prohibit external CDNs, external web fonts, and network calls.
• Credential isolation: Artifact frames cannot access local cookies, tokens, or workspace storage.
• No unauthorized script execution: Remove <script> tags by default. Allow interactive UI components only via verified template handlers.

---

Questions for Maintainers

1. Renderer Location: Should the draft renderer live in the frontend web package (@multica/views), or as an ACP agent tool/skill?
2. Artifact Persistence: Should answer artifacts be stored as formal issue deliverables with version history, or as transient message viewports?
3. Integration with Existing PRs: Should this extend the pending <multica-artifact> format (feat(views): implement frontend UI artifact viewer and markdown parsing [MYL-41] #5122 / MYL-40: Add chat inline artifact wrapper instructions #5129), or run as a standalone dynamic block?

---

中文摘要 (Chinese Summary)

当前 Agent 执行完毕后，最终回复通常是长篇 Markdown 纯文字，人类用户的阅读与理解成本极高，成为人机协同的核心瓶颈。

Andrej Karpathy 提出的"理解阶梯"指出：纯文字 → 受控语言 (ASD-STE100) → 图表 → 可视化 HTML 页面 → 视频。针对复杂工程总结，可视化页面能显著降低认知负荷。

Multica 当前已有强大的渲染底座（v0.6.0 已支持动态代码块内联渲染），但缺少专门面向"Agent 最终答复"的轻量化机制。前期 PR #7188 被回退（#7584）的核心原因是未闭环（无法交回 Agent）以及范围失控（波及所有人类消息）。

本提案基于 ASD-STE100 规范设计，聚焦解决核心痛点：

1. 范围收敛：严格限定于 Agent 最终任务交付回复。
2. 成本骤降（8倍节省）：Agent 仅输出轻量 Markdown 结构草稿（约 600 tokens），排版、CSS 与 SVG 坐标由渲染器本地生成，避免大模型直接手写数千 tokens 的原生 HTML。
3. 闭环交互：卡片支持操作项与选项选择，结果可直接作为下一轮输入回传给 Agent。
4. 安全隔离：沿用经过验证的 Sandboxed iframe + 严格 CSP，禁止外联与凭据读取。

[Thinker-Joe]

讲真，现在真的有点认知过载，同时并行跑N个任务，卡住了需要快速理解问题在那，然后再决策让Agent推进，需要一些新时代的工作来帮助解决问题