Skip to content

[P1][0.5] Standardize agent progress commentary and one final answer #85

Description

@mightnent

Outcome

Give every LemmaComputer agent one consistent, release-qualified turn lifecycle for user-visible progress, final answers, retries, input requests, failures, cancellation, reconnect, and resumption.

The product contract applies to every enabled agent adapter—currently Claude, Codex, and Hermes—and to future adapters before they can be registered for employee use. Each agent may use a different CLI, provider API, streaming format, tool loop, approval mechanism, and vendor-session model. Those differences must end at the adapter boundary. Control, Web Chat, Activity, and conversation persistence must receive the same normalized semantics regardless of which agent runs the turn.

While any agent works, meaningful user-visible commentary should stream into one expandable Thinking surface rather than being appended to the final answer. When the turn reaches a terminal outcome, that surface collapses and exactly one concise durable answer follows it.

This is not a request to expose hidden chain-of-thought. Thinking is the product name for model-authored commentary that is intentionally safe to show the employee, together with sanitized plans, milestones, tool states, and failure summaries. It remains visually and structurally separate from the terminal answer.

Current evidence

The defect was first reproduced through Hermes during a synthetic artifact-generation request while qualifying #80. No customer content was involved. Hermes emitted intermediate assistant text during its tool loop, the adapter forwarded each delta as ordinary chat text, and Control accumulated those deltas into the same assistant content as the final response. The employee therefore saw repeated promises and progress paragraphs before the actual answer.

Hermes is the original reproduction, not the product boundary. Claude, Codex, and future agents expose different native event streams and execution mechanisms, but all can produce the same class of defect if commentary, retries, tool-loop text, and final output are not normalized explicitly.

The required fix is an agent-neutral lifecycle contract plus adapter-specific translation—not a Hermes-only text filter and not identical internal implementations across agents.

Execution

Scope

1. Define one normalized lifecycle for every agent

  • Define versioned Control-owned semantics for user-visible commentary, sanitized activity/tool state, exactly one final answer, needs_input, failed, and cancelled.
  • Make event type, run identity, message identity, sequence, and terminality explicit. Do not infer them from prose, silence, timing, or provider-specific event names.
  • Emit user-visible interim commentary independently from the terminal answer. If a native event is ambiguous, the adapter must buffer it until it can classify it safely.
  • Allow at most one terminal outcome per logical run. Late, repeated, or replayed provider events must be idempotent.
  • Preserve one logical run and stable identifiers across provider retry, tool retry, reconnect, browser refresh, Control restart, and supported agent-session resumption.
  • Treat needs_input, failed, and cancelled as terminal outcomes with one concise, actionable durable message.
  • Keep raw provider payloads, hidden reasoning, unrestricted tool arguments/results, credentials, and private paths out of chat messages and Activity records.

2. Keep native mechanisms inside each adapter

  • Require every registered adapter to translate its native provider or CLI events into the normalized lifecycle.
  • Keep native parsing, tool-loop behavior, approval handling, retry mechanics, cancellation, and vendor-session resumption adapter-local.
  • Do not hard-code product behavior to Hermes, Claude, Codex, or any other agent name.
  • Add a provider-independent conformance suite that runs against every enabled adapter.
  • Cover commentary, tool activity, final answers, input or approval requests, cancellation, deterministic failure, transient retry, reconnect, and resumed sessions.
  • Prove that Claude, Codex, and Hermes produce the same normalized employee-visible outcome for equivalent scripted event sequences even though their native streams differ.
  • If an adapter has no native user-visible commentary, show only real sanitized lifecycle or tool milestones; do not fabricate model commentary.
  • Keep vendor session bindings separate. Changing agents between turns must not merge proprietary session state or reclassify prior commentary as a new final answer.
  • Reject registration or release qualification for an adapter that cannot prove lifecycle conformance.

3. Stream and coalesce commentary consistently in Web Chat

  • While a turn is active, render one live, expanded Thinking surface in the conversation regardless of the selected agent.
  • Stream distinct model-authored commentary and sanitized milestones into that surface as they occur; do not reduce the experience to a spinner or one static status label.
  • Update the same surface in place, preserve useful order, and coalesce semantically equivalent statements that add no new milestone or outcome.
  • Show the newest meaningful update prominently while allowing the employee to review earlier visible updates.
  • When the run reaches a terminal outcome, collapse the surface to a compact summary such as Thought for 1m 22s; keep it expandable and render exactly one terminal assistant answer beneath it.
  • Keep View activity as the deeper sanitized operational trace owned by [P0][Demo] Add a versioned, sanitized Activity event protocol and replay API #6 and [P0][Demo] Build the streaming Activity, Sources & Actions side panel #16. The Thinking surface is the conversational summary; Activity is the structured plan/progress/tool timeline.
  • Avoid redundant UI rows such as separate low-value Work complete, Turn completed, and repeated final-answer milestones for the same transition.
  • Meet keyboard, screen-reader, reduced-motion, narrow viewport, and long-running-turn requirements.
  • Rate-limit live-region announcements so token-level streaming does not overwhelm assistive technology.

4. Make retries, reconnect, and persistence idempotent

  • Store or derive user-visible commentary separately from canonical final-answer text.
  • Reconstruct the same active or collapsed Thinking surface after browser refresh, reconnect, workspace restart, Control restart, and later conversation reload without duplicating content.
  • Do not create a second progress surface, final answer, artifact card, or durable message when a provider, tool, transport, or client retries.
  • Classify deterministic failures once and do not retry them as transient failures.
  • Bound transient retry and backoff without changing the logical run or restarting visible progress.
  • Preserve completed commentary and the terminal answer when the workspace later stops, through the durable boundary owned by [P1][Post-0.5] Make conversations and generated artifacts durable outside workspace runtimes #80.
  • Define the normalized lifecycle records and identifiers consumed by [P1][Post-0.5] Make conversations and generated artifacts durable outside workspace runtimes #80 without making workspace paths, provider payloads, or vendor session files the long-term conversation authority.

5. Define deterministic terminal UI and data semantics

  • final: collapse Thinking, retain it as expandable history, and show exactly one final answer.
  • needs_input: stop active progress and show one actionable request without also presenting a success answer.
  • failed: stop active progress and show one sanitized failure summary without replaying recovered internal errors as separate terminal messages.
  • cancelled: stop active progress and show one cancellation outcome without later provider events reviving the run.
  • Reconnect or replay after any terminal outcome must reconstruct that same outcome rather than emit another terminal message.

Non-goals

  • Displaying or storing hidden chain-of-thought.
  • Persisting unrestricted provider streams or raw tool results as chat history.
  • Replacing streamed commentary with only a spinner or static Working… label.
  • Making Claude, Codex, Hermes, or future agents use the same native execution or streaming mechanism.
  • Implementing equivalent UI, deduplication, or persistence behavior separately inside each agent.
  • Office document generation, PowerPoint/Word/Excel runtime qualification, visual artifact inspection, or agent-specific Office tooling.
  • Adding packages, browser runtimes, or agent-specific execution capabilities.
  • Replacing the governed routing layer, Activity, or the durable conversation/artifact architecture in [P1][Post-0.5] Make conversations and generated artifacts durable outside workspace runtimes #80.
  • Treating a larger timeout or higher tool-iteration budget as the lifecycle fix.

Definition of success

All-agent protocol behavior

  • Claude, Codex, Hermes, and every other enabled adapter pass the same normalized lifecycle conformance suite.
  • An equivalent multi-tool turn from each adapter produces multiple distinct updates in one Thinking surface followed by exactly one terminal assistant answer.
  • Native provider events never reach Control or Web Chat as ambiguous untyped assistant text.
  • Completed, failed, cancelled, and needs-input outcomes each have deterministic agent-independent protocol, UI, and persistence semantics.
  • Provider, tool, transport, and client retries do not duplicate commentary, the progress surface, terminal answer, artifact references, or durable messages.
  • Adding or upgrading an adapter cannot bypass the conformance gate.

Web Chat behavior

  • The Thinking surface updates in place, preserves useful milestone order, and coalesces near-duplicate statements without hiding genuinely new progress.
  • On a terminal outcome, the surface collapses to a compact duration or summary row, remains expandable, and stays visually separate from the terminal answer.
  • Refresh, reconnect, workspace restart, Control restart, conversation reload, and supported agent handoff reconstruct the same state without duplicated content.
  • The thought surface and Activity view contain only intentionally user-visible commentary and sanitized lifecycle records, never hidden reasoning or raw provider/tool payloads.
  • Keyboard, screen-reader, reduced-motion, narrow viewport, live-region, and long-running-turn behavior pass focused browser coverage.

Regression evidence

  • A Hermes regression fixture proves tool-loop assistant deltas no longer accumulate into the final answer.
  • Adapter-specific fixtures prove Claude, Codex, and Hermes native streams normalize to the same commentary/final/input/failure/cancellation semantics.
  • A provider-independent fixture covers transient retry, deterministic failure, cancellation, reconnect, replay, and resumption and proves each transition is bounded and idempotent.
  • Persistence coverage proves user-visible commentary is separate from the normalized terminal answer and artifact references.
  • Browser coverage switches among enabled agents and verifies live updates, coalescing, refresh during work, automatic collapse, expansion, terminal outcomes, and conversation reload.
  • npm run verify:quick, the smallest relevant Playwright chat flow, and focused adapter conformance tests pass on the exact candidate SHA.

Related issues

Done when

  • The Control protocol documents commentary, activity, final, needs-input, failure, cancellation, retry, reconnect, replay, and resumption semantics independently of any agent vendor.
  • Every enabled agent adapter has passing conformance evidence against that protocol.
  • Web Chat renders one coalesced, accessible Thinking surface and exactly one deterministic terminal answer for every adapter.
  • Conversation and Activity boundaries retain only normalized, tenant-scoped records and reconstruct turns without duplication.
  • Documentation explains how an adapter maps native events to the shared lifecycle and how to diagnose a failed conformance fixture from sanitized evidence.
  • The final handoff records exact commands/results, adapter-matrix results, deployment-profile impact, residual limitations, and the integration-eligible commit SHA.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions