Support client-local UI extensions in daemon-backed interactive chats

[alvinycheung]

Problem

Prime Agent 0.7.2 cannot run custom editors, raw terminal-input handlers, or autocomplete providers from discovered extensions in a normal interactive chat. The active extension runner is in the daemon worker, but these callbacks need the terminal client's live TUI objects.

This is the gap reported in issue #734. The issue was closed during the discussion-first backlog cleanup, not because the behavior was fixed. Release 0.7.2 and current main still contain the no-op bindings.

A minimal reproduction is:

1. Register a session_start handler that calls ctx.ui.setEditorComponent(factory) and ctx.ui.onTerminalInput(handler).
2. Start a normal interactive chat.
3. Read the editor slot with ctx.ui.getEditorComponent().

The context reports hasUI: true, but the getter returns undefined and neither callback runs. dist/modes/daemon/daemon-extension-binding implements these APIs as no-ops. Process-local extensionFactories make the real TUI callbacks available, but deliberately select the reduced in-process path instead of preserving daemon-backed agents and attach/resume.

There is a related complication. Daemon-client startup prepares local runtime services and imports discovered worker extension modules, even though InteractiveMode never binds that local runner or emits its lifecycle events. The worker imports them again and owns the active runner. A fix should stop importing worker entry points in the client, not turn this existing double execution into a supported contract.

Suggested direction

Add an explicit client-local UI entry point. Keep the current worker entry point for tools, providers, agent events, and session-owned behavior.

One possible package contract is:

{
  "pi": {
    "extensions": ["./dist/worker-extension.js"],
    "uiExtensions": ["./dist/ui-extension.js"]
  }
}

A package could declare either entry point or both. Prime would not infer a UI entry point by executing every worker extension again.

For a first version, I would limit the client API to:

• custom editor and custom TUI component factories;
• raw terminal-input handlers;
• autocomplete providers and shortcuts;
• optional UI-local commands.

Serializable notifications, status text, widget lines, titles, and dialogs already use daemon UI requests. They do not need a second path.

A UI entry point needs a smaller API than a worker extension. It should not register tools, providers, prompt transforms, or agent-event hooks. A separate UIExtensionAPI type would make the boundary clear, but it would not be a sandbox: local extension JavaScript still has the client's Node, filesystem, process, and network privileges.

UI-local commands must intercept input before AgentConnection.prompt(). The daemon command list is a catalog; its handlers execute in the worker after submission. The client therefore needs a small, named AgentConnection facade rather than a forwarded ExtensionCommandContext, whose SessionManager callbacks are worker-only. The first version could reject UI/worker command name collisions rather than add another priority rule.

Discovery and trust

I see two viable same-host options:

1. The client resolves uiExtensions from local user, project, and package configuration.
2. The daemon describes a separate uiExtensions resource collection, and the client loads only local files allowed by its trust policy.

The existing AgentConnectionResourceSnapshot.extensions list contains successfully loaded worker extensions, so it cannot represent a UI-only package. Its artifact references are metadata; they provide neither bytes nor integrity digests.

Remote or gateway-selected UI code should be out of scope initially. Automatically executing code chosen by a remote peer creates a remote-code-execution boundary. Remote support would need explicit client trust, authenticated artifact transfer, integrity metadata, and visible provenance. Snapshot membership alone is not authorization.

Prime should not transmit keystrokes or draft text implicitly. A trusted local extension can still make explicit connection calls, as other extension code can.

Lifecycle

On attach or reload, the client would bind the real TUI context and start the active local UI entries. On detach, session replacement, resource change, or terminal shutdown, Prime should await bounded cleanup, dispose host-tracked registrations, restore the stock editor, and drop the runner reference. The extension remains responsible for global timers and other effects that it created outside the host API.

Tree navigation should not reload UI code unless the resource set or working directory changes. Daemon-owned and client-owned UI state also need separate ownership, because current status/widget keys do not carry an extension-source owner.

Costs

This adds package and settings discovery. A daemon-described option also adds protocol surface. Authors may need split worker/UI packages or shared initialization code. Client and worker versions can disagree about capabilities. Prime must also stop client startup from importing worker entry points that only the daemon should execute.

These costs seem preferable to serializing executable callbacks, executing every extension twice, or relying on a version-coupled terminal proxy.

Initial proof

A focused implementation should prove that:

1. a daemon-backed chat installs a client-local editor, terminal handler, and autocomplete provider;
2. agents view, attach/resume, messages, schedules, and heartbeats remain daemon-backed;
3. reload, detach, session replacement, and exit restore the stock editor and dispose tracked registrations once;
4. headless clients do not load UI entries, and worker calls to client-only APIs produce a clear capability diagnostic.

Open questions

1. Should same-host clients discover UI entries locally, or should the daemon describe a separate UI resource collection?
2. Which session actions belong in the curated UI-command facade?
3. Should UI/worker command collisions fail, or use a namespace?
4. What capability signal distinguishes serializable daemon UI requests from process-local TUI callbacks?

Would maintainers accept this separate-entry-point direction for a same-host first version? If so, the next step could be a maintainer-scoped issue that settles these questions before implementation.

[jasoncarreira]

Supporting this from a concrete use case, and adding one measurement that may help scope it.

We ship the same build-orchestration tool as an opencode plugin and as a prime extension. In opencode the plugin
renders a sidebar component showing live run state — the current gate, each parallel slice and its attempt count,
the lock owner, the last verification result — updating every few seconds across hour-long autonomous runs. Our
prime extension ships no UI at all, and this discussion is half the reason.

The other half is placement, which I have opened separately as
#1489: even once client-local UI callbacks
execute, WidgetPlacement is "aboveEditor" | "belowEditor", so a widget can only render inline around the
editor. A panel that persists beside the conversation needs both this proposal (the callbacks run) and a side
placement (somewhere for them to live). They look independently implementable, which is why I filed the placement
part separately rather than piling it on here.

One note in support of your uiExtensions entry point specifically: a separate client-local entry is what makes
this safe for us. Our worker-side extension holds the run's lock identity and drives state transitions; we would
not want UI code in that module, and we would not want the UI to be able to reach the tools. The split you propose
draws that line in the right place — UI gets editors, components, terminal handlers and autocomplete, and
explicitly not tools, providers or agent-event hooks.

Verified on 0.7.2 (protocol-7-schema-16, build 83a0f9f-dirty), consistent with what you reported.

[alvinycheung]

In short, i'm trying to build voice-mode into this harness.

[Explorer1092]

Same gap, one slot further: extension tool renderers (renderCall / renderResult) also never run in daemon-backed interactive chats — and they fit squarely in the uiExtensions scope proposed here.

What we hit. Our extension (a connector that registers six custom tools) implements renderCall/renderResult per docs/extensions.md and docs/tui.md. In a normal interactive chat (which is daemon-backed by default, with no user-facing opt-out), every tool row falls back to raw-JSON output. The same build renders correctly when the session runs in-process (SDK extensionFactories), so the renderers themselves are fine.

Why it happens (traced through 0.8.0):

1. ToolExecutionComponent resolves renderers from the tool definition, which InteractiveMode.loadToolDefinition() assembles from two sources: agentConnection.getToolDefinition(name) and localSessionHost?.getToolRendererDefinition(name).
2. In daemon-attach mode the connection is a DaemonAgentConnection; getToolDefinition is a JSON-RPC (get_tool_definition), so only serializable fields (name, label, description, parameters) survive — the renderer functions die at the wire boundary.
3. The local fallback is unavailable because the attach path constructs InteractiveMode with bindLocalSessionExtensions: false and no localSessionHost (main.js), so getToolRendererDefinition is never consulted.

Net effect: for extension tools the panel header (label · status) works, but the body is always the default raw-JSON fallback. Built-in tools are unaffected because their renderers ship inside the TUI bundle (builtInToolDefinition).

Ask: include tool renderers in the client-local uiExtensions contract — they are exactly the "pure UI, no session side effects" category this proposal targets. A renderer needs only (args|result, options, theme, context) in and a Component out; nothing it does belongs in the worker. Happy to contribute a PR along these lines if the direction is accepted.

(Precedent for the split execution model: #1057 / PR #1071 kept shortcut handler execution in the daemon while proxying the UI trigger — tool renderers are the mirror image, UI-only code that should live entirely client-side.)