Skip to content

v0.80.150

Choose a tag to compare

@github-actions github-actions released this 17 Oct 15:25
· 2728 commits to main since this release

0.80.150 makes the fullscreen TUI the default, brings hands-free voice calls and a floating HUD window to the desktop app, and adds an experimental computer tool for desktop control. The engine gains decision-model support for fast, approval-free judgments and a bounded task graph, alongside new providers (Sign in with ChatGPT, vLLM, SGLang) and Composio integrations over MCP. Several boot-time races, HUD window issues, and a TUI crash are fixed.

Highlights

  • Fullscreen TUI by default: the terminal UI now runs in the alternate screen with a fixed editor/status dock and a scrolling, searchable transcript; --tui-mode <regular|fullscreen> overrides per invocation and /settings switches live.
  • Voice call mode (desktop): a hands-free, voice-operated loop around the active session — RMS-gated turn endpointing, sentence-split TTS playback, half-duplex turn-taking with opt-in barge-in — now housed in a right-rail call panel instead of a full-screen overlay.
  • HUD mode (desktop): \hud opens a transparent, frameless, always-on-top floating chat window showing the real transcript and composer over whatever you're working on; it shares the engine connection and event pipeline rather than spawning a second engine.
  • Experimental computer tool (opt-in via the computerUse setting): desktop control on macOS/Windows/Linux through the cua-driver binary, permission-gated, with hard blocks on destructive shortcuts and payloads and fail-closed recovery.
  • Decision-model support: a read-only, approval-free decide tool routes typed judgments through a keyless local server (Ollama, LM Studio, llama.cpp, Unsloth) or OpenRouter, and the auto-mode permission classifier prefers it when reachable.
  • New providers and integrations: Sign in with ChatGPT (/login openai), vLLM and SGLang keyless local servers, Composio apps via the hosted Connect MCP server, and chrome-devtools MCP pre-configured on fresh installs.

Details

Desktop

  • Browser tool permission modes in the tab bar: Agent: ask (every new host asks once, remembered answers pass) and Agent: auto (ordinary sites run without asking; the sensitive list — mailbox, sign-in, payments — keeps asking in every mode). Persisted as browser.agentPermissionMode, validated on write, surfaced via the new browser_get/set_permission_mode command pair.
  • Browser panel auto-reveal: when the assistant's turn invokes a browser_* tool and the right rail isn't showing the browser panel, the rail slides to the browser tab automatically (deduped per tool-call id so a streaming tool's updates don't re-pop it).
  • Esc stops an in-flight run from the main window or the HUD via the shared abort path (haptic, user-cancel flag, engine abort), layered through the new escape-layer system (src/lib/escape-layers.ts, ported from hermes desktop): a single Esc dismisses exactly the top-most urgent surface — pending approval (deny) > command palette > find bar > model picker/command center > tab context menu > titlebar rename > engine stop — with each layer registering while open and lower handlers bailing on defaultPrevented/priority.
  • HUD architecture: the engine's rpc://event stream is emitted to every webview window and the reducer moved out of App.tsx into src/lib/agent-event-pipeline.ts, so the HUD paints state and shares the connection; capabilities live in src-tauri/capabilities/hud.json, and a titlebar toggle (picture-in-picture icon) sits beside the /hud slash command.
  • Voice settings model/voice pickers: the STT and TTS model fields and the TTS voice field are dropdowns that load catalogs from {base}/models and {base}/voices on focus or refresh (fetched through the new voice_fetch_json Rust CORS-bridge command); OpenAI {data:[{id}]}, plain arrays, {voices:[…]}, and id/name/voice shapes are accepted, with the manual text-input fallback for endpoints without a catalog or when Custom… is picked.

Voice calls

  • The call moved out of the full-screen click-through overlay into the right sidebar's own tab (phone icon; auto-opens when a call starts, pulsing dot on the rail when hidden). The panel carries the call card — mic-level orb, phase caption, tool chip, live dictation, sentence progress, mute/skip/hang-up — plus a capped call log of the user's turns and the agent's replies as they send/play. Esc still interrupts and hangs up; approvals and the composer are no longer covered by a scrim.
  • Call mechanics: a plain RMS gate on the mic stream endpoints each utterance (speech confirmed after ~250 ms, sends after configurable silence — default 1200 ms — or a 15 s cap, no ML VAD); mid-run utterances stack as follow-ups via the session's prompt/queue path; the reply plays through a sentence-split TTS queue (two sentences in flight, ordered playback, per-sentence skip, spoken-char limit). Turn-taking defaults to half-duplex (mic parks while the reply speaks; Space/Esc/button tap interrupts; TTS-adjacent audio is always discarded as possible echo) with opt-in open-mic barge-in for headphone users; approval prompts stay clickable.
  • While a call is live, the desktop applies a runtime system-prompt addendum (new set_system_prompt_addendum RPC — appended to the base/extension prompt on every turn while set, cleared when the call ends, never persisted) so spoken replies read like a natural phone call: short spoken-first answers, no markdown/lists/code fences, descriptions instead of pasted output, one clarifying question instead of guessing.
  • Status sounds reuse the synthesized completion-sound bank (Settings → Look & feel volume applies): a bundled thinking-sound-voice-mode.mp3 plays as the thinking cue (synthesized ping fallback), tool calls tick with the tri-tone, connect/hang-up use their own presets; the turn-end chime is suppressed during a call since the spoken reply is the end-of-turn signal.
  • Call-mode tuning in Settings → Voice → Call mode: barge-in, status sounds, silence-before-send (300–5000 ms), max utterance (3–120 s), and spoken reply char limit.

Engine

  • Decision models (System One wire): the decide tool routes typed judgments ({state, 1-16 named questions: choice/noul/score}) through a decision model. Resolution order: explicit decide arguments or settings.decisions {provider, model}, then the first reachable keyless local server (Ollama, LM Studio, llama.cpp, Unsloth — zero-config after e.g. ollama pull clef-flash), then OpenRouter's JeV beta when OPENROUTER_API_KEY is set, with an actionable error naming every option. Results render as a probability card (chosen option, bar, confidence) and ship raw answers JSON; the desktop/rpc surface the same card through the generic tool-result flow. The auto-mode permission classifier prefers a decision model over the LLM classifier when one is reachable: the gate is a {verdict: choice[allow|block], risk: score} question pair scored on the tool-call proposal, mapped onto the same verdict/circuit-breaker semantics (failures degrade to the LLM classifier, blocks degrade to a prompt), and the block reason carries the engine and probability (Auto-mode classifier (clef-flash p=0.94): ...). Docs: docs/decision-models.md; decision models deliberately stay out of the chat /model list.
  • Bounded task graphs: the graph is now a work frontier instead of ever-growing history. Completed tasks get a completedAt stamp; past the 15 most recent, completed tasks auto-archive to archive.jsonl (read-only history — ids stay resolvable via task_get, task_list gains include_archived: true, archived tasks refuse task_update mutations except explicit deletion, and delete cascades reach archived rows). Tool-result snapshots carry archivedCount so the /tasks panel and the desktop Task panel show the open frontier first with a "+N archived" tail. Stale per-session task-list dirs are removed once per process at startup after 30 days untouched — settings key taskRetentionDays (0 disables, mirrored in the desktop settings). Deleting an archived task no longer leaves dangling references in live siblings.
  • RPC engines announce themselves: the first stdout frame in --mode rpc is an engine_info event carrying the engine version and a protocol contract number (RPC_CONTRACT in rpc-types.ts, currently 1), so embedded clients can detect engine/client skew early instead of failing cryptically downstream; clients predating the event ignore it.
  • Engine binary builds are hardened against the upstream Bun non-AVX2 regression (oven-sh/bun#30613): x64 engine targets are compiled with bun 1.3.8 — the last version that works on CPUs/VMs without SSE4.2/POPCNT (bun >= 1.3.9's WebKit -march=nehalem change makes compiled engines crash with 0xC0000005 at startup) — while arm64 targets (including the windows-arm64 cross-compile, which exists only since bun 1.3.10) build with bun 1.3.13. CI supplies both (BUN_BASELINE_BIN), and build-binaries.sh enforces the per-ISA policy with instructions.

Computer use

  • The experimental computer tool (default off; the A_CODER_CLI_COMPUTER_USE=1 env still overrides) drives desktop control on macOS/Windows/Linux through the external cua-driver binary over MCP-on-stdio. One tool with an action discriminator: capture (som = screenshot with numbered element overlays + AX element list, vision = plain screenshot, ax = tree only), click/double_click/right_click/middle_click, drag, scroll, type, key, set_value, wait, list_apps, list_windows, focus_app. Element-index clicking with snapshot element_token staleness detection; background-first input delivery (no focus steal) with delivery_mode: "foreground" gated on the driver's live schema (refused, never silently downgraded); sticky-target guard returns input_target_mismatch instead of typing into the wrong window; every result carries a verdict (done / verify fresh state / escalate). Destructive system shortcuts (logout/lock/empty-trash, alias- and hyphen-proof) and dangerous type payloads (pipe-to-shell, sudo rm -rf, fork bombs) are hard-blocked before approval. Permission-aware: read actions (capture/list/wait) are free; input actions weigh like mutating tools in every mode (blocked in read-only, prompt in ask, classifier-checked in auto). Fail-closed recovery: a timed-out driver call's outcome is unknown — never replayed, transport rebuilt before the next call. Driver resolution: A_CODER_CUA_DRIVER_CMD override, then PATH, then ~/.a-coder/cli/agent/bin/ (cua-driver.exe on Windows; where instead of which on Windows); child env strips provider API keys and disables cua-driver telemetry by default. The TUI settings screen gates enabling behind a disclaimer submenu with an explicit "I understand — enable" choice; the toggle takes effect when a session rebuilds its tool runtime.
  • Actionable install guidance: when the cua-driver binary is missing, failed computer calls (including action: "doctor") return a backend_unavailable payload with an install object — platform/arch-matched release asset (tag cua-driver-rs-v0.33.3), direct URL, the agent bin dir, and copy-pasteable install commands. New docs/computer-use.md documents enabling, backend resolution, install commands, doctor diagnostics, and the security posture.

TUI

  • Fullscreen mode configuration: tuiMode, fullscreenExitOutput ("transcript" reprints the transcript on exit, "resume-hint" prints only the resume hint), fullscreenScrollbar, fullscreenCopyOnSelect, and fullscreenWheelScrollLines. Keybinding additions: fullscreen transcript actions (tui.altScreen.*: page/half-page/line scrolling, prompt navigation, search) with Windows/WSL-aware defaults, and app.message.copy (ctrl+x) which copies an active selection or the last assistant message. /settings switching is blocked while overlays are open; editor/scroll state carries across the switch.
  • Mermaid diagrams: mermaid code fences render as themed Unicode diagrams in the chat instead of raw code blocks (markdown.mermaid setting: off / final / streaming, default streaming; oversized or unsupported diagrams fall back to the original fence, and partial-render warnings are summarized on a single line once the render settles). Adds the registerMarkdownTransformer extension API — extensions can rewrite user/assistant/thinking markdown before terminal rendering with width awareness (MarkdownTransformContext), applied via the new Markdown transform option in the chat components. docs/settings.md documents the setting.
  • Six new built-in color themes beyond the dark/light pair: Nord, Gruvbox (dark + light), Catppuccin (Mocha + Latte), Tokyo Night, and Solarized (dark + light) — hand-tuned 56-color palettes with the full theme surface (thinking ramp, syntax, diff, markdown, export tokens), selectable via /theme or settings.theme alongside custom themes.
  • Clipboard modernization: copy goes through pi-tui's native host-clipboard module when present (with command fallbacks and OSC 52), plus readClipboardText / readClipboardFilePaths; pasting now handles Finder file paths, images, and text.

Providers & integrations

  • Sign in with ChatGPT (/login openai): the openai provider gains an OAuth method (ChatGPT subscription) via the new pi-ai flow — fixed-port browser callback with manual paste fallback, port-in-use failures instead of state-mismatch races, and a device ID (settings deviceId) sent as the agent host ID.
  • vLLM and SGLang local servers: served models discovered from /v1/models with real context windows (max_model_len when reported), listed and selectable like other keyless local providers (/login vllm / /login sglang set a base URL; defaults http://localhost:8000/v1 and http://localhost:30000/v1), refreshable on a 2-min cadence, and configurable via the desktop Settings "Local models" section (VLLM_BASE_URL / SGLANG_BASE_URL env or localProviders.vllmBaseUrl / sglangBaseUrl settings).
  • Fresh installations come with the chrome-devtools MCP server pre-configured: the first run seeds a mcpServers entry (chrome-devtools-mcp@latest via npx, cmd /c npx on Windows) with --isolated (throwaway per-session Chrome profile) and --no-usage-statistics / --no-performance-crux. The seed persists immediately — subsequent partial settings saves keep it — and is then an ordinary user-owned entry: deleting it sticks, and any pre-existing settings file (even {}) disables seeding entirely.
  • Composio app integrations via the hosted Composio Connect MCP server (https://connect.composio.dev/mcp), seeded into fresh-install settings next to chrome-devtools. Auth uses a dashboard consumer API key (COMPOSIO_API_KEY env or the entry's x-consumer-api-key header); header values now support ${ENV_VAR} placeholders (unresolvable placeholders are dropped, never sent half-substituted), and the server runs with a 5-minute request timeout. The MCP transport resolves the meta tools (COMPOSIO_SEARCH_TOOLS, COMPOSIO_MULTI_EXECUTE_TOOL, COMPOSIO_MANAGE_CONNECTIONS, …) as first-class mcp__composio__* agent tools — no SDK involved.
  • Removed: the @composio/core + @composio/experimental SDK integration — the bespoke session-tools/custom-tools wiring, the Composio system-prompt section, and the composio.* settings (enabled, apiKey, toolkits, sandbox, includeWorkbenchTools, callbackUrl; leftovers in old settings files are ignored). App discovery/connect moved entirely into the MCP meta tools, so the /apps CLI command, the composio_list_apps / composio_connect_app / composio_disconnect_app RPC commands, and the desktop Apps panel were removed with it; the desktop Composio settings card is empty by design, pointing at the MCP servers editor.

Skills & sub-agents

  • Bundled first-party skills: the package ships a vendored skill directory (packages/coding-agent/skills/, lowest precedence, source bundled, user/project overrides win) starting with lemo-opuscar — Lemomo's code-driven film directing skill (43 styles, MIT). On first use its setup.sh self-downloads the style library to ~/lemo-opuscar (sparse clone: guides, core tools, every STYLE.md), so film-making works out of the box in the CLI and the desktop. Requires Node 20+, ffmpeg, Python 3.11+/uv for rendering.
  • Self-extensible sub-agents are documented and shipped with examples: docs/subagents.md covers the two built-ins (general-purpose, read-only Explore), the Markdown definition format (frontmatter name/description/tools/disallowedTools/model/maxTurns/permissionMode/isolation with worktree filesystem isolation), project-vs-user precedence, per-spawn overrides, and the spawn_subagent/wait_subagent/get_subagent_status/kill_subagent tool surface; examples/agents/ ships pr-reviewer, test-loop, and an Explore override (with a test asserting they parse), and the agent can scaffold new definitions on request.

Upstream port (tier-1/2)

  • New extension hook before_provider_headers (upstream earendil-works#6350): extensions mutate per-request provider headers in place after static assembly and before the HTTP call; throwing handlers are isolated.
  • Session metadata reaches bash as A_CODER_SESSION_ID / A_CODER_SESSION_FILE / A_CODER_PROVIDER / A_CODER_MODEL / A_CODER_REASONING_LEVEL, resolved per command start (upstream earendil-works#6967), with a new environment-variables doc and a bash exposeSessionEnvironment opt-out.
  • Skills stay available with bash-only toolsets (read OR bash is a loader; upstream earendil-works#8552).
  • Tool-result images are auto-resized per the existing autoResize setting, including images injected by extensions (upstream earendil-works#7330).
  • AI_AGENT env marker for child-tooling attribution (upstream earendil-works#7493); shellPath expands ~ (upstream earendil-works#6470); sub-agents inherit the session's model/per-model overrides and agent config (upstream earendil-works#7897); edit accepts a single {oldText, newText} object in every caller shape it arrives in (upstream earendil-works#8011); image MIME detector export (upstream earendil-works#8600); truncation-recovery failures now say whether context-overflow compaction or an output-cap cut failed (upstream c7c763f); length-truncated summaries are rejected rather than checkpointed (upstream earendil-works#7048 semantics, already present, now tested).

Docs

  • Documentation sweep (2026-10-05): all docs pages verified against the current feature surface. Fixed shipped git-conflict markers in settings.md, stale .pi/ project paths, pi CLI invocations and product naming, dead pi.dev endpoint references, and wrong shortcut claims (Shift+Tab is the permission-mode selector; thinking cycles on Shift+Ctrl+M; tool-output toggle is Ctrl+Shift+O); removed obsolete env rows (PI_CACHE_RETENTION as a CLI-managed env; session env is A_CODER_SESSION_*). settings.md gained the TUI/fullscreen, computer-use, MCP servers, memory, status line/output style, workflows, updates, and local-model-server setting tables; providers.md documents the keyless local model servers (Ollama/Ollama Cloud/LM Studio/llama.cpp/Unsloth/vLLM/SGLang), the new OAuth providers, and the new API-key providers; rpc.md documents all previously-undocumented methods (queue management, permission/plan mode, auth/models, session management, Composio apps, office errands) and the new events; extensions.md gained registerMarkdownTransformer, the before_provider_headers hook, and ui_prompt_start/end; the desktop README now matches the shipped app (MCP editor, local model servers, mobile access, computer use, shortcuts, settings panels); quickstart/index got the current defaults, install script, and browser-control-out-of-the-box note.

Fixed

  • First message after opening the app not appearing in the transcript (until restart): the engine event pipeline registered only after the boot-time state fetches (tree + history + commands) finished, so anything sent inside that window — typically the user's first message — had its echo silently dropped and the reply later appended alone. The pipeline now registers before the fetches (the history-load latch already guards duplicates) in both the main window and the HUD, closing the boot-window race. Desktop-browser settings provisioning also never rewrites a foreign-shaped mcpServers (map-style configs from other tools are left untouched instead of being replaced).
  • Built-in browser tools never appearing in coding sessions: the desktop now provisions the desktop-browser http entry into the engine's global settings at app start (added when missing, re-pointed after a port hop, updated/removed only when the entry still matches the shape it provisions — hand-customized entries are left alone) and clears it when the tools are disabled or the platform can't run the server. The browser settings card documents the auto-entry instead of printing manual JSON.
  • HUD window drag/resize/close all failing: the hud capability file existed but was never registered in tauri.conf.json's capabilities list, so the HUD window ran with no ACL grants — every ACL-gated core:window invoke (startDragging, startResizeDragging, close) was denied while ungated custom commands kept working. The capability is now registered, and drag/close failures are reported through the renderer diagnostics log instead of being swallowed.
  • HUD open failing with Command plugin:window|get_all_windows not allowed by ACL: WebviewWindow.getByLabel needs core:window:allow-get-all-windows, now granted on the main-window capability.
  • OAuth sign-in (/login) in compiled Bun binaries (the shipped pi/a-coder engine executables) failing with ResolveMessage: Cannot find module './openai-codex.ts' from '/$bunfs/root/pi': the binary entrypoint stopped calling registerBunOAuthFlows() (the line was lost in the rebranding), so every OAuth flow fell back to its runtime dynamic import, which cannot resolve inside a standalone executable. The bundled flows (Codex, ChatGPT, Anthropic, Copilot, OpenRouter, Kimi, xAI, Radius) are registered again.
  • Project-scope mcpServers no longer replace the global server list: settings merge is scope-aware and the effective set is the union of global + project entries keyed by name — same-name project entries fully replace the global one, and everything is annotated with a source (global / project). The MCP status store carries the source and /mcp labels project-defined servers (project) (also in disabled/error lines); the desktop MCP editor captions each scope with the merge behavior.
  • TUI crash when a long-running bash command finished: the partial-update path returned the live progress component, which the finished-render path then reused — type-confused — as the bash result component, so the output-preview cache closure rendered against a missing .state and the first synchronous repaint (the fullscreen renderer's mouse-dispatch hit-testing) threw an uncaught TypeError and exited pi. The result renderer now reuses a lastComponent only when it actually is a BashResultRenderComponent.