Repository navigation
v0.80.150
·
2728 commits
to main
since this release
0.80.150 makes the fullscreen TUI the default, brings hands-free voice calls and a floating HUD window to the desktop app, and adds an experimental computer tool for desktop control. The engine gains decision-model support for fast, approval-free judgments and a bounded task graph, alongside new providers (Sign in with ChatGPT, vLLM, SGLang) and Composio integrations over MCP. Several boot-time races, HUD window issues, and a TUI crash are fixed.
Highlights
- Fullscreen TUI by default: the terminal UI now runs in the alternate screen with a fixed editor/status dock and a scrolling, searchable transcript;
--tui-mode <regular|fullscreen>overrides per invocation and/settingsswitches live. - Voice call mode (desktop): a hands-free, voice-operated loop around the active session — RMS-gated turn endpointing, sentence-split TTS playback, half-duplex turn-taking with opt-in barge-in — now housed in a right-rail call panel instead of a full-screen overlay.
- HUD mode (desktop):
\hudopens a transparent, frameless, always-on-top floating chat window showing the real transcript and composer over whatever you're working on; it shares the engine connection and event pipeline rather than spawning a second engine. - Experimental
computertool (opt-in via thecomputerUsesetting): desktop control on macOS/Windows/Linux through the cua-driver binary, permission-gated, with hard blocks on destructive shortcuts and payloads and fail-closed recovery. - Decision-model support: a read-only, approval-free
decidetool routes typed judgments through a keyless local server (Ollama, LM Studio, llama.cpp, Unsloth) or OpenRouter, and the auto-mode permission classifier prefers it when reachable. - New providers and integrations: Sign in with ChatGPT (
/login openai), vLLM and SGLang keyless local servers, Composio apps via the hosted Connect MCP server, and chrome-devtools MCP pre-configured on fresh installs.
Details
Desktop
- Browser tool permission modes in the tab bar: Agent: ask (every new host asks once, remembered answers pass) and Agent: auto (ordinary sites run without asking; the sensitive list — mailbox, sign-in, payments — keeps asking in every mode). Persisted as
browser.agentPermissionMode, validated on write, surfaced via the newbrowser_get/set_permission_modecommand pair. - Browser panel auto-reveal: when the assistant's turn invokes a
browser_*tool and the right rail isn't showing the browser panel, the rail slides to the browser tab automatically (deduped per tool-call id so a streaming tool's updates don't re-pop it). - Esc stops an in-flight run from the main window or the HUD via the shared abort path (haptic, user-cancel flag, engine abort), layered through the new escape-layer system (
src/lib/escape-layers.ts, ported from hermes desktop): a single Esc dismisses exactly the top-most urgent surface — pending approval (deny) > command palette > find bar > model picker/command center > tab context menu > titlebar rename > engine stop — with each layer registering while open and lower handlers bailing ondefaultPrevented/priority. - HUD architecture: the engine's
rpc://eventstream is emitted to every webview window and the reducer moved out of App.tsx intosrc/lib/agent-event-pipeline.ts, so the HUD paints state and shares the connection; capabilities live insrc-tauri/capabilities/hud.json, and a titlebar toggle (picture-in-picture icon) sits beside the/hudslash command. - Voice settings model/voice pickers: the STT and TTS model fields and the TTS voice field are dropdowns that load catalogs from
{base}/modelsand{base}/voiceson focus or refresh (fetched through the newvoice_fetch_jsonRust CORS-bridge command); OpenAI{data:[{id}]}, plain arrays,{voices:[…]}, and id/name/voice shapes are accepted, with the manual text-input fallback for endpoints without a catalog or when Custom… is picked.
Voice calls
- The call moved out of the full-screen click-through overlay into the right sidebar's own tab (phone icon; auto-opens when a call starts, pulsing dot on the rail when hidden). The panel carries the call card — mic-level orb, phase caption, tool chip, live dictation, sentence progress, mute/skip/hang-up — plus a capped call log of the user's turns and the agent's replies as they send/play. Esc still interrupts and hangs up; approvals and the composer are no longer covered by a scrim.
- Call mechanics: a plain RMS gate on the mic stream endpoints each utterance (speech confirmed after ~250 ms, sends after configurable silence — default 1200 ms — or a 15 s cap, no ML VAD); mid-run utterances stack as follow-ups via the session's prompt/queue path; the reply plays through a sentence-split TTS queue (two sentences in flight, ordered playback, per-sentence skip, spoken-char limit). Turn-taking defaults to half-duplex (mic parks while the reply speaks; Space/Esc/button tap interrupts; TTS-adjacent audio is always discarded as possible echo) with opt-in open-mic barge-in for headphone users; approval prompts stay clickable.
- While a call is live, the desktop applies a runtime system-prompt addendum (new
set_system_prompt_addendumRPC — appended to the base/extension prompt on every turn while set, cleared when the call ends, never persisted) so spoken replies read like a natural phone call: short spoken-first answers, no markdown/lists/code fences, descriptions instead of pasted output, one clarifying question instead of guessing. - Status sounds reuse the synthesized completion-sound bank (Settings → Look & feel volume applies): a bundled
thinking-sound-voice-mode.mp3plays as the thinking cue (synthesized ping fallback), tool calls tick with the tri-tone, connect/hang-up use their own presets; the turn-end chime is suppressed during a call since the spoken reply is the end-of-turn signal. - Call-mode tuning in Settings → Voice → Call mode: barge-in, status sounds, silence-before-send (300–5000 ms), max utterance (3–120 s), and spoken reply char limit.
Engine
- Decision models (System One wire): the
decidetool routes typed judgments ({state, 1-16 named questions: choice/noul/score}) through a decision model. Resolution order: explicitdecidearguments orsettings.decisions {provider, model}, then the first reachable keyless local server (Ollama, LM Studio, llama.cpp, Unsloth — zero-config after e.g.ollama pull clef-flash), then OpenRouter's JeV beta whenOPENROUTER_API_KEYis set, with an actionable error naming every option. Results render as a probability card (chosen option, bar, confidence) and ship raw answers JSON; the desktop/rpc surface the same card through the generic tool-result flow. The auto-mode permission classifier prefers a decision model over the LLM classifier when one is reachable: the gate is a{verdict: choice[allow|block], risk: score}question pair scored on the tool-call proposal, mapped onto the same verdict/circuit-breaker semantics (failures degrade to the LLM classifier, blocks degrade to a prompt), and the block reason carries the engine and probability (Auto-mode classifier (clef-flash p=0.94): ...). Docs: docs/decision-models.md; decision models deliberately stay out of the chat/modellist. - Bounded task graphs: the graph is now a work frontier instead of ever-growing history. Completed tasks get a
completedAtstamp; past the 15 most recent, completed tasks auto-archive toarchive.jsonl(read-only history — ids stay resolvable viatask_get,task_listgainsinclude_archived: true, archived tasks refusetask_updatemutations except explicit deletion, and delete cascades reach archived rows). Tool-result snapshots carryarchivedCountso the/taskspanel and the desktop Task panel show the open frontier first with a "+N archived" tail. Stale per-session task-list dirs are removed once per process at startup after 30 days untouched — settings keytaskRetentionDays(0 disables, mirrored in the desktop settings). Deleting an archived task no longer leaves dangling references in live siblings. - RPC engines announce themselves: the first stdout frame in
--mode rpcis anengine_infoevent carrying the engine version and a protocol contract number (RPC_CONTRACTin rpc-types.ts, currently 1), so embedded clients can detect engine/client skew early instead of failing cryptically downstream; clients predating the event ignore it. - Engine binary builds are hardened against the upstream Bun non-AVX2 regression (oven-sh/bun#30613): x64 engine targets are compiled with bun 1.3.8 — the last version that works on CPUs/VMs without SSE4.2/POPCNT (bun >= 1.3.9's WebKit
-march=nehalemchange makes compiled engines crash with 0xC0000005 at startup) — while arm64 targets (including the windows-arm64 cross-compile, which exists only since bun 1.3.10) build with bun 1.3.13. CI supplies both (BUN_BASELINE_BIN), andbuild-binaries.shenforces the per-ISA policy with instructions.
Computer use
- The experimental
computertool (default off; theA_CODER_CLI_COMPUTER_USE=1env still overrides) drives desktop control on macOS/Windows/Linux through the external cua-driver binary over MCP-on-stdio. One tool with anactiondiscriminator:capture(som = screenshot with numbered element overlays + AX element list, vision = plain screenshot, ax = tree only),click/double_click/right_click/middle_click,drag,scroll,type,key,set_value,wait,list_apps,list_windows,focus_app. Element-index clicking with snapshotelement_tokenstaleness detection; background-first input delivery (no focus steal) withdelivery_mode: "foreground"gated on the driver's live schema (refused, never silently downgraded); sticky-target guard returnsinput_target_mismatchinstead of typing into the wrong window; every result carries a verdict (done / verify fresh state / escalate). Destructive system shortcuts (logout/lock/empty-trash, alias- and hyphen-proof) and dangeroustypepayloads (pipe-to-shell,sudo rm -rf, fork bombs) are hard-blocked before approval. Permission-aware: read actions (capture/list/wait) are free; input actions weigh like mutating tools in every mode (blocked in read-only, prompt in ask, classifier-checked in auto). Fail-closed recovery: a timed-out driver call's outcome is unknown — never replayed, transport rebuilt before the next call. Driver resolution:A_CODER_CUA_DRIVER_CMDoverride, then PATH, then~/.a-coder/cli/agent/bin/(cua-driver.exeon Windows;whereinstead ofwhichon Windows); child env strips provider API keys and disables cua-driver telemetry by default. The TUI settings screen gates enabling behind a disclaimer submenu with an explicit "I understand — enable" choice; the toggle takes effect when a session rebuilds its tool runtime. - Actionable install guidance: when the cua-driver binary is missing, failed
computercalls (includingaction: "doctor") return abackend_unavailablepayload with aninstallobject — platform/arch-matched release asset (tagcua-driver-rs-v0.33.3), direct URL, the agent bin dir, and copy-pasteable install commands. Newdocs/computer-use.mddocuments enabling, backend resolution, install commands, doctor diagnostics, and the security posture.
TUI
- Fullscreen mode configuration:
tuiMode,fullscreenExitOutput("transcript" reprints the transcript on exit, "resume-hint" prints only the resume hint),fullscreenScrollbar,fullscreenCopyOnSelect, andfullscreenWheelScrollLines. Keybinding additions: fullscreen transcript actions (tui.altScreen.*: page/half-page/line scrolling, prompt navigation, search) with Windows/WSL-aware defaults, andapp.message.copy(ctrl+x) which copies an active selection or the last assistant message./settingsswitching is blocked while overlays are open; editor/scroll state carries across the switch. - Mermaid diagrams:
mermaidcode fences render as themed Unicode diagrams in the chat instead of raw code blocks (markdown.mermaidsetting:off/final/streaming, defaultstreaming; oversized or unsupported diagrams fall back to the original fence, and partial-render warnings are summarized on a single line once the render settles). Adds theregisterMarkdownTransformerextension API — extensions can rewrite user/assistant/thinking markdown before terminal rendering with width awareness (MarkdownTransformContext), applied via the newMarkdowntransform option in the chat components.docs/settings.mddocuments the setting. - Six new built-in color themes beyond the dark/light pair: Nord, Gruvbox (dark + light), Catppuccin (Mocha + Latte), Tokyo Night, and Solarized (dark + light) — hand-tuned 56-color palettes with the full theme surface (thinking ramp, syntax, diff, markdown, export tokens), selectable via
/themeorsettings.themealongside custom themes. - Clipboard modernization: copy goes through pi-tui's native host-clipboard module when present (with command fallbacks and OSC 52), plus
readClipboardText/readClipboardFilePaths; pasting now handles Finder file paths, images, and text.
Providers & integrations
- Sign in with ChatGPT (
/login openai): theopenaiprovider gains an OAuth method (ChatGPT subscription) via the new pi-ai flow — fixed-port browser callback with manual paste fallback, port-in-use failures instead of state-mismatch races, and a device ID (settingsdeviceId) sent as the agent host ID. - vLLM and SGLang local servers: served models discovered from
/v1/modelswith real context windows (max_model_lenwhen reported), listed and selectable like other keyless local providers (/login vllm//login sglangset a base URL; defaultshttp://localhost:8000/v1andhttp://localhost:30000/v1), refreshable on a 2-min cadence, and configurable via the desktop Settings "Local models" section (VLLM_BASE_URL/SGLANG_BASE_URLenv orlocalProviders.vllmBaseUrl/sglangBaseUrlsettings). - Fresh installations come with the chrome-devtools MCP server pre-configured: the first run seeds a
mcpServersentry (chrome-devtools-mcp@latestvia npx,cmd /c npxon Windows) with--isolated(throwaway per-session Chrome profile) and--no-usage-statistics/--no-performance-crux. The seed persists immediately — subsequent partial settings saves keep it — and is then an ordinary user-owned entry: deleting it sticks, and any pre-existing settings file (even{}) disables seeding entirely. - Composio app integrations via the hosted Composio Connect MCP server (
https://connect.composio.dev/mcp), seeded into fresh-install settings next to chrome-devtools. Auth uses a dashboard consumer API key (COMPOSIO_API_KEYenv or the entry'sx-consumer-api-keyheader); header values now support${ENV_VAR}placeholders (unresolvable placeholders are dropped, never sent half-substituted), and the server runs with a 5-minute request timeout. The MCP transport resolves the meta tools (COMPOSIO_SEARCH_TOOLS,COMPOSIO_MULTI_EXECUTE_TOOL,COMPOSIO_MANAGE_CONNECTIONS, …) as first-classmcp__composio__*agent tools — no SDK involved. - Removed: the
@composio/core+@composio/experimentalSDK integration — the bespoke session-tools/custom-tools wiring, the Composio system-prompt section, and thecomposio.*settings (enabled,apiKey,toolkits,sandbox,includeWorkbenchTools,callbackUrl; leftovers in old settings files are ignored). App discovery/connect moved entirely into the MCP meta tools, so the/appsCLI command, thecomposio_list_apps/composio_connect_app/composio_disconnect_appRPC commands, and the desktop Apps panel were removed with it; the desktopComposiosettings card is empty by design, pointing at the MCP servers editor.
Skills & sub-agents
- Bundled first-party skills: the package ships a vendored skill directory (
packages/coding-agent/skills/, lowest precedence, sourcebundled, user/project overrides win) starting withlemo-opuscar— Lemomo's code-driven film directing skill (43 styles, MIT). On first use itssetup.shself-downloads the style library to~/lemo-opuscar(sparse clone: guides, core tools, every STYLE.md), so film-making works out of the box in the CLI and the desktop. Requires Node 20+, ffmpeg, Python 3.11+/uv for rendering. - Self-extensible sub-agents are documented and shipped with examples:
docs/subagents.mdcovers the two built-ins (general-purpose, read-onlyExplore), the Markdown definition format (frontmattername/description/tools/disallowedTools/model/maxTurns/permissionMode/isolationwithworktreefilesystem isolation), project-vs-user precedence, per-spawn overrides, and thespawn_subagent/wait_subagent/get_subagent_status/kill_subagenttool surface;examples/agents/shipspr-reviewer,test-loop, and anExploreoverride (with a test asserting they parse), and the agent can scaffold new definitions on request.
Upstream port (tier-1/2)
- New extension hook
before_provider_headers(upstream earendil-works#6350): extensions mutate per-request provider headers in place after static assembly and before the HTTP call; throwing handlers are isolated. - Session metadata reaches bash as
A_CODER_SESSION_ID/A_CODER_SESSION_FILE/A_CODER_PROVIDER/A_CODER_MODEL/A_CODER_REASONING_LEVEL, resolved per command start (upstream earendil-works#6967), with a new environment-variables doc and a bashexposeSessionEnvironmentopt-out. - Skills stay available with bash-only toolsets (
readORbashis a loader; upstream earendil-works#8552). - Tool-result images are auto-resized per the existing
autoResizesetting, including images injected by extensions (upstream earendil-works#7330). AI_AGENTenv marker for child-tooling attribution (upstream earendil-works#7493);shellPathexpands~(upstream earendil-works#6470); sub-agents inherit the session's model/per-model overrides and agent config (upstream earendil-works#7897); edit accepts a single{oldText, newText}object in every caller shape it arrives in (upstream earendil-works#8011); image MIME detector export (upstream earendil-works#8600); truncation-recovery failures now say whether context-overflow compaction or an output-cap cut failed (upstream c7c763f); length-truncated summaries are rejected rather than checkpointed (upstream earendil-works#7048 semantics, already present, now tested).
Docs
- Documentation sweep (2026-10-05): all docs pages verified against the current feature surface. Fixed shipped git-conflict markers in settings.md, stale
.pi/project paths,piCLI invocations and product naming, deadpi.devendpoint references, and wrong shortcut claims (Shift+Tab is the permission-mode selector; thinking cycles on Shift+Ctrl+M; tool-output toggle is Ctrl+Shift+O); removed obsolete env rows (PI_CACHE_RETENTIONas a CLI-managed env; session env isA_CODER_SESSION_*). settings.md gained the TUI/fullscreen, computer-use, MCP servers, memory, status line/output style, workflows, updates, and local-model-server setting tables; providers.md documents the keyless local model servers (Ollama/Ollama Cloud/LM Studio/llama.cpp/Unsloth/vLLM/SGLang), the new OAuth providers, and the new API-key providers; rpc.md documents all previously-undocumented methods (queue management, permission/plan mode, auth/models, session management, Composio apps, office errands) and the new events; extensions.md gainedregisterMarkdownTransformer, thebefore_provider_headershook, andui_prompt_start/end; the desktop README now matches the shipped app (MCP editor, local model servers, mobile access, computer use, shortcuts, settings panels); quickstart/index got the current defaults, install script, and browser-control-out-of-the-box note.
Fixed
- First message after opening the app not appearing in the transcript (until restart): the engine event pipeline registered only after the boot-time state fetches (tree + history + commands) finished, so anything sent inside that window — typically the user's first message — had its echo silently dropped and the reply later appended alone. The pipeline now registers before the fetches (the history-load latch already guards duplicates) in both the main window and the HUD, closing the boot-window race. Desktop-browser settings provisioning also never rewrites a foreign-shaped
mcpServers(map-style configs from other tools are left untouched instead of being replaced). - Built-in browser tools never appearing in coding sessions: the desktop now provisions the
desktop-browserhttp entry into the engine's global settings at app start (added when missing, re-pointed after a port hop, updated/removed only when the entry still matches the shape it provisions — hand-customized entries are left alone) and clears it when the tools are disabled or the platform can't run the server. The browser settings card documents the auto-entry instead of printing manual JSON. - HUD window drag/resize/close all failing: the
hudcapability file existed but was never registered intauri.conf.json'scapabilitieslist, so the HUD window ran with no ACL grants — every ACL-gatedcore:windowinvoke (startDragging, startResizeDragging, close) was denied while ungated custom commands kept working. The capability is now registered, and drag/close failures are reported through the renderer diagnostics log instead of being swallowed. - HUD open failing with
Command plugin:window|get_all_windows not allowed by ACL:WebviewWindow.getByLabelneedscore:window:allow-get-all-windows, now granted on the main-window capability. - OAuth sign-in (
/login) in compiled Bun binaries (the shippedpi/a-coder engine executables) failing withResolveMessage: Cannot find module './openai-codex.ts' from '/$bunfs/root/pi': the binary entrypoint stopped callingregisterBunOAuthFlows()(the line was lost in the rebranding), so every OAuth flow fell back to its runtime dynamic import, which cannot resolve inside a standalone executable. The bundled flows (Codex, ChatGPT, Anthropic, Copilot, OpenRouter, Kimi, xAI, Radius) are registered again. - Project-scope
mcpServersno longer replace the global server list: settings merge is scope-aware and the effective set is the union of global + project entries keyed by name — same-name project entries fully replace the global one, and everything is annotated with asource(global/project). The MCP status store carries the source and/mcplabels project-defined servers(project)(also in disabled/error lines); the desktop MCP editor captions each scope with the merge behavior. - TUI crash when a long-running bash command finished: the partial-update path returned the live progress component, which the finished-render path then reused — type-confused — as the bash result component, so the output-preview cache closure rendered against a missing
.stateand the first synchronous repaint (the fullscreen renderer's mouse-dispatch hit-testing) threw an uncaught TypeError and exited pi. The result renderer now reuses alastComponentonly when it actually is aBashResultRenderComponent.