Hands-free thread dispatch from CarPlay and Siri

[ntindle]

Problem

Ideas for agent work come up while driving, but starting a thread needs the phone in hand. A driver should be able to say what they want built and have it start in the right project.

What exists today

• Voice input. Mobile composer dictation records with expo-audio and transcribes on device with Apple's SpeechAnalyzer/SpeechTranscriber (iOS 26+) through @react-native-ai/apple. Audio never leaves the phone, and nothing is sent until the user submits. Web has no voice input, and the server has no audio or transcription endpoint.
• Internal inference. The server's TextGeneration service runs one-shot structured prompts through the user's own provider (textGenerationModelSelection, Codex by default) for thread titles, commit messages, PR text and branch names. It has four fixed operations, so a new kind of inference, such as picking a project, means a new operation in every provider's text generation.
• Starting threads. Clients call orchestration.launchThread. Mobile's new-task flow resolves project defaults (model, runtime mode, local or worktree, base branch, t3.json) and sends through the offline-capable thread outbox.

CarPlay constraints (CarPlay Developer Guide, June 2026)

• iOS 26.4 added a CarPlay category for voice-based conversational apps, entitlement com.apple.developer.carplay-voice-based-conversation. Apple grants it per team after review.
• Allowed templates include voice control (26.4+, with up to two action buttons), list, grid, alert and action sheet. Template depth is limited to 3.
• The app must launch into voice, hold its audio session only while voice is in use, and record only alongside the voice control template. There is no wake word; Siri remains the system assistant.
• CarPlay is often used while the iPhone is locked. Keychain items with WhenUnlocked accessibility can't be read then, which is what mobile uses for connection credentials.
• Siri App Intents work in CarPlay without any CarPlay entitlement.

Proposal

v0: the driver picks the project

1. CarPlay voice app. It opens straight into "Which project?" and matches the spoken name against connected projects, with a tappable list as fallback. Then "What do you want to build?", on-device transcription, a read-back, and "Should I start it?". Nothing starts without a spoken or tapped confirmation.
2. Siri. A "Start a thread in T3 Code" App Intent takes a project and a request, plus a shortcut per project. It works in the car through Siri before Apple grants the CarPlay entitlement.
3. No server change. Threads start with the project's defaults through the existing outbox, so this works with existing servers and an offline environment gets the thread when it reconnects. Transcription stays on device.

v1: T3 Code picks the project

A server dispatch service takes the voice note and chooses the project (or an existing thread to continue) with internal inference, returning a ranked choice with confidence. Low confidence falls back to asking. As a service method behind an RPC and an MCP tool, agents and scheduled tasks could dispatch the same way. It needs a new TextGeneration operation per provider.

Later: spoken status ("what's running?"), answering pending agent questions by voice, and Android Auto, which has no equivalent conversational category today.

Prototype

v0 is prototyped on ntindle/t3code@feat/carplay-voice-dispatch (diff against main). It adds:

• a local Expo module with the CarPlay scene, speech I/O and a bridge;
• a config plugin that adds the CarPlay scene, App Intents, and an opt-in entitlement (T3CODE_IOS_CARPLAY=1);
• the JS conversation and dispatch, with unit tests.

CarPlay and Siri can launch the app before its phone screen exists, so the bridge starts React Native in a hidden window and the template SceneDelegate adopts it later. There is only ever one React tree.

Verified on an iOS 27 simulator against an isolated dev server:

Siri shortcuts, one per project	Confirmation	Result	Cold start: app terminated first

• Siri, warm and cold. Both created threads; the server titled them. After the cold start, opening the app showed the live thread list from the same process, so the phone scene adopted the headless app.
• CarPlay conversation. It ran end to end through the real outbox, with transcripts injected through a temporary harness.

Not verified:

• The CarPlay screen. Xcode 27.0's Device Hub has no control for the simulator's CarPlay display (FB24785359). It needs Apple's CarPlay Simulator app with a cabled iPhone, or a car, on a team that has the entitlement.
• Live microphone and speech output on the simulator. These need a macOS microphone grant for CoreSimulator.

Asking for

Approval of the v0 direction and scope before I open a PR, and your take on:

1. Requesting the CarPlay voice-based conversation entitlement for the T3 Tools team. Until then the CarPlay screen stays opt-in and simulator-only, while Siri works in every build.
2. Moving connection credentials to AfterFirstUnlock, as Clerk's token cache already does, so a cold launch from CarPlay or Siri works while the phone is locked.
3. Whether voice-started threads should always use a worktree, since nobody watches the first turn.

(I opened #18273 and #18275 before checking CONTRIBUTING.md; both are closed in favor of this discussion.)