-
Notifications
You must be signed in to change notification settings - Fork 0
Realtime Voice
Experimental realtime voice is enabled by default for direct launches and installed services. The existing compatibility flag remains accepted:
codori start --root ~/Project --experimental-realtime-voiceRealtime voice requires the selected backend to expose the upstream
realtime_conversation feature and V3 voice discovery. Codori asks a newly
started remote-control daemon to enable that feature; if an existing daemon is
incompatible, Codori leaves it untouched and uses the managed app-server
fallback instead. Codori does not modify ~/.codex/config.toml.
To opt out, set "realtimeVoice": { "enabled": false } in
~/.codori/config.json and restart Codori.
Voice is experimental and requires a supported ChatGPT-authenticated Codex runtime plus browser microphone access on localhost or a secure HTTPS origin. Codori uses browser-owned WebRTC signaling through app-server; it does not ask for an OpenAI API key and does not use the direct Realtime WebSocket transport.
After enabling realtime voice, open an existing thread and use the microphone action in the composer. The first activation requests microphone permission and prepares the WebRTC session. Once it is ready, hold the microphone action with a pointer, touch, Space, or Enter while that button is focused; releasing immediately mutes input while keeping the session ready for a follow-up. The adjacent controls mute remote speech or stop and release the entire voice session.
The dashboard landing screen offers Start voice companion. It creates a
persistent projectless gpt-5.6-luna thread with xhigh reasoning, stays on
/, and replaces the landing content with a centered server avatar rather than
opening the chat transcript. The area above that avatar shows subtitles for what
Codori heard and answered, using the same two-exchange window and inactivity
timeout as the in-thread companion bubble. Both subtitle surfaces are
translucent with a backdrop blur, so the content behind them stays faintly
visible. Select the centered avatar to stop the session; navigating elsewhere
keeps the same session alive in the normal corner companion.
While a non-preview voice session is active, Codori requests a screen wake lock in supported secure-context browsers. This is best-effort: browser or operating system policy can deny or release the lock, and manually locking the phone can still suspend browser audio. Native background and screen-off voice capture remains a separate follow-up in issue #91.
Settings → Voice discovers the V3 voice list from the most recently used, materialized workspace runtime and falls back to the built-in Codex-compatible list when no runtime context is available. "Use Codex setting" is the default and sends no per-session voice override; an explicit choice is stored only in that browser and applies to the next session. "Protocol default" identifies the app-server protocol fallback and does not claim to show the active Codex configuration. If a saved voice is no longer advertised, Codori preserves it for diagnostics but safely falls back to the Codex setting.
Voices with a bundled sample have an inline preview action. The nine samples are compact Opus public assets that play locally, so preview does not require a runtime or thread, request microphone access, create a turn, or add hidden conversation history. An active realtime conversation blocks preview, and leaving Voice settings stops local playback.
Voice instructions use the root-level experimental_realtime_ws_backend_prompt
value from config.toml when present. Settings shows that value first, while
"Save browser override" stores a browser-only replacement under
codori:realtime-voice-prompt:v1; "Use config.toml" removes the replacement. If
neither value exists, Codori supplies its language-independent bright, youthful
voice prompt. Instruction changes apply when the next realtime voice session
starts.
The voice status surface shows live and final transcripts and whether Codex is listening, transcribing, delegating, working, or speaking. Spoken requests use app-server's automatic handoff into the active thread, so its existing turn, tool, approval, and final-response UI remains authoritative. Codori does not resubmit the recognized text as a second turn.
While the realtime session is active, the server's selected pet appears above the bottom-right edge of the composer. Its speech bubble keeps only the latest two user/assistant exchange pairs, renders user speech with muted emphasis, and closes five seconds after the newest transcript update while the pet remains visible. The companion is clipped to one sprite frame and scales between 64 and 88 pixels wide according to the viewport; it never displays the 192x208 source frame at native size.
Browser autoplay policy may require an explicit unmute or play gesture before remote speech is audible. Losing focus, hiding the page, releasing or cancelling the pointer, switching threads, disconnecting RPC, or leaving the page disables capture or tears down the owned session. Voice controls are app-scoped: there are no global hotkeys and no background listening. Plain HTTP is supported only on localhost; private remote use requires a secure HTTPS origin supplied outside Codori.