Repository navigation
Rip Voice Gateway Research
Task: 6hJWPgW22rrhCx9P
Status: recommendation only, current 2026-09-22; no paid calls or physical-device test.
Conditionally yes for a later, one-way “speak an annotation → show editable text” enhancement; no for adopting a full realtime voice agent now. AI Gateway’s streaming transcription is a plausible fit because it accepts live raw PCM and yields transcript deltas/finals over a WebSocket. It is beta, however, so start behind a feature flag and only after a small authenticated prototype proves the browser-to-Gateway topology and iPhone behavior. Vercel says live transcription streams captured audio and returns incremental transcript updates (changelog); its current speech-to-text docs explicitly mark the feature beta/rolling out and distinguish it from two-way voice (docs).
This is not a current rip capability. ripSession.ts is a pure card-frame state machine (onFrame), not an audio pipeline. The actual capture hook requests getUserMedia with literal audio: false (camera.ts); CaptureStage owns a muted camera <video>. And deckeVoice.ts is deterministic greeting/farewell text, not sound. The repo already pins ai 7.0.94 and @ai-sdk/gateway 4.0.76, above Vercel’s documented minimum for transcription; installed exports/types prove experimental_streamTranscribe, gateway.transcriptionModel, and gateway.experimental_transcription.getToken exist. Deck-E’s existing Gateway credential gate is server-only chat configuration, not a microphone implementation.
Use streaming transcription, not batch transcribe, for in-session annotations. Batch STT takes a completed file and returns a transcript; it is appropriate only if recording is intentionally added. Do not substitute realtime conversation: that is bidirectional audio-in/audio-out, with a browser WebSocket and a server-minted short-lived credential. Vercel documents a separate browser token flow for streaming transcription too (single use; 60 seconds default, 300 seconds maximum), so do not assume a realtime-voice hook authenticates STT identically. The docs show Gateway↔model WebSockets and direct browser token use, but do not establish that an Express/Vercel server function can reliably host an unbounded PCM relay; prototype that specific hosting route before selecting it.
“Sound later” splits in two. Spoken replies: yes—add TTS or a realtime voice model later, independently; transcription alone only produces text. Replay original rip audio: no—transcription does not retain an audio artifact. Replay would require an explicit recording design: visible consent, start/stop and deletion behavior, encrypted/authorized storage and retention policy, timestamp alignment, and cost/privacy review.
- Prototype one-way mic capture only: authenticated API mints a model-scoped, short-lived transcription token; the browser sends PCM and renders provisional/final text. Keep the long-lived Gateway credential server-side, apply per-user credit/cost limits, abort on cancel/navigation, and never write collection data from a delta.
- Convert only a final transcript into a proposed card annotation; require review/confirmation before collection writes. Ask for confirmation (or force explicit selection) for uncertain card/quantity/variant language—voice text is not ground truth.
- Decide separately whether TTS/realtime conversation improves the rip workflow, and separately whether recording/replay is worth its compliance/storage burden. Pricing was not measured here; do not infer that it is cheap.
An installed-PWA iPhone camera+microphone test remains required and was not performed. Gateway cannot bypass getUserMedia or platform restrictions; browser SpeechRecognition support would not prove simultaneous camera/mic support. Test one shared stream or correct camera/mic lifecycle, permission denial, interruption/resume, screen lock/background, reconnect/cancel, and—if speech playback is added—echo/feedback while camera capture continues. Realtime’s server-secret pattern is documented separately (realtime docs); it should remain a later architecture decision, not an adoption claimed by this research.
Last updated by Codex research worker on behalf of @cheyras -- 2026-09-22
Research evidence is retained with task 6hJWPgW22rrhCx9P; this page is a recommendation publication only, not an implementation or decision record.