Skip to content

(CODUX-219) Add ordered prefetching TTS playback - #3022

Merged
numnx merged 1 commit into
feature/CODUX-219-fix-chat-continuations-and-tts-playbackfrom
task/feature-codux-21-t04-codex-dc59a37c-mrjht7zu
Jul 13, 2026
Merged

(CODUX-219) Add ordered prefetching TTS playback#3022
numnx merged 1 commit into
feature/CODUX-219-fix-chat-continuations-and-tts-playbackfrom
task/feature-codux-21-t04-codex-dc59a37c-mrjht7zu

Conversation

@numnx

@numnx numnx commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

🧩 Task: Add ordered prefetching TTS playback

Automated task PR opened by Code UX for T04.

📋 Summary

Sprint: Sprint 219 · Fix chat continuations and TTS playback · Task: T04

Keep agent-owned Project Manager continuations bound to their originating chat thread without cancellation cascades or duplicate turns, and deliver long Chat replies through a fast, sentence-chunked, ordered TTS pipeline.

🤖 Model & Provider

Provider Codex CLI
Model gpt-5.6-sol

⏱️ Timing

Started 2026-07-13 17:25:06 UTC
Finished 2026-07-13 17:34:33 UTC
Duration 9m 27s

📊 Token Usage

Metric Value
Input tokens 145,162
Cached input tokens 3,228,160
Output tokens 21,211
Total tokens 3,394,533
Invocations 1
Tool calls 37
Active time 8m 54s

Included usage estimate (subscription/local login): $2.98

✅ QA Review

⏳ QA review runs after this PR is opened. Check the task card in the dashboard for the outcome once it lands.

📝 Full Task Prompt
## Objective
Upgrade the shared dashboard speech hook into a cancellable, bounded, ordered TTS pipeline used by 3D Chat, thread replay, and invocation replay.

## Scope
- `dashboard/src/v2/hooks/use-speech-playback.ts`
- `dashboard/src/v2/lib/speech-api.ts`
- `dashboard/src/v2/hooks/__tests__/use-speech-playback.test.tsx` (create)
- `dashboard/src/v2/lib/__tests__/speech-api.test.ts`
- `dashboard/src/v2/ChatPage.tsx`
- Existing consumers in `dashboard/src/v2/components/chat/cinematic/CinematicStage.tsx`, `ChatMessageBubble.tsx`, and `InvocationMessageBubble.tsx`
- `docs/architecture/speech-output.md`
- Mirrored public speech and Chat pages under `docs-web/architecture/speech-output.md`, `docs-web/content/docs/architecture-speech-output.mdx`, `docs-web/user/dashboard/chat.md`, and `docs-web/content/docs/user-dashboard-chat.mdx`

## Implementation Requirements
1. Extend `synthesizeSpeech` with an optional `AbortSignal`, pass it to the synthesis fetch, and give each playback run its own abort controller so stop, replay replacement, thread/mode changes, and unmount cancel pending synthesis and release audio resources.
2. Start synthesis for the first sentence immediately, prefetch a bounded number of later chunks while the current audio plays, store results by chunk index, and play only the next contiguous index; stop the pipeline on stale sequence, synthesis error, browser playback error, or explicit stop without allowing late promises to restart audio.
3. Revoke every object URL, detach audio handlers, clear active state only for the current run, surface accessible playback/synthesis errors, and preserve per-project mute behavior, silent history baselines, explicit replay outside 3D Chat, and one-channel replacement semantics.
4. Add deterministic hook and API tests with deferred synthesis and fake audio objects proving first-audio latency, prefetch overlap, strict playback order, complete multi-chunk playback, cancellation cleanup, stale-result suppression, and visible error propagation; wire transcript replay errors into the existing Chat status surface where they are currently unused.

## Constraints
- Consume the sentence-chunking contract from T03 and never exceed the backend 8,000-character request bound.
- Do not autoplay loaded history, change voice settings, expose API credentials, or introduce a new client protocol shape.
- Keep browser compatibility with existing `Audio`, `URL.createObjectURL`, and Preact hook patterns; do not add dependencies.

## Verification
- Run `pnpm exec vitest run dashboard/src/v2/hooks/__tests__/use-speech-playback.test.tsx dashboard/src/v2/lib/__tests__/speech-api.test.ts dashboard/src/v2/components/chat/__tests__/ChatPage.speech.test.tsx`.
- Run `pnpm run typecheck:dashboard` and `pnpm run build:dashboard`.
- Run `pnpm run check:docs-web` and confirm long replies begin with the first ready sentence, continue through every ordered chunk, and report failures without silencing the transcript.
🌿 Branch Info

Base: feature/CODUX-219-fix-chat-continuations-and-tts-playback
Head: task/feature-codux-21-t04-codex-dc59a37c-mrjht7zu


🤖 Generated by Code UX

@numnx
numnx merged commit 565cf0d into feature/CODUX-219-fix-chat-continuations-and-tts-playback Jul 13, 2026
20 checks passed
@numnx
numnx deleted the task/feature-codux-21-t04-codex-dc59a37c-mrjht7zu branch July 13, 2026 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant