Follow-up cleanups for recent chat and voice PRs#1497
Conversation
Follow up on PR #1458 by preserving Flux turn transcripts across lifecycle events and using model-detected speech start for low-latency barge-in. Co-authored-by: Cursor <cursoragent@cursor.com>
Follow-up to PR #1463: route stream-resume negotiation sends through close-safe helpers so WebSocket close races do not crash resume handling in think and ai-chat. Co-authored-by: Cursor <cursoragent@cursor.com>
Follow-up to PR #1462: make the voice text stream parser honor its documented NDJSON support while preserving SSE parsing for AI text streams. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
🟡 Client-side interrupt detection bypasses #playbackGeneration guard, allowing stale audio to play after interrupt
The PR introduces #playbackGeneration and #stopPlayback() (at packages/voice/src/voice-client.ts:766-779) to prevent a race where a pending decodeAudioData resolves after playback is stopped and starts unwanted audio. #stopPlayback() increments #playbackGeneration, and #playAudio() checks it at lines 726 and 731. However, the client-side interrupt path in #processAudioLevel still uses the old manual pattern — it clears #activeSource, #playbackQueue, and #isPlaying without incrementing #playbackGeneration. If decodeAudioData is pending when the client-side interrupt fires, the decode resolves, the generation check passes (generation hasn't changed), and unwanted audio plays after the user interrupted.
(Refers to lines 863-867)
Was this helpful? React with 👍 or 👎 to provide feedback.
agents
@cloudflare/ai-chat
@cloudflare/codemode
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
Co-authored-by: Cursor <cursoragent@cursor.com>
|
| Name | Type |
|---|---|
| agents | Patch |
| @cloudflare/ai-chat | Patch |
| @cloudflare/voice | Patch |
| @cloudflare/think | Patch |
Click here to learn what changesets are, and how to add one.
Click here if you're a maintainer who wants to add a changeset to this PR
Co-authored-by: Cursor <cursoragent@cursor.com>
Ensure recovered agent-tool finish hooks are only executed after a successful user onStart. Await _runDeferredAgentToolFinishHooks inside the onStart flow so deferred finishes are skipped when startup fails. Add a test and helper (reconcileCompletedChildWithFailedStartupForTest) to verify finish hooks are not run on failed startup and to cover lifecycle ordering and event emission.
Thank you @whoiskatrin for the very good PRs here. I spent some time reviewing the recent fixes in detail and I am adding a handful of cleanups on top because I am, unfortunately, a little anal about tightening the nearby edge cases once I see them. Sorry for the extra follow-up churn, but these are all meant to preserve the direction of the landed PRs while hardening the corners around them.
Summary
useVoiceAgent, agent-tool recovery, chat resume negotiation, and ai-chat socket disconnect fixes.Commit Details
bd1348a4-fix(voice): harden Workers AI STT turn handlingBuilds on PR #1458.
This expands the Workers AI STT fix beyond the original late/empty final transcript issue:
Update,StartOfTurn, andTurnResumed, so an emptyEndOfTurncan still emit the best known utterance.StartOfTurnas a model-driven speech-start signal for server-side barge-in.playback_interrupthandling so the server can tell the client to stop playback when user speech starts during assistant audio.VoiceClientso a staledecodeAudioData()task cannot start playing after an interrupt has already arrived.f7b1b344-fix(chat): harden stream resume negotiation close racesBuilds on PR #1463.
This extends the
sendIfOpenpattern from the original replay fallback fix to the rest of the resume negotiation paths:@cloudflare/thinkand@cloudflare/ai-chatsoTypeError: WebSocket send() after closedoes not bubble out of close races.STREAM_RESUMINGnotification send actually succeeds.ContinuationState.sendResumeNone()in the shared agents chat continuation path.2954bd35-fix(voice): parse raw NDJSON text streamsBuilds on PR #1462.
This fills the NDJSON half of the documented
iterateText()byte-stream support:data:lines.choices[0].delta.content,response, split byte chunks, final unterminated lines, raw[DONE], and malformed raw JSON lines.data:{...}lines without requiring a space afterdata:.textStreamcustom async iterator preference from PR Fix voice TTS for AI SDK textStream responses #1462.5aefa984-fix(agents): defer recovered agent-tool finish hooks (#1476)Builds on PR #1476.
This addresses recovery lifecycle gaps found during review of agent-tool finalization:
onAgentToolFinishhooks until after the agent's useronStarthas completed, so user startup/mirror initialization runs before recovered finish hooks execute.onError, so one failed hook does not prevent later recovered runs from draining or agent startup from completing.527c5ba2-test(voice): cover useVoiceAgent enabled lifecycle (#1478)Builds on PR #1478.
This adds follow-up tests/docs around the new
useVoiceAgent({ enabled })gate:onReconnectfires for real connection identity changes while enabled, but not for first enable.b1497715-fix(ai-chat): close resumed streams on disconnect (#1487)Builds on PR #1487.
PR #1487 correctly closed the original
sendMessages()stream when the socket closes before a terminal frame. This extends the same transport-owned stream cleanup to the other WebSocket-fed paths:done: true.Test Plan
Ran focused verification while developing the commits:
npm run build -w agentsnpm run build -w @cloudflare/ai-chatnpm --workspace @cloudflare/voice run buildnpm run test:workers -w @cloudflare/ai-chat -- src/tests/agent-tools.test.tsnpm run test:workers -w @cloudflare/ai-chat -- src/tests/ws-transport-resume.test.tsnpm run test:react -w @cloudflare/ai-chat -- use-agent-chat.test.tsxnpm run test:workers -w @cloudflare/think -- agent-tools.test.tsnpm --workspace @cloudflare/voice run test:react -- useVoiceAgent.test.tsxnpx tsc -p packages/ai-chat/src/tests/tsconfig.json --noEmitoxfmt --check/ lint checks for edited filesMade with Cursor