fix: recognition resilience — keep task alive across jumps, retry transient recognizer unavailability#67
Merged
Conversation
jumpTo tore down the recognition task and started a fresh one, dropping the audio spoken during the restart window (~1s to warm) — exactly the words the user re-speaks right after tapping a word to jump, so the highlight appeared to stop following. Jumps now keep the task running and record a transcript anchor: matching ignores everything transcribed before the jump and continues from the new offset with zero dropped audio. The anchor resets whenever a new task starts a fresh transcript (soft restart, new session). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
beginRecognition and restartTask both gave up permanently when SFSpeechRecognizer reported unavailable — a state that is usually transient (the recognition service churns briefly after a task cancellation or an audio device config change). beginRecognition's guard didn't even set isListening = false, so the app went silently deaf: engine stopped by the preceding cleanup, no retry, no error surfaced, until a full app relaunch. Observed live: a USB mic (Elgato Wave) sample-rate renegotiation ~100ms after engine start triggered restartTask, the cancelled task's error callback scheduled a fresh beginRecognition, and that hit the unavailable guard and returned. Both guards now retry with the same backoff the invalid-format guard already uses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Findings from an adversarial self-review of the first two commits: - restartTask/beginRecognition now clear lastSpokenText along with the anchor: a jump taken after a task restart but before its first result anchored on the old transcript and froze matching until the next ~55s restart. - The anchor is now the transcript prefix string, trimmed by surviving common prefix (with bounded slack) instead of a raw char count, so the recognizer revising earlier text no longer swallows post-jump speech or leaks the pre-jump transcript. - Results delivered within 300ms of a jump are ignored — they were computed against the pre-jump position. - jumpTo no longer resets retryCount (tapping words could keep a failing availability-retry loop alive forever) and falls back to a full restart for far jumps (>500 chars, refreshing contextualStrings for the new section) or when the engine died silently. - A nil SFSpeechRecognizer (unsupported locale) fails fast instead of silently retrying a permanent condition; exhausted retries in restartTask stop the audio engine instead of leaving the mic hot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two resilience fixes for the speech recognition session, both hit during real daily use of word-tracking mode.
1. Jumps restarted the recognition task and dropped the words spoken right after
jumpTo(charOffset:)(word taps) tore down the recognition task and started a fresh one. A freshSFSpeechRecognitionTasktakes up to ~1s to warm, and audio in that window is dropped — which is exactly when the user re-speaks the words they jumped back for. The highlight appeared to stop following after any jump.Jumps now keep the task alive and record a transcript anchor:
matchCharactersignores everything transcribed before the jump and matches from the new offset, so tracking resumes on the first word with zero dropped audio. The anchor resets whenever a genuinely new transcript starts (soft restart, new session), so the ~55s preemptive restarts are unaffected.2. Transient recognizer unavailability made the app permanently deaf
beginRecognitionandrestartTaskboth gave up whenSFSpeechRecognizer.isAvailablewas false. That state is usually transient — the recognition service churns briefly after a task cancellation or an audio device config change. Worse,beginRecognition's guard didn't setisListening = false, so nothing surfaced: the audio engine had already been stopped by the preceding cleanup, no retry was scheduled, and the app stayed deaf until relaunch.Observed live with a USB mic (Elgato Wave): the device renegotiates its sample rate ~100ms after engine start →
AVAudioEngineConfigurationChange→restartTask→ the cancelled task's error callback schedules a freshbeginRecognition→ it lands on the unavailable guard mid-churn and returns. System log showed the engine stop with no restart and no error.Both guards now retry with the same 0.5s backoff (bounded by the existing
maxRetries) that the invalid-format guard a few lines below already uses.Relation to #64
Independent changes — this touches
jumpTo/session lifecycle, #64 touches scroll anchoring and match arbitration. Happy to rebase whichever lands second.Test plan
🤖 Generated with Claude Code