Fail closed instead of pasting Hebrew when English is forced#2
Open
chkp-roniz wants to merge 1 commit into
Open
Fail closed instead of pasting Hebrew when English is forced#2chkp-roniz wants to merge 1 commit into
chkp-roniz wants to merge 1 commit into
Conversation
The Ivrit.ai Hebrew fine-tune can drift into Hebrew script even when a non-Hebrew language is explicitly requested, especially with the whisper temperature-fallback decode pass enabled. Disable that fallback for forced languages, detect Hebrew script in output that shouldn't contain it, retry once, and refuse to paste rather than inserting the wrong language. Also split the ASR client, language, and transcript-cleaning logic into a new RisperTranscription library target with zero AppKit/AVFoundation/OSLog dependencies, so this logic is unit-testable (see new Tests/RisperTests/ASRClientGuardrailTests.swift) without needing the full macOS app or a bundled model. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I don't currently have a machine with a Swift toolchain, the bundled Ivrit
model, or whisper.cpp built, so none of this has been compiled, unit
tested, or run against the app. Please review carefully and run
swift build/swift test/ the manual QA steps below before merging.Summary
Closes #1 — selecting English as the dictation language could still result in
Hebrew text being pasted.
Root cause: the app bundles a single Hebrew-tuned Whisper model
(Ivrit.ai
whisper-large-v3-turbo). Its own documentation notes degradedlanguage handling, and whisper's temperature-fallback decode pass could let
this Hebrew-biased model drift into Hebrew script even when English was
explicitly forced via the per-request
language=enfield (which was alreadycorrect).
(English or Hebrew); keep it only for auto-detect.
Hebrew script, retry once; if it still comes back in Hebrew, fail closed
(no paste) instead of inserting the wrong-language text.
--language heserver startup default (nowauto),since per-request fields already govern actual behavior.
Refactor (same PR, supports the above)
Extracted
ASRClient,DictationLanguage,TranscriptCleaner, and a newRecordingResultinto a newRisperTranscriptionlibrary target with zeroAppKit/AVFoundation/OSLog dependencies. This makes the guardrail logic
unit-testable on any machine with a Swift toolchain, without needing the full
macOS app or a bundled model — see
Tests/RisperTests/ASRClientGuardrailTests.swift,which mocks the HTTP layer with
URLProtocolto test the retry/fail-closedbehavior.
Test plan
swift buildswift test(newASRClientGuardrailTests+TranscriptCleanerTests)script/asr_harness.shstill passes (Hebrew regression)fntrigger, confirm English is always inserted and Hebrew is neverpasted
🤖 Generated with Claude Code