feat: improve Foundry local ASR setup#292
Conversation
# Conflicts: # openless-all/app/src/pages/QaPanel.tsx
…-asr # Conflicts: # openless-all/app/src-tauri/Cargo.lock # openless-all/app/src-tauri/src/commands.rs # openless-all/app/src-tauri/src/coordinator.rs # openless-all/app/src-tauri/src/lib.rs
PR Reviewer Guide 🔍(Review updated until commit e0f7ca2)Here are some key observations to aid the review process:
|
|
Persistent review updated to latest commit 5c92e87 |
|
Persistent review updated to latest commit 5c92e87 |
|
Persistent review updated to latest commit 5c92e87 |
|
Persistent review updated to latest commit 593cf76 |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 593cf766a2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (isFoundryAlias(status.activeModel)) { | ||
| setSelectedFoundryAlias(status.activeModel); | ||
| } |
There was a problem hiding this comment.
Avoid forcing selectedFoundryAlias to status.activeModel on every status refresh. When a user picks a different model in the dropdown but hasn't clicked “Prepare/Set default” yet, any subsequent refreshFoundryStatus() call resets their choice back to the currently active model, so the next action can run against the wrong model unexpectedly. This is user-visible and can invalidate local-ASR setup/testing flows.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0bb8d3ebcd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const choices: Array<[HotkeyMode, string]> = [ | ||
| ['toggle', t('settings.recording.modeToggle')], | ||
| ['hold', t('settings.recording.modeHold')], | ||
| ['doubleClick', t('settings.recording.modeDoubleClick')], |
There was a problem hiding this comment.
Adding 'doubleClick' to the recording mode picker makes users able to persist prefs.hotkey.mode = "doubleClick", but the backend hotkey state machine still only handles Toggle/Hold in coordinator::handle_pressed and only Hold in handle_released (src-tauri/src/coordinator.rs), so selecting this mode causes hotkey edges to do nothing and dictation cannot start/stop. This is a user-facing break of the core workflow whenever the new mode is chosen.
Useful? React with 👍 / 👎.
| if (message.includes('API Key')) return t('settings.providers.apiKeyMissing'); | ||
| if (message.includes('Endpoint')) return t('settings.providers.endpointMissing'); | ||
| if (message.includes('timeout') || message.includes('超时')) return t('settings.providers.requestTimeout'); | ||
| if (message.includes('timeout') || message.includes('瓒呮椂')) return t('settings.providers.requestTimeout'); |
There was a problem hiding this comment.
The timeout fallback check now matches '瓒呮椂' instead of '超时', so Chinese timeout messages no longer map to settings.providers.requestTimeout and will fall through to a generic failure message. This regresses provider error UX specifically for Chinese-language timeout responses.
Useful? React with 👍 / 👎.
|
Persistent review updated to latest commit 0bb8d3e |
|
Persistent review updated to latest commit 0bb8d3e |
…-asr # Conflicts: # openless-all/app/src-tauri/src/lib.rs # openless-all/app/src-tauri/src/types.rs # openless-all/app/src/lib/ipc.ts # openless-all/app/src/pages/Settings.tsx
There was a problem hiding this comment.
💡 Codex Review
https://github.com/appergb/openless/blob/0bb8d3ebcd737c5887eb0b52521c3ba034d16460/openless-all/app/src-tauri/src/commands.rs#L1028-L1030
After this change, the UI reads the recording-key label from hotkey.keys (getHotkeyBindingLabel), but sync_dictation_hotkey_legacy_fields only updates hotkey.trigger and leaves hotkey.keys untouched. When a user changes the dictation shortcut (for example from Right Ctrl to Right Alt or a custom combo), persisted keys can remain stale (ControlRight), so QA/Settings hotkey labels become incorrect even though backend behavior changed; this is a user-visible regression that can mislead users about which key starts/stops dictation.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (/^Key[A-Z]$/.test(code)) return true; | ||
| if (/^Digit[0-9]$/.test(code)) return true; | ||
| if (/^F([1-9]|1[0-9]|2[0-4])$/.test(code)) return true; | ||
| if (/^Numpad[0-9]$/.test(code)) return true; |
There was a problem hiding this comment.
This candidate filter now treats nearly all keyboard codes as fallback candidates (Key*, Digit*, F*, numpad, and many symbols), so the app forwards ordinary typing events to handle_window_hotkey_event on Windows. In the current backend path, fallback is disabled (explicit_fallback_available is false), so these extra events are immediately ignored but still incur IPC and "ignored" logging for normal keypresses, causing avoidable log noise and unnecessary runtime overhead while users type in app windows.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0bb8d3ebcd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const knownAsr = visibleAsrPresets.find(x => x.id === prefs.activeAsrProvider); | ||
| const asrId = knownAsr ? knownAsr.id : 'volcengine'; | ||
| setAsrProvider(asrId); |
There was a problem hiding this comment.
When running on non-Windows, this effect hides foundry-local-whisper and then unconditionally falls back the selected provider to volcengine if the active provider is not in visibleAsrPresets. If a user profile/vault still has Foundry active (for example after moving settings from Windows), the UI shows Volcengine while the backend remains on Foundry, so dictation continues to fail with the Windows-only provider error until the user manually re-selects another provider. Keep the unsupported active provider visible/disabled or explicitly switch the backend provider when applying this fallback.
Useful? React with 👍 / 👎.
|
Persistent review updated to latest commit 70455aa |
|
Persistent review updated to latest commit e0f7ca2 |
|
@claude 审查他的代码，严格审查，对他的这些逻辑性整理一下，检查还有哪些问题然后进行修复。 |
|
Claude finished @appergb's task in 10m 20s —— View job • 代码审查与修复完成已完成严格审查，整理所有逻辑问题并修复了其中两个确定的 Bug。 已修复问题Bug 1 —
|
| 问题描述 | 核查结论 |
|---|---|
schedule_foundry_local_asr_release 读错 keep_loaded 字段 |
代码第 3219 行已正确读 foundry_local_asr_keep_loaded_secs，第 4103 行有专项测试 |
| 转写失败时未 schedule release | 第 2565 行失败路径已调用 schedule_foundry_local_asr_release，已覆盖 |
cancel() 未传播到 runtime prepare cancel |
foundry_provider.rs:134 已有 self.runtime.request_cancel_prepare() |
LocalAsr.tsx 刷新时重置用户选择的模型 |
foundrySelectionDirty.current ref 已正确保护，用户选择后不会被刷新覆盖 |
剩余已知问题（不修复原因）
| 级别 | 位置 | 问题 | 建议 |
|---|---|---|---|
| P1 | foundry_provider.rs:241 |
转写出错时仍清空 PCM 缓冲 | 每次会话创建新 provider，buffer 不跨会话，实际无数据丢失 |
| P1 | coordinator.rs:2301 |
Windows 无条件视 foundry-local-whisper 为凭据就绪 | Foundry 是本地 provider，无凭据概念，可接受；错误会在转写时捕获并显示 |
| P2 | Settings.tsx:1094 |
非 Windows 时 UI 回退显示 volcengine，但 backend 仍是 foundry | 纯展示问题；只有用户主动点击 volcengine 选项才触发 setActiveAsrProvider，一般不发生 |
| P2 | windowHotkeyFallback.ts:5 |
候选键过广（所有字母/数字键），产生无效 IPC 调用 | explicit_fallback_available=false 时后端立即丢弃，不影响功能，仅多余 IPC 噪声 |
3 participants
User description
Summary
Verification
cargo check --manifest-path openless-all/app/src-tauri/Cargo.tomlnpm.cmd --prefix openless-all/app run buildcargo test --manifest-path openless-all/app/src-tauri/Cargo.toml foundry --lib --no-rungit diff --check upstream/main...HEADNot Run
PR Type
Enhancement
Description
Introduce Foundry Local Whisper ASR provider for Windows with model catalog, download, and transcription via Foundry SDK
Add Tauri commands and coordinator integration for local ASR sessions including prepare, cancel, release, and progress events
Extend hotkey binding system to support multi-key combos, double‑click mode, and a hotkey recorder UI
Surface Foundry model status, language hint selection, and keep‑loaded preferences in settings and local ASR management
Add shared WAV encoding module and adapt Whisper batch ASR to use it
Diagram Walkthrough
File Walkthrough
17 files
Foundry local ASR commands, provider switch release plan, testsIntegrate Foundry ASR into session life cycle with custom timeoutNew Foundry runtime managing model lifecycle, prepare, cancel, catalogNew Foundry Whisper ASR provider with PCM buffering and WAVtranscriptionFoundry model definitions, status payloads, constantsShared WAV encoding utilityExpand hotkey binding with multi‑key combos, double‑click mode,Foundry prefsHotkey recorder UI, Foundry provider visibility, local ASR hintsTypeScript types for new hotkey binding and Foundry preferencesExport Foundry modulesAdd WAV module to ASR module treeChinese i18n additions for Foundry and hotkeysEnglish i18n additions for Foundry and hotkeysJapanese i18n additions for Foundry and hotkeysKorean i18n additions for Foundry and hotkeysTraditional Chinese i18n additions for Foundry and hotkeysNew hotkey recorder state machine logic2 files
Default ASR provider set to Foundry on WindowsRegister new commands and Foundry runtime state1 files
Refactor Whisper batch ASR to use shared WAV encoding17 files