feat: Scalar API docs, community health files, Quickstart cards - #41
Merged
Conversation
… Docker Backend: - Replace Swagger UI with Scalar at /docs (scalar-fastapi) - Add OpenAI-compatible /v1/audio endpoints (openai_compat router) - Add TTS streaming endpoint (tts_stream router) - Add voice marketplace router (marketplace) - Update TTS backend registry Frontend: - Refine CaptureWidget, WaveformTimeline, App layout - CSS polish and index.css updates Community health: - SECURITY.md — vulnerability reporting policy - CODE_OF_CONDUCT.md — Contributor Covenant v2.1 - .github/FUNDING.yml — GitHub Sponsors - .github/ISSUE_TEMPLATE/ — bug report + feature request - .github/pull_request_template.md — PR checklist README: - Quickstart redesigned as 3-column progressive cards - Docker section updated with GHCR pull instructions - API Docs row added to service table Infra: - scalar-fastapi added to pyproject.toml + uv.lock - research/ added to .gitignore
Contributor
📝 WalkthroughWalkthroughAdds community and governance files; implements OpenAI-compatible HTTP audio APIs (TTS, STT, voices), a WebSocket streaming TTS endpoint, a local voice-profile marketplace, new TTS backend adapters and wiring, a compact “capture pill” UI with Tauri autostart/shortcut support, and assorted docs/config updates. ChangesCommunity & Repo Metadata
Audio APIs, Marketplace, TTS Backends, and Capture UI
Sequence DiagramsequenceDiagram
participant Client as Client (web / SDK)
participant API as FastAPI (/v1 & /ws/tts)
participant GPU as GPU pool / TTS backends
participant DBFS as DB & Filesystem
Client->>API: POST /v1/audio/speech or open WS /ws/tts (StreamTTSRequest)
API->>DBFS: Resolve voice/profile (ref audio, ref text, metadata)
API->>GPU: Submit generation job (text + description/ref + params)
GPU-->>API: Return waveform tensor
API->>DBFS: Read/write profile audio or publish bundle (marketplace)
API-->>Client: Stream encoded audio (HTTP) or chunked PCM16 frames (WebSocket) with start/done messages
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Review rate limit: 9/10 reviews remaining, refill in 6 minutes. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 9
🧹 Nitpick comments (1)
frontend/src/index.css (1)
2361-2378: ⚡ Quick winAdd reduced-motion handling for the new overlay animations.
The new fade/scale animations should be disabled for users with reduced-motion preferences.
Proposed fix
`@keyframes` capture-ov-panel { from { opacity: 0; transform: translateY(20px) scale(0.92); } to { opacity: 1; transform: translateY(0) scale(1); } } + +@media (prefers-reduced-motion: reduce) { + .capture-overlay, + .capture-overlay__panel { + animation: none !important; + } +}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@frontend/src/index.css` around lines 2361 - 2378, Add reduced-motion support by disabling the overlay animations when the user prefers reduced motion: wrap rules for .capture-overlay and .capture-overlay__panel (which currently use animation: capture-ov-fade and animation: capture-ov-panel) in a `@media` (prefers-reduced-motion: reduce) block and set animation: none; and remove any transform/transition by setting transform: none; and transition: none; (or ensure overflow/visibility stays the same) so these elements render statically. Leave keyframe definitions (capture-ov-fade, capture-ov-panel) intact but ensure they are not applied under the reduced-motion media query.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/api/routers/marketplace.py`:
- Around line 138-165: Current code materializes the entire uploaded bundle via
file.read() and uses zf.read(...) for each member which can OOM; instead enforce
a hard max upload size (check file size from the incoming UploadFile/stream
before reading) and open the zip via zipfile.ZipFile(io.BytesIO(...)) only after
validating size or, better, avoid reading all bytes by saving a streamed temp
file or using file.file to seek/stream; then replace zf.read(name) calls (e.g.,
where metadata = json.loads(zf.read("metadata.json")) and where you write ref
audio into VOICES_DIR using ref_audio_filename/profile_id) with
ZipFile.open(name) and copy to disk via shutil.copyfileobj(zf.open(name),
dst_file) (and for metadata stream into a small buffer or parse with json.load
on the opened file) so entries are streamed and per-entry size checks are
applied; apply the same fix pattern to the other import path that also uses
zf.read() for members.
In `@backend/api/routers/openai_compat.py`:
- Around line 149-194: The _encode_audio function currently advertises the
requested format (fmt) even when it falls back to WAV/PCM/OGG bytes; change the
fallbacks so the returned mime type matches the actual bytes produced (e.g.,
when mp3 fallback writes WAV, return "audio/wav"; when opus fallback writes WAV,
return "audio/wav"; when AAC fallback writes WAV, return "audio/wav"), and
ensure callers use the returned mime/type to derive the download filename
(instead of blindly using req.response_format or fmt) so saved filenames (e.g.,
"speech.{ext}") match the actual content.
- Around line 274-323: The create_transcription handler currently ignores the
model parameter by always calling get_active_asr_backend(); update it to honor
model by mapping model=="whisper-1" to the active backend and other values
(e.g., "whisperx", "faster-whisper", "mlx-whisper", "pytorch-whisper") to their
specific backends via a selector function (either add/get or use an existing
get_asr_backend_by_name/get_backend(name) API) and call that backend's
transcribe; validate the model string, raise HTTPException(400) for unknown
backends, and keep the thread-pool call (loop.run_in_executor) using the
selected backend.transcribe so behavior and word_timestamps logic remain
unchanged.
- Around line 224-250: The current profile-resolution block tries to treat
req.voice as a profile ID but only sets kw["voice"] on exception; when the DB
query returns no row valid engine preset names are dropped — fix by forwarding
non-profile voices: inside the try block after the existing "if row:" handling
add an explicit else branch that sets kw["voice"] = voice (so when get_db/SELECT
succeeds but returns no row we preserve engine-specific preset names); keep the
existing exception handler that also sets kw["voice"] to cover DB errors. Ensure
you reference voice, req.voice, _OPENAI_VOICE_ALIASES, kw and row when making
the change.
In `@backend/api/routers/tts_stream.py`:
- Around line 149-171: The start frame is sent using backend.sample_rate before
lazy-loading occurs, so move sending the websocket.send_json start message until
after _generate() runs and real sample rate is known: run the generation via
loop.run_in_executor(_gpu_pool, _generate) to obtain wav (wav_tensor), read the
actual sample rate from the returned wav (or backend.sample_rate after
generation), then send the "start" JSON (using sample_rate, channels, format,
engine) before streaming PCM chunks; update references to backend.sample_rate,
_generate, wav_tensor, and websocket.send_json accordingly so the start frame
reflects the real sample rate.
- Around line 119-147: The DB lookup for voice profiles currently only sets
kw["voice"] in the exception handler, so when the query returns no row the
requested non-profile voice is never passed through; update the logic in the
voice resolution block (the code that calls get_db(), queries "SELECT * FROM
voice_profiles WHERE id=?", inspects row, and populates kw) so that if row is
falsy (no profile found) you explicitly set kw["voice"] = voice before leaving
the try block; keep the existing behavior that if a row exists you populate
kw["ref_audio"], kw["ref_text"], kw["instruct"], etc., and preserve the
exception handler that also sets kw["voice"] = voice.
In `@frontend/src/App.jsx`:
- Around line 2066-2073: The backdrop onClick directly calls
setShowCapture(false) which unmounts CaptureWidget immediately and can abort
active sessions; change this to invoke a close callback that lets CaptureWidget
finish cleanup (e.g., call a provided onRequestClose or onClose prop) or waits
for a cleanup promise before toggling visibility. Update the capture overlay
handler to call a new close handler (referencing setShowCapture and the
CaptureWidget component) and implement/consume an onRequestClose/onClose
callback on CaptureWidget that performs recording/transcription cleanup and only
then resolves so the parent can call setShowCapture(false).
- Around line 118-124: The global hotkey handler inside the useEffect (function
h) is toggling setShowCapture even when the CaptureWidget is already open,
causing a race with the widget's own hotkey handler; fix by guarding the handler
so it does nothing when the overlay is open or when the event originates from
inside the widget: add an early return in h (e.g., if (showCapture) return; or
if (event.target.closest && event.target.closest('#capture-widget')) return;)
and include showCapture in the useEffect dependency array so the handler sees
the current state; keep the existing key detection logic and ensure the effect
still adds/removes the same listener.
In `@frontend/src/index.css`:
- Around line 2351-2355: The overlay's z-index (selector .capture-overlay) is
too low and can be under elements like .rail-label (z-index: 10000); update
.capture-overlay to use a higher stacking value (e.g., >10000 such as 10001) or
switch to a top-layer CSS variable and apply that to .capture-overlay so the
modal reliably appears above floating chrome layers; ensure you update the
.capture-overlay rule (position: fixed; inset: 0; z-index: ...) accordingly.
---
Nitpick comments:
In `@frontend/src/index.css`:
- Around line 2361-2378: Add reduced-motion support by disabling the overlay
animations when the user prefers reduced motion: wrap rules for .capture-overlay
and .capture-overlay__panel (which currently use animation: capture-ov-fade and
animation: capture-ov-panel) in a `@media` (prefers-reduced-motion: reduce) block
and set animation: none; and remove any transform/transition by setting
transform: none; and transition: none; (or ensure overflow/visibility stays the
same) so these elements render statically. Leave keyframe definitions
(capture-ov-fade, capture-ov-panel) intact but ensure they are not applied under
the reduced-motion media query.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 4408bf9a-4d41-4a17-bc5a-4d75147f97de
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (18)
.github/FUNDING.yml.github/ISSUE_TEMPLATE/bug_report.md.github/ISSUE_TEMPLATE/feature_request.md.github/pull_request_template.md.gitignoreCODE_OF_CONDUCT.mdREADME.mdSECURITY.mdbackend/api/routers/marketplace.pybackend/api/routers/openai_compat.pybackend/api/routers/tts_stream.pybackend/main.pybackend/services/tts_backend.pyfrontend/src/App.jsxfrontend/src/components/CaptureWidget.jsxfrontend/src/components/WaveformTimeline.jsxfrontend/src/index.csspyproject.toml
Comment on lines
+274
to
+323
| @router.post("/transcriptions") | ||
| async def create_transcription( | ||
| file: UploadFile = File(..., description="Audio file to transcribe"), | ||
| model: str = Form( | ||
| default="whisper-1", | ||
| description=( | ||
| "ASR model. Accepts 'whisper-1' (maps to active engine), or an " | ||
| "OmniVoice engine ID: whisperx, faster-whisper, mlx-whisper, pytorch-whisper." | ||
| ), | ||
| ), | ||
| language: Optional[str] = Form( | ||
| default=None, | ||
| description="Language of the input audio (ISO 639-1). Optional.", | ||
| ), | ||
| prompt: Optional[str] = Form( | ||
| default=None, | ||
| description="Optional text to guide the model's style or continue a previous segment.", | ||
| ), | ||
| response_format: str = Form( | ||
| default="json", | ||
| description="Output format: json, text, verbose_json, srt, vtt.", | ||
| ), | ||
| temperature: Optional[float] = Form( | ||
| default=None, | ||
| description="Sampling temperature (0–1). Not used by all backends.", | ||
| ), | ||
| ): | ||
| """Transcribe audio to text. Compatible with OpenAI's POST /v1/audio/transcriptions.""" | ||
| from services.asr_backend import get_active_asr_backend | ||
|
|
||
| # Write uploaded file to a temp location | ||
| suffix = os.path.splitext(file.filename or "audio.wav")[1] or ".wav" | ||
| try: | ||
| with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tmp: | ||
| content = await file.read() | ||
| tmp.write(content) | ||
| tmp_path = tmp.name | ||
| except Exception as e: | ||
| raise HTTPException(status_code=400, detail=f"Could not read audio file: {e}") | ||
|
|
||
| try: | ||
| backend = get_active_asr_backend() | ||
|
|
||
| # Run transcription in the thread pool to avoid blocking the event loop | ||
| loop = asyncio.get_event_loop() | ||
| word_ts = response_format == "verbose_json" | ||
| result = await loop.run_in_executor( | ||
| _gpu_pool, | ||
| lambda: backend.transcribe(tmp_path, word_timestamps=word_ts), | ||
| ) |
Contributor
There was a problem hiding this comment.
Honor the model parameter for transcription requests.
This endpoint documents model as a backend selector, but the implementation always calls get_active_asr_backend(). Any caller asking for whisperx, faster-whisper, etc. gets the current default instead, so the API contract here is silently wrong.
🧰 Tools
🪛 Ruff (0.15.12)
[warning] 276-276: Do not perform function call File in argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable
(B008)
[warning] 298-298: String contains ambiguous – (EN DASH). Did you mean - (HYPHEN-MINUS)?
(RUF001)
[warning] 311-311: Do not catch blind exception: Exception
(BLE001)
[warning] 312-312: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/api/routers/openai_compat.py` around lines 274 - 323, The
create_transcription handler currently ignores the model parameter by always
calling get_active_asr_backend(); update it to honor model by mapping
model=="whisper-1" to the active backend and other values (e.g., "whisperx",
"faster-whisper", "mlx-whisper", "pytorch-whisper") to their specific backends
via a selector function (either add/get or use an existing
get_asr_backend_by_name/get_backend(name) API) and call that backend's
transcribe; validate the model string, raise HTTPException(400) for unknown
backends, and keep the thread-pool call (loop.run_in_executor) using the
selected backend.transcribe so behavior and word_timestamps logic remain
unchanged.
…packaging dependencies and capture UI performance.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
docs/DESKTOP_RELEASE.md (2)
60-60:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winSetup download stream path in docs doesn’t match implemented route.
The doc uses
/setup/download/stream, while backend exposes/setup/download-stream. This mismatch will break consumers following the release plan.Also applies to: 152-152
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/DESKTOP_RELEASE.md` at line 60, Docs reference the wrong SSE route; update all occurrences of the path `/setup/download/stream` in the documentation to match the backend's implemented route `/setup/download-stream` (or, if you prefer to standardize on the dotted form, change the server route to `/setup/download/stream` instead). Locate and edit the doc strings that mention the SSE endpoint (instances of `/setup/download/stream`) and make them consistent with the backend route name (`/setup/download-stream`) so consumers and the implementation match.
31-33:⚠️ Potential issue | 🟡 Minor | ⚡ Quick win
/healthresponse contract is out of sync with backend.This section says
status == "healthy"withmodel_loaded/gpu_available, but backend/healthcurrently returns{"status":"ok","device":...}. Please align the doc or endpoint contract.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/DESKTOP_RELEASE.md` around lines 31 - 33, The docs mention the /health response using fields status == "healthy" and model_loaded/gpu_available but the backend currently returns {"status":"ok","device":...}; update either the docs or the backend to make them consistent: either change the DESKTOP_RELEASE.md health-check steps to expect status == "ok" and the actual device/fields the server returns (replace model_loaded/gpu_available with the backend's device field and its shape), or modify the backend /health endpoint to return the documented contract (status: "healthy", model_loaded: bool, gpu_available: bool); refer to the /health endpoint and the specific field names (status, model_loaded, gpu_available, device) when making the change.frontend/src-tauri/src/lib.rs (1)
75-82:⚠️ Potential issue | 🟠 MajorUse the new instance's argv to determine which window to focus.
This callback ignores
_argvand always picks the target from the already-running instance'spill_mode. When the app is started with--pill, clicking "Open OmniVoice Studio" (line 257) spawns a new process without the--pillflag, but the second-instance handler will still focuswidgetbecause the first instance'spill_modeis true. This prevents the Studio window from opening correctly.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@frontend/src-tauri/src/lib.rs` around lines 75 - 82, The second-instance handler passed to tauri_plugin_single_instance::init currently ignores the provided _argv and uses the existing pill_mode to pick target window; change the closure to inspect the incoming argv (the _argv parameter) for the "--pill" flag and set target = if argv contains "--pill" { "widget" } else { "main" } instead of using the existing pill_mode, then proceed to get_webview_window(target) and show/unminimize/set_focus as before so the new process's intent determines which window is focused.
♻️ Duplicate comments (1)
frontend/src/index.css (1)
2350-2356:⚠️ Potential issue | 🟡 MinorRaise the pill host above existing floating chrome layers.
z-index: 9999is still below.rail-labelat10000, so the capture pill can render under nav/tooltips instead of above them.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@frontend/src/index.css` around lines 2350 - 2356, The .capture-pill-host z-index (currently 9999) is lower than .rail-label (10000) so the pill can render underneath; update the .capture-pill-host selector to use a higher z-index (e.g., 10001 or 10010) so it stacks above .rail-label and other floating chrome layers while leaving other properties like position and pointer-events unchanged.
🧹 Nitpick comments (2)
backend/api/routers/setup/download.py (2)
104-110: ⚡ Quick winAdd
Retry-Afterto the 429 response.The cooldown response is good, but clients can’t reliably back off without a standard
Retry-Afterheader.Suggested diff
raise HTTPException( status_code=429, detail=( f"Model {req.repo_id!r} install failed recently. " f"Retry in {remaining}s or check your network." ), + headers={"Retry-After": str(max(1, remaining))}, )🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/api/routers/setup/download.py` around lines 104 - 110, The 429 HTTPException raised for a recent install cooldown should include a standard Retry-After header so clients can back off; modify the HTTPException call (the one using HTTPException(status_code=429, detail=...)) to pass headers={"Retry-After": str(remaining)} (or an RFC-compliant timestamp if you prefer) so the response includes Retry-After (use the existing remaining variable and keep the same detail message referencing req.repo_id).
192-194: ⚡ Quick winPreserve traceback on install failure logs.
Using
logger.infohere hides stack traces for unexpected failures, which makes root-cause debugging harder.Suggested diff
- logger.info("model install failed for %s: %s", req.repo_id, e) + logger.exception("model install failed for %s", req.repo_id)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/api/routers/setup/download.py` around lines 192 - 194, The current catch block logs install failures with logger.info which hides the traceback; change the log to include the exception traceback (e.g., use logger.exception or logger.error(..., exc_info=True)) so the full stack trace for the exception variable e is preserved while keeping the existing behavior of setting _install_cooldowns[req.repo_id] = _time_fail.time(); locate the code that references logger, _install_cooldowns and req.repo_id in the install failure path and replace the logger.info call with a logger.exception/ logger.error call that includes the exception details.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/api/routers/capture_ws.py`:
- Line 40: The MIN_BUFFER_BYTES constant is being used for both partials and
final transcription, causing utterances shorter than ~2s to skip calling
_transcribe_buffer_full() on EOF and return a synthetic empty final; introduce a
separate threshold (e.g., MIN_FINAL_BYTES or MIN_FINAL_BUFFER_BYTES) and use
MIN_BUFFER_BYTES for partials while gating the final-transcription path (the
EOF/finish branch that currently checks MIN_BUFFER_BYTES) against the new
MIN_FINAL_BYTES so that _transcribe_buffer_full() is invoked for final sends
even for shorter utterances; update all checks that currently reference
MIN_BUFFER_BYTES for final/EOF handling to use the new constant and leave
partial buffering logic (and variable name MIN_BUFFER_BYTES) unchanged.
- Around line 301-310: In the except block that handles ffmpeg conversion
failures, only return tmp_in.name when the buffered input is actually a WebM
container: detect WebM by opening tmp_in, reading the initial bytes and checking
for the EBML magic sequence (b'\x1A\x45\xDF\xA3') or another reliable WebM
signature, rewind/seek the file as needed, and only log "Falling back to raw
WebM input for ASR" and return tmp_in.name when that check passes; otherwise,
clean up tmp_in/tmp_out as appropriate and return None (the original fallback)
so PCM inputs are not misclassified. Ensure you update the except block around
the ffmpeg conversion (where tmp_in and tmp_out are referenced) and preserve
existing OSError handling for os.unlink.
In `@frontend/src-tauri/src/commands.rs`:
- Around line 392-441: The exe_str is injected verbatim into the plist XML and
the Linux desktop entry, so escape/serialize it before writing: XML-escape
special chars (&, <, >, ", ') for the plist creation block (the plist
variable/format! call that inserts {exe_str}), and properly escape or
shell-quote the Exec value for the Linux desktop entry (the desktop
variable/format! that contains Exec="{exe_str}" --pill). Add small helpers like
escape_for_plist(exe: &str) and escape_for_desktop(exe: &str) and call them
where exe_str is interpolated (and adjust the Windows reg value if needed) using
an XML-escape library for the plist and a safe shell-quoting or
backslash-escaping approach for the .desktop Exec line.
In `@frontend/src-tauri/src/lib.rs`:
- Around line 264-266: The tray "dictate" handler currently only emits a start
event ("tray-dictate") so the CaptureWidget never receives the matching stop;
update the "dictate" match arm in lib.rs to toggle by checking the current
recording state or by emitting "tray-dictate-stop" when recording —
specifically, change the block that calls app.emit("tray-dictate", ()) to emit
"tray-dictate-stop" if CaptureWidget (or the app state) reports recording,
otherwise emit "tray-dictate"; ensure the emitted event names exactly match the
CaptureWidget listeners ("tray-dictate" and "tray-dictate-stop").
In `@frontend/src-tauri/tauri.conf.json`:
- Line 44: The Content Security Policy string under the "csp" key currently
allows broad localhost access via http/ws://localhost:* and http://127.0.0.1:*,
which is too permissive; update the "csp" value so connect-src, media-src, and
img-src only allow the specific backend/dev ports OmniVoice actually uses
(replace the wildcard ports with the exact ports or named placeholders for your
app's backend/dev server), e.g., change occurrences of http://localhost:*
ws://localhost:* and http://127.0.0.1:* ws://127.0.0.1:* to explicit origins
like http://localhost:PORT and ws://localhost:PORT while keeping required tokens
(ipc://localhost, blob:, data:, and style-src 'unsafe-inline' etc.) intact.
In `@frontend/src/components/CaptureWidget.css`:
- Around line 15-29: In the .capture-pill block remove the unexpected empty
line(s) between property declarations (fixing declaration-empty-line-before) and
normalize the font-family value to use double quotes for named families and no
quotes for system/generic families (e.g. change font-family: 'Inter Variable',
'Inter', -apple-system, sans-serif; to font-family: "Inter Variable", "Inter",
-apple-system, sans-serif;), then ensure the properties remain consecutively
listed with proper semicolons and no extra blank lines so Stylelint passes.
In `@frontend/src/components/CaptureWidget.jsx`:
- Around line 118-149: The current logic always treats data.text as "pasted" and
auto-dismisses even when navigator.clipboard.writeText or the Tauri
invoke('simulate_paste') call fail; change it to detect success before updating
UI and hiding the window: in the block that handles data.text, perform
navigator.clipboard.writeText and then invoke('simulate_paste') inside try/catch
but record a boolean like pasteSucceeded; only run the auto-dismiss code
(setState('idle')/setTranscript('')/setDuration(0)/getCurrentWindow().hide()/onDismiss())
when pasteSucceeded is true; if paste fails, do not hide or clear the transcript
(or set a non-pasted state) so the visible transcript remains for the user.
Reference functions/idents: navigator.clipboard.writeText,
invoke('simulate_paste'), pasteSucceeded flag, setTimeout callback, setState,
setTranscript, setDuration, getCurrentWindow, onDismiss.
- Around line 57-79: The Tauri listeners inside the useEffect are re-created on
every state change, creating a gap where a quick tray-dictate-stop can be
missed; change the effect so listeners are mounted once (no state dep) and read
the current state via a ref: create a stateRef = useRef(state), update
stateRef.current in a separate small effect whenever state changes, then in the
listener callbacks check stateRef.current before calling startRecording or
stopRecording; keep the existing unlistenStart/unlistenStop cleanup but ensure
the mounting effect has an empty deps array so listeners remain stable during
async import/listen.
---
Outside diff comments:
In `@docs/DESKTOP_RELEASE.md`:
- Line 60: Docs reference the wrong SSE route; update all occurrences of the
path `/setup/download/stream` in the documentation to match the backend's
implemented route `/setup/download-stream` (or, if you prefer to standardize on
the dotted form, change the server route to `/setup/download/stream` instead).
Locate and edit the doc strings that mention the SSE endpoint (instances of
`/setup/download/stream`) and make them consistent with the backend route name
(`/setup/download-stream`) so consumers and the implementation match.
- Around line 31-33: The docs mention the /health response using fields status
== "healthy" and model_loaded/gpu_available but the backend currently returns
{"status":"ok","device":...}; update either the docs or the backend to make them
consistent: either change the DESKTOP_RELEASE.md health-check steps to expect
status == "ok" and the actual device/fields the server returns (replace
model_loaded/gpu_available with the backend's device field and its shape), or
modify the backend /health endpoint to return the documented contract (status:
"healthy", model_loaded: bool, gpu_available: bool); refer to the /health
endpoint and the specific field names (status, model_loaded, gpu_available,
device) when making the change.
In `@frontend/src-tauri/src/lib.rs`:
- Around line 75-82: The second-instance handler passed to
tauri_plugin_single_instance::init currently ignores the provided _argv and uses
the existing pill_mode to pick target window; change the closure to inspect the
incoming argv (the _argv parameter) for the "--pill" flag and set target = if
argv contains "--pill" { "widget" } else { "main" } instead of using the
existing pill_mode, then proceed to get_webview_window(target) and
show/unminimize/set_focus as before so the new process's intent determines which
window is focused.
---
Duplicate comments:
In `@frontend/src/index.css`:
- Around line 2350-2356: The .capture-pill-host z-index (currently 9999) is
lower than .rail-label (10000) so the pill can render underneath; update the
.capture-pill-host selector to use a higher z-index (e.g., 10001 or 10010) so it
stacks above .rail-label and other floating chrome layers while leaving other
properties like position and pointer-events unchanged.
---
Nitpick comments:
In `@backend/api/routers/setup/download.py`:
- Around line 104-110: The 429 HTTPException raised for a recent install
cooldown should include a standard Retry-After header so clients can back off;
modify the HTTPException call (the one using HTTPException(status_code=429,
detail=...)) to pass headers={"Retry-After": str(remaining)} (or an
RFC-compliant timestamp if you prefer) so the response includes Retry-After (use
the existing remaining variable and keep the same detail message referencing
req.repo_id).
- Around line 192-194: The current catch block logs install failures with
logger.info which hides the traceback; change the log to include the exception
traceback (e.g., use logger.exception or logger.error(..., exc_info=True)) so
the full stack trace for the exception variable e is preserved while keeping the
existing behavior of setting _install_cooldowns[req.repo_id] =
_time_fail.time(); locate the code that references logger, _install_cooldowns
and req.repo_id in the install failure path and replace the logger.info call
with a logger.exception/ logger.error call that includes the exception details.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: aadbfc74-5b5d-41f0-8e44-55f45ce371c6
⛔ Files ignored due to path filters (1)
frontend/src-tauri/Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (22)
backend/api/routers/capture_ws.pybackend/api/routers/setup/download.pybackend/main.pybackend/services/ffmpeg_utils.pybackend/utils/hf_progress.pydocs/DESKTOP_RELEASE.mddocs/STRUCTURE.mddocs/desktop-build.mdfrontend/src-tauri/Cargo.tomlfrontend/src-tauri/src/commands.rsfrontend/src-tauri/src/lib.rsfrontend/src-tauri/tauri.conf.jsonfrontend/src/App.jsxfrontend/src/components/CaptureWidget.cssfrontend/src/components/CaptureWidget.jsxfrontend/src/components/FloatingPill.cssfrontend/src/components/FloatingPill.jsxfrontend/src/components/ReadinessChecklist.cssfrontend/src/components/ReadinessChecklist.jsxfrontend/src/index.cssfrontend/src/main-app.jsxfrontend/src/store/pillSlice.ts
💤 Files with no reviewable changes (2)
- docs/STRUCTURE.md
- backend/utils/hf_progress.py
✅ Files skipped from review due to trivial changes (8)
- frontend/src/components/ReadinessChecklist.css
- frontend/src/components/FloatingPill.jsx
- frontend/src/components/ReadinessChecklist.jsx
- frontend/src/components/FloatingPill.css
- docs/desktop-build.md
- frontend/src-tauri/Cargo.toml
- frontend/src/store/pillSlice.ts
- frontend/src/App.jsx
Comment on lines
+301
to
+310
| except Exception as e: | ||
| logger.debug("ffmpeg conversion failed: %s", e) | ||
| try: | ||
| os.unlink(tmp_in.name) | ||
| os.unlink(tmp_out.name) | ||
| except OSError: | ||
| pass | ||
| # Fallback: return the raw WebM — ASR backends (MLX Whisper, | ||
| # WhisperX) can decode WebM/Opus containers natively. | ||
| logger.debug("Falling back to raw WebM input for ASR") | ||
| return tmp_in.name |
Contributor
There was a problem hiding this comment.
Only fall back to raw WebM when the buffered input is actually WebM.
This helper still advertises PCM/WebM support, but on ffmpeg failure it now always returns tmp_in.name as if the buffer were a WebM container. If a PCM client hits this path, the backend will hand mislabeled raw PCM bytes to ASR and fail harder than the old None fallback.
Suggested guard
except Exception as e:
logger.debug("ffmpeg conversion failed: %s", e)
try:
os.unlink(tmp_out.name)
except OSError:
pass
- # Fallback: return the raw WebM — ASR backends (MLX Whisper,
- # WhisperX) can decode WebM/Opus containers natively.
- logger.debug("Falling back to raw WebM input for ASR")
- return tmp_in.name
+ # Only fall back when the input is actually an EBML/WebM container.
+ if blob.startswith(b"\x1A\x45\xDF\xA3"):
+ logger.debug("Falling back to raw WebM input for ASR")
+ return tmp_in.name
+ try:
+ os.unlink(tmp_in.name)
+ except OSError:
+ pass
+ return None🧰 Tools
🪛 Ruff (0.15.12)
[warning] 301-301: Do not catch blind exception: Exception
(BLE001)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/api/routers/capture_ws.py` around lines 301 - 310, In the except
block that handles ffmpeg conversion failures, only return tmp_in.name when the
buffered input is actually a WebM container: detect WebM by opening tmp_in,
reading the initial bytes and checking for the EBML magic sequence
(b'\x1A\x45\xDF\xA3') or another reliable WebM signature, rewind/seek the file
as needed, and only log "Falling back to raw WebM input for ASR" and return
tmp_in.name when that check passes; otherwise, clean up tmp_in/tmp_out as
appropriate and return None (the original fallback) so PCM inputs are not
misclassified. Ensure you update the except block around the ffmpeg conversion
(where tmp_in and tmp_out are referenced) and preserve existing OSError handling
for os.unlink.
| ], | ||
| "security": { | ||
| "csp": "default-src 'self' 'unsafe-inline' 'unsafe-eval'; connect-src 'self' http://localhost:* ws://localhost:* http://127.0.0.1:* ws://127.0.0.1:* blob: data:; media-src 'self' blob: data: http://localhost:* http://127.0.0.1:* asset: https://asset.localhost; img-src 'self' blob: data: asset: https://asset.localhost https://fonts.gstatic.com; font-src 'self' data: https://fonts.googleapis.com https://fonts.gstatic.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com;", | ||
| "csp": "default-src 'self' 'unsafe-inline' 'unsafe-eval'; connect-src 'self' ipc://localhost http://localhost:* ws://localhost:* http://127.0.0.1:* ws://127.0.0.1:* blob: data:; media-src 'self' blob: data: http://localhost:* http://127.0.0.1:* asset: https://asset.localhost; img-src 'self' blob: data: asset: https://asset.localhost http://localhost:* http://127.0.0.1:* https://fonts.gstatic.com; font-src 'self' data: https://fonts.googleapis.com https://fonts.gstatic.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com;", |
Contributor
There was a problem hiding this comment.
Tighten the localhost CSP to the ports OmniVoice actually owns.
Allowing http/ws://localhost:* and 127.0.0.1:* lets any renderer injection probe arbitrary local services on the machine. This shell appears to need only the app's own backend/dev ports, so keeping connect-src, media-src, and img-src scoped to those exact ports would preserve least privilege.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@frontend/src-tauri/tauri.conf.json` at line 44, The Content Security Policy
string under the "csp" key currently allows broad localhost access via
http/ws://localhost:* and http://127.0.0.1:*, which is too permissive; update
the "csp" value so connect-src, media-src, and img-src only allow the specific
backend/dev ports OmniVoice actually uses (replace the wildcard ports with the
exact ports or named placeholders for your app's backend/dev server), e.g.,
change occurrences of http://localhost:* ws://localhost:* and http://127.0.0.1:*
ws://127.0.0.1:* to explicit origins like http://localhost:PORT and
ws://localhost:PORT while keeping required tokens (ipc://localhost, blob:,
data:, and style-src 'unsafe-inline' etc.) intact.
Comment on lines
+57
to
+79
| // ── Hold-to-talk: listen for tray-dictate (start) and tray-dictate-stop (release) ── | ||
| useEffect(() => { | ||
| let unlistenStart, unlistenStop; | ||
| (async () => { | ||
| try { | ||
| const { listen } = await import('@tauri-apps/api/event'); | ||
| unlistenStart = await listen('tray-dictate', () => { | ||
| if (state === 'idle' || state === 'done' || state === 'error') { | ||
| startRecording(); | ||
| } | ||
| }); | ||
| unlistenStop = await listen('tray-dictate-stop', () => { | ||
| if (state === 'recording') { | ||
| stopRecording(); | ||
| } | ||
| }); | ||
| } catch { /* not in Tauri */ } | ||
| })(); | ||
| return () => { | ||
| if (unlistenStart) unlistenStart(); | ||
| if (unlistenStop) unlistenStop(); | ||
| }; | ||
| }, [state]); |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify the tray listeners are tied to state changes and re-created asynchronously.
rg -n -C3 'useEffect|tray-dictate|tray-dictate-stop|\[state\]' frontend/src/components/CaptureWidget.jsxRepository: debpalash/OmniVoice-Studio
Length of output: 1943
🏁 Script executed:
cat -n frontend/src/components/CaptureWidget.jsx | sed -n '100,160p'Repository: debpalash/OmniVoice-Studio
Length of output: 2436
Stabilize the Tauri listeners; re-subscribing on every state change can miss the release event.
This effect tears down both listeners whenever state changes, then re-attaches them asynchronously. On a quick hold-to-talk cycle, the tray-dictate-stop event can arrive during the gap between cleanup (synchronous) and re-registration (async import/listen), leaving the widget stuck in the recording state. Use useRef to track state instead, so the listeners remain stable and read the current state from refs.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@frontend/src/components/CaptureWidget.jsx` around lines 57 - 79, The Tauri
listeners inside the useEffect are re-created on every state change, creating a
gap where a quick tray-dictate-stop can be missed; change the effect so
listeners are mounted once (no state dep) and read the current state via a ref:
create a stateRef = useRef(state), update stateRef.current in a separate small
effect whenever state changes, then in the listener callbacks check
stateRef.current before calling startRecording or stopRecording; keep the
existing unlistenStart/unlistenStop cleanup but ensure the mounting effect has
an empty deps array so listeners remain stable during async import/listen.
Comment on lines
+118
to
+149
| if (data.text) { | ||
| try { | ||
| await navigator.clipboard.writeText(data.text); | ||
| setCopied(true); | ||
| try { | ||
| const { invoke } = await import('@tauri-apps/api/core'); | ||
| await invoke('simulate_paste'); | ||
| toast.success('Pasted into active app', { duration: 2000 }); | ||
| } catch { | ||
| toast.success('Copied to clipboard — paste with ⌘V', { duration: 2000 }); | ||
| } | ||
|
|
||
| // Auto-dismiss the floating widget after 2.5 seconds so it gets out of the way | ||
| setTimeout(async () => { | ||
| setState('idle'); | ||
| setTranscript(''); | ||
| setDuration(0); | ||
| setCopied(false); | ||
| try { | ||
| const { getCurrentWindow } = await import('@tauri-apps/api/window'); | ||
| await getCurrentWindow().hide(); | ||
| } catch { /* not in Tauri */ } | ||
| }, 2500); | ||
|
|
||
| } catch { /* clipboard API may fail in some contexts */ } | ||
| } catch { /* not in Tauri */ } | ||
| } catch { /* clipboard API may fail */ } | ||
|
|
||
| // Auto-dismiss after 1.5s | ||
| setTimeout(async () => { | ||
| setState('idle'); | ||
| setTranscript(''); | ||
| setDuration(0); | ||
| try { | ||
| const { getCurrentWindow } = await import('@tauri-apps/api/window'); | ||
| await getCurrentWindow().hide(); | ||
| } catch { /* not in Tauri */ } | ||
| if (onDismiss) onDismiss(); | ||
| }, 1500); | ||
| } else { | ||
| // No speech — auto-dismiss after 2.5s | ||
| setTimeout(async () => { | ||
| setState('idle'); | ||
| setTranscript(''); | ||
| setDuration(0); | ||
| try { | ||
| const { getCurrentWindow } = await import('@tauri-apps/api/window'); | ||
| await getCurrentWindow().hide(); | ||
| } catch { /* not in Tauri */ } | ||
| if (onDismiss) onDismiss(); | ||
| }, 2500); |
Contributor
There was a problem hiding this comment.
Don't auto-dismiss as "Pasted" when clipboard/paste failed.
Both failure paths are swallowed here, but the success branch still shows done, labels the result as "Pasted", and hides the pill. In web mode or any denied-clipboard case, the user loses the only visible transcript even though nothing was inserted.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@frontend/src/components/CaptureWidget.jsx` around lines 118 - 149, The
current logic always treats data.text as "pasted" and auto-dismisses even when
navigator.clipboard.writeText or the Tauri invoke('simulate_paste') call fail;
change it to detect success before updating UI and hiding the window: in the
block that handles data.text, perform navigator.clipboard.writeText and then
invoke('simulate_paste') inside try/catch but record a boolean like
pasteSucceeded; only run the auto-dismiss code
(setState('idle')/setTranscript('')/setDuration(0)/getCurrentWindow().hide()/onDismiss())
when pasteSucceeded is true; if paste fails, do not hide or clear the transcript
(or set a non-pasted state) so the visible transcript remains for the user.
Reference functions/idents: navigator.clipboard.writeText,
invoke('simulate_paste'), pasteSucceeded flag, setTimeout callback, setState,
setTranscript, setDuration, getCurrentWindow, onDismiss.
Backend: - marketplace: stream zip entries via ZipFile.open()/copyfileobj, add 100MB upload cap, fix raise-from exception chaining (OOM prevention) - openai_compat: _encode_audio returns actual file ext so Content-Disposition matches real format; forward non-profile voices when DB row not found - tts_stream: send 'start' frame after generation so sample_rate is real; forward non-profile voices on DB miss - capture_ws: split MIN_BUFFER_BYTES into separate partial/final thresholds so short utterances (<2s) still get transcribed Frontend (Tauri): - lib.rs: tray 'dictate' now toggles start/stop based on widget visibility - commands.rs: XML-escape exe path in LaunchAgent plist, shell-quote in .desktop Exec line to prevent injection from special-char paths - CaptureWidget.css: fix Stylelint violations (empty lines, font-family quotes)
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
♻️ Duplicate comments (4)
frontend/src/components/CaptureWidget.css (1)
23-23:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winRemove quotes around
Interto satisfy StylelintLine 23 still violates
font-family-name-quotes; keep quotes for"Inter Variable"but remove them forInter.Proposed fix
- font-family: "Inter Variable", "Inter", -apple-system, sans-serif; + font-family: "Inter Variable", Inter, -apple-system, sans-serif;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@frontend/src/components/CaptureWidget.css` at line 23, Update the font-family declaration in CaptureWidget.css so it satisfies Stylelint by keeping quotes only around "Inter Variable" and removing the quotes around Inter; specifically edit the font-family line currently set to `font-family: "Inter Variable", "Inter", -apple-system, sans-serif;` to use unquoted Inter (e.g., `font-family: "Inter Variable", Inter, -apple-system, sans-serif;`) so the rule `font-family-name-quotes` is not violated.backend/api/routers/capture_ws.py (1)
305-314:⚠️ Potential issue | 🟠 Major | ⚡ Quick winGuard raw fallback by container signature before returning
tmp_in.name.On ffmpeg failure, Line 314 currently returns
tmp_in.nameunconditionally. For PCM inputs, that path mislabels raw bytes as WebM and can make ASR fail harder. Only fall back when the buffered input is actually EBML/WebM; otherwise clean up and returnNone.Proposed fix
except Exception as e: logger.debug("ffmpeg conversion failed: %s", e) try: os.unlink(tmp_out.name) except OSError: pass - # Fallback: return the raw WebM — ASR backends (MLX Whisper, - # WhisperX) can decode WebM/Opus containers natively. - logger.debug("Falling back to raw WebM input for ASR") - return tmp_in.name + # Fallback only if input is actually EBML/WebM. + if blob.startswith(b"\x1A\x45\xDF\xA3"): + logger.debug("Falling back to raw WebM input for ASR") + return tmp_in.name + try: + os.unlink(tmp_in.name) + except OSError: + pass + return None#!/bin/bash # Verify fallback behavior and whether a WebM signature guard exists. rg -n -C3 'ffmpeg conversion failed|Falling back to raw WebM input for ASR|return tmp_in.name|blob\.startswith\(b"\\x1A\\x45\\xDF\\xA3"\)' backend/api/routers/capture_ws.py🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/api/routers/capture_ws.py` around lines 305 - 314, When ffmpeg conversion in capture_ws (the except block around tmp_in/tmp_out) fails, don't unconditionally return tmp_in.name; first check the buffered input's container signature by reading the start of the tmp_in file and confirming it begins with the EBML/WebM magic bytes (b"\x1A\x45\xDF\xA3") before falling back. If the signature matches, log the fallback and return tmp_in.name; otherwise remove/close tmp_in (clean up) and return None. Use tmp_in.seek(0) then read a small prefix and tmp_in_blob.startswith(b"\x1A\x45\xDF\xA3") to decide; keep the existing logger.debug messages and the tmp_out unlink logic intact.backend/api/routers/openai_compat.py (1)
280-285:⚠️ Potential issue | 🟠 Major | ⚡ Quick win
modelis documented here but never used to choose the ASR backend.This handler always calls
get_active_asr_backend(), so requests forwhisperx,faster-whisper, etc. silently run on the default backend instead of the requested one.Also applies to: 305-325
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/api/routers/openai_compat.py` around lines 280 - 285, The request handler documents a "model" Form parameter but always ignores it by calling get_active_asr_backend(); change the handler to use the model value to select the ASR backend: when model == "whisper-1" keep the existing get_active_asr_backend() call, otherwise resolve the requested backend by mapping known IDs (e.g., "whisperx", "faster-whisper", "mlx-whisper", "pytorch-whisper") to their corresponding backend implementation or by calling a resolver like get_asr_backend_by_id(model) and use that backend for processing; update the same logic in the other affected handler block (around the 305-325 region) so incoming model values actually determine which ASR backend is used and return an error if the requested backend is not available.backend/api/routers/marketplace.py (1)
141-150:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftBundle handling is still RAM-bound and only checks compressed size.
await file.read()andf.read()still materialize the entire archive before validation, and/installhas no bundle-size check at all. A highly compressed bundle can also expand to oversized audio because extraction never checksZipInfo.file_sizebeforecopyfileobj.Also applies to: 169-182, 352-357, 371-381
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/api/routers/marketplace.py` around lines 141 - 150, The code currently materializes the entire upload via await file.read() (variable content) and opens zipfile.ZipFile(io.BytesIO(content)) which allows very large compressed-to-expanded attacks and missing checks in the /install handler and the other similar blocks; change to stream-read the incoming UploadFile in fixed-size chunks and early-reject when the cumulative compressed bytes exceed MAX_BUNDLE_BYTES (do not call await file.read() to completion), only construct an in-memory buffer (io.BytesIO) after the compressed size is validated, and when extracting use ZipFile.infolist() / ZipInfo.file_size to validate each entry against a MAX_EXPANDED_BYTES per-file (and cumulative expanded size) before calling ZipFile.open() + shutil.copyfileobj, enforcing limits and raising HTTPException 413 on breach; apply the same streaming+ZipInfo size checks for the other occurrences that currently use content/zf/copyfileobj (the blocks at the other mentioned locations and the /install endpoint).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/api/routers/marketplace.py`:
- Around line 348-350: Reject filenames that escape the marketplace directory by
resolving the candidate path and ensuring it is inside MARKETPLACE_DIR before
touching the filesystem: compute base = MARKETPLACE_DIR.resolve() and candidate
= (MARKETPLACE_DIR / filename).resolve() (or use candidate.relative_to(base) in
a try/except), and if candidate is not under base raise HTTPException(404,
detail=...) instead of directly using bundle_path; apply the same check to the
other occurrence handling filenames in the same module (the block around the
later bundle handling).
- Around line 161-162: The json.load(mf) call when reading "metadata.json" can
raise JSONDecodeError or UnicodeDecodeError and should be treated as a client
error; catch json.JSONDecodeError and UnicodeDecodeError around the
zf.open("metadata.json") block (both occurrences where metadata = json.load(mf)
is used) and convert them to an HTTP 400 response (raise
HTTPException(status_code=400, detail="Invalid metadata.json" or similar) so
malformed or non-UTF-8 metadata is reported as bad input rather than an internal
server error). Ensure you reference the same error handling in both code paths
that use zf and metadata.
In `@backend/api/routers/openai_compat.py`:
- Around line 329-387: The verbose_json/srt/vtt branches only iterate the local
segments list, so when the backend returns transcription in chunks you lose
timestamps; normalize chunks into segments right after computing full_text by
setting segments = segments if segments else chunks (and then recompute duration
using the last entry from segments if needed) so VerboseTranscriptionResponse,
the srt loop (uses _format_ts_srt), and vtt loop (uses _format_ts_vtt) operate
on the fallback data; ensure any uses of enumerate(segments) and the duration
calculation reference the normalized segments variable.
- Around line 307-313: The current code buffers the whole upload via await
file.read() before writing to disk; change to stream the upload in chunks into
the NamedTemporaryFile to avoid high memory use: inside the with
tempfile.NamedTemporaryFile(...) as tmp block, loop reading fixed-size chunks
(e.g., chunk = await file.read(8192)) until empty and write each chunk to tmp
(optionally track total bytes and raise an error if a configured max size is
exceeded). Keep the existing suffix/tmp_path logic and ensure tmp is
flushed/closed by the context manager; reference the variables file, tmp,
tmp_path and the tempfile.NamedTemporaryFile usage to locate where to replace
the single await file.read() call.
In `@backend/api/routers/tts_stream.py`:
- Line 35: Guard against zero/negative chunk sizes by validating
OMNIVOICE_STREAM_CHUNK when initializing CHUNK_SAMPLES: parse the env var,
convert to int, and if value <= 0 fallback to the default (4800) or a minimum
positive value, e.g., max(1, parsed) so CHUNK_SAMPLES is always > 0; also ensure
the send loop that advances sent_samples (the websocket streaming loop
referencing sent_samples and CHUNK_SAMPLES) will always make progress when
CHUNK_SAMPLES is used.
In `@frontend/src-tauri/src/commands.rs`:
- Around line 473-478: The registry-delete Command::new(...) call currently
ignores both the Result and the ExitStatus so failures are swallowed; capture
the result of .status(), handle the Err from process spawn and non-success exit
statuses from the returned ExitStatus, log an error (e.g., via log::error!) with
the command, status and any stderr info, and return an Err from the surrounding
function instead of unconditionally returning Ok(()). Locate the
Command::new("reg") invocation in commands.rs and replace the ignored call with
error-handling that maps process::ExitStatus failures into a meaningful error
(using your crate's error type/path) so callers see failure when autostart
deletion fails.
---
Duplicate comments:
In `@backend/api/routers/capture_ws.py`:
- Around line 305-314: When ffmpeg conversion in capture_ws (the except block
around tmp_in/tmp_out) fails, don't unconditionally return tmp_in.name; first
check the buffered input's container signature by reading the start of the
tmp_in file and confirming it begins with the EBML/WebM magic bytes
(b"\x1A\x45\xDF\xA3") before falling back. If the signature matches, log the
fallback and return tmp_in.name; otherwise remove/close tmp_in (clean up) and
return None. Use tmp_in.seek(0) then read a small prefix and
tmp_in_blob.startswith(b"\x1A\x45\xDF\xA3") to decide; keep the existing
logger.debug messages and the tmp_out unlink logic intact.
In `@backend/api/routers/marketplace.py`:
- Around line 141-150: The code currently materializes the entire upload via
await file.read() (variable content) and opens
zipfile.ZipFile(io.BytesIO(content)) which allows very large
compressed-to-expanded attacks and missing checks in the /install handler and
the other similar blocks; change to stream-read the incoming UploadFile in
fixed-size chunks and early-reject when the cumulative compressed bytes exceed
MAX_BUNDLE_BYTES (do not call await file.read() to completion), only construct
an in-memory buffer (io.BytesIO) after the compressed size is validated, and
when extracting use ZipFile.infolist() / ZipInfo.file_size to validate each
entry against a MAX_EXPANDED_BYTES per-file (and cumulative expanded size)
before calling ZipFile.open() + shutil.copyfileobj, enforcing limits and raising
HTTPException 413 on breach; apply the same streaming+ZipInfo size checks for
the other occurrences that currently use content/zf/copyfileobj (the blocks at
the other mentioned locations and the /install endpoint).
In `@backend/api/routers/openai_compat.py`:
- Around line 280-285: The request handler documents a "model" Form parameter
but always ignores it by calling get_active_asr_backend(); change the handler to
use the model value to select the ASR backend: when model == "whisper-1" keep
the existing get_active_asr_backend() call, otherwise resolve the requested
backend by mapping known IDs (e.g., "whisperx", "faster-whisper", "mlx-whisper",
"pytorch-whisper") to their corresponding backend implementation or by calling a
resolver like get_asr_backend_by_id(model) and use that backend for processing;
update the same logic in the other affected handler block (around the 305-325
region) so incoming model values actually determine which ASR backend is used
and return an error if the requested backend is not available.
In `@frontend/src/components/CaptureWidget.css`:
- Line 23: Update the font-family declaration in CaptureWidget.css so it
satisfies Stylelint by keeping quotes only around "Inter Variable" and removing
the quotes around Inter; specifically edit the font-family line currently set to
`font-family: "Inter Variable", "Inter", -apple-system, sans-serif;` to use
unquoted Inter (e.g., `font-family: "Inter Variable", Inter, -apple-system,
sans-serif;`) so the rule `font-family-name-quotes` is not violated.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 05da7664-1621-4552-be4c-47803683c7df
📒 Files selected for processing (7)
backend/api/routers/capture_ws.pybackend/api/routers/marketplace.pybackend/api/routers/openai_compat.pybackend/api/routers/tts_stream.pyfrontend/src-tauri/src/commands.rsfrontend/src-tauri/src/lib.rsfrontend/src/components/CaptureWidget.css
🚧 Files skipped from review as they are similar to previous changes (1)
- frontend/src-tauri/src/lib.rs
Comment on lines
+161
to
+162
| with zf.open("metadata.json") as mf: | ||
| metadata = json.load(mf) |
Contributor
There was a problem hiding this comment.
Treat malformed metadata.json as a 400, not a 500.
A bad or non-UTF-8 metadata.json currently bubbles out of json.load(...) as an internal error from both import paths even though the bundle is just invalid input.
Proposed fix
- with zf.open("metadata.json") as mf:
- metadata = json.load(mf)
+ try:
+ with zf.open("metadata.json") as mf:
+ metadata = json.load(mf)
+ except (json.JSONDecodeError, UnicodeDecodeError) as exc:
+ raise HTTPException(
+ status_code=400,
+ detail="Invalid .omnivoice bundle: malformed metadata.json",
+ ) from excAlso applies to: 364-365
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/api/routers/marketplace.py` around lines 161 - 162, The json.load(mf)
call when reading "metadata.json" can raise JSONDecodeError or
UnicodeDecodeError and should be treated as a client error; catch
json.JSONDecodeError and UnicodeDecodeError around the zf.open("metadata.json")
block (both occurrences where metadata = json.load(mf) is used) and convert them
to an HTTP 400 response (raise HTTPException(status_code=400, detail="Invalid
metadata.json" or similar) so malformed or non-UTF-8 metadata is reported as bad
input rather than an internal server error). Ensure you reference the same error
handling in both code paths that use zf and metadata.
Comment on lines
+348
to
+350
| bundle_path = MARKETPLACE_DIR / filename | ||
| if not bundle_path.is_file(): | ||
| raise HTTPException(status_code=404, detail=f"Bundle not found: {filename}") |
Contributor
There was a problem hiding this comment.
Reject path separators in filename before touching the filesystem.
MARKETPLACE_DIR / filename will resolve ..\... or absolute-path inputs on Windows, so /install can read and DELETE can unlink files outside the marketplace directory. Resolve the candidate path and reject anything that does not stay under MARKETPLACE_DIR.
Proposed fix
+def _resolve_marketplace_bundle(filename: str) -> Path:
+ root = MARKETPLACE_DIR.resolve()
+ candidate = (root / filename).resolve()
+ if candidate.parent != root or candidate.name != filename:
+ raise HTTPException(status_code=400, detail="Invalid bundle filename")
+ return candidate
+
`@router.post`("/install/{filename}")
async def install_from_marketplace(filename: str):
"""Import a voice profile from a bundle in the local marketplace directory."""
- bundle_path = MARKETPLACE_DIR / filename
+ bundle_path = _resolve_marketplace_bundle(filename)
if not bundle_path.is_file():
raise HTTPException(status_code=404, detail=f"Bundle not found: {filename}")
@@
`@router.delete`("/{filename}")
def remove_from_marketplace(filename: str):
"""Remove a bundle from the local marketplace directory."""
- bundle_path = MARKETPLACE_DIR / filename
+ bundle_path = _resolve_marketplace_bundle(filename)
if not bundle_path.is_file():
raise HTTPException(status_code=404, detail=f"Bundle not found: {filename}")Also applies to: 421-427
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/api/routers/marketplace.py` around lines 348 - 350, Reject filenames
that escape the marketplace directory by resolving the candidate path and
ensuring it is inside MARKETPLACE_DIR before touching the filesystem: compute
base = MARKETPLACE_DIR.resolve() and candidate = (MARKETPLACE_DIR /
filename).resolve() (or use candidate.relative_to(base) in a try/except), and if
candidate is not under base raise HTTPException(404, detail=...) instead of
directly using bundle_path; apply the same check to the other occurrence
handling filenames in the same module (the block around the later bundle
handling).
Comment on lines
+307
to
+313
| # Write uploaded file to a temp location | ||
| suffix = os.path.splitext(file.filename or "audio.wav")[1] or ".wav" | ||
| try: | ||
| with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tmp: | ||
| content = await file.read() | ||
| tmp.write(content) | ||
| tmp_path = tmp.name |
Contributor
There was a problem hiding this comment.
Stream the upload to disk instead of buffering the whole audio file first.
await file.read() loads the entire transcription upload into RAM before the temp-file write. Large recordings can spike worker memory and are easy to abuse without a size cap.
Proposed fix
suffix = os.path.splitext(file.filename or "audio.wav")[1] or ".wav"
try:
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tmp:
- content = await file.read()
- tmp.write(content)
+ total = 0
+ while chunk := await file.read(1024 * 1024):
+ total += len(chunk)
+ tmp.write(chunk)
tmp_path = tmp.name
except Exception as e:
raise HTTPException(status_code=400, detail=f"Could not read audio file: {e}")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/api/routers/openai_compat.py` around lines 307 - 313, The current
code buffers the whole upload via await file.read() before writing to disk;
change to stream the upload in chunks into the NamedTemporaryFile to avoid high
memory use: inside the with tempfile.NamedTemporaryFile(...) as tmp block, loop
reading fixed-size chunks (e.g., chunk = await file.read(8192)) until empty and
write each chunk to tmp (optionally track total bytes and raise an error if a
configured max size is exceeded). Keep the existing suffix/tmp_path logic and
ensure tmp is flushed/closed by the context manager; reference the variables
file, tmp, tmp_path and the tempfile.NamedTemporaryFile usage to locate where to
replace the single await file.read() call.
Comment on lines
+329
to
+387
| segments = result.get("segments", []) | ||
| chunks = result.get("chunks", []) | ||
| full_text = " ".join( | ||
| seg.get("text", "").strip() | ||
| for seg in (segments if segments else chunks) | ||
| ).strip() | ||
| detected_lang = result.get("language", language or "en") | ||
|
|
||
| # Format response based on requested format | ||
| if response_format == "text": | ||
| from fastapi.responses import PlainTextResponse | ||
| return PlainTextResponse(full_text) | ||
|
|
||
| if response_format == "verbose_json": | ||
| duration = result.get("duration", 0.0) | ||
| if not duration and segments: | ||
| last = segments[-1] | ||
| duration = last.get("end", 0.0) | ||
| return VerboseTranscriptionResponse( | ||
| task="transcribe", | ||
| language=detected_lang, | ||
| duration=duration, | ||
| text=full_text, | ||
| segments=[ | ||
| { | ||
| "id": i, | ||
| "text": seg.get("text", ""), | ||
| "start": seg.get("start", 0.0), | ||
| "end": seg.get("end", 0.0), | ||
| } | ||
| for i, seg in enumerate(segments) | ||
| ], | ||
| ) | ||
|
|
||
| if response_format == "srt": | ||
| from fastapi.responses import PlainTextResponse | ||
| srt_lines = [] | ||
| for i, seg in enumerate(segments, 1): | ||
| start = seg.get("start", 0.0) | ||
| end = seg.get("end", 0.0) | ||
| text = seg.get("text", "").strip() | ||
| srt_lines.append( | ||
| f"{i}\n" | ||
| f"{_format_ts_srt(start)} --> {_format_ts_srt(end)}\n" | ||
| f"{text}\n" | ||
| ) | ||
| return PlainTextResponse("\n".join(srt_lines), media_type="text/plain") | ||
|
|
||
| if response_format == "vtt": | ||
| from fastapi.responses import PlainTextResponse | ||
| vtt_lines = ["WEBVTT\n"] | ||
| for seg in segments: | ||
| start = seg.get("start", 0.0) | ||
| end = seg.get("end", 0.0) | ||
| text = seg.get("text", "").strip() | ||
| vtt_lines.append( | ||
| f"{_format_ts_vtt(start)} --> {_format_ts_vtt(end)}\n{text}\n" | ||
| ) | ||
| return PlainTextResponse("\n".join(vtt_lines), media_type="text/vtt") |
Contributor
There was a problem hiding this comment.
Normalize chunks into segments before building timed responses.
full_text already falls back to chunks, but verbose_json, srt, and vtt still iterate only segments. Backends that populate chunks will therefore return empty timestamps/subtitles even when transcription succeeded.
Proposed fix
- segments = result.get("segments", [])
- chunks = result.get("chunks", [])
+ chunks = result.get("chunks", [])
+ segments = result.get("segments") or chunks or []
full_text = " ".join(
seg.get("text", "").strip()
for seg in (segments if segments else chunks)
).strip()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/api/routers/openai_compat.py` around lines 329 - 387, The
verbose_json/srt/vtt branches only iterate the local segments list, so when the
backend returns transcription in chunks you lose timestamps; normalize chunks
into segments right after computing full_text by setting segments = segments if
segments else chunks (and then recompute duration using the last entry from
segments if needed) so VerboseTranscriptionResponse, the srt loop (uses
_format_ts_srt), and vtt loop (uses _format_ts_vtt) operate on the fallback
data; ensure any uses of enumerate(segments) and the duration calculation
reference the normalized segments variable.
|
|
||
| # Chunk size for streaming PCM audio (in samples). At 24kHz, 4800 samples = 200ms. | ||
| # Smaller chunks = lower latency but more WebSocket overhead. | ||
| CHUNK_SAMPLES = int(os.environ.get("OMNIVOICE_STREAM_CHUNK", "4800")) |
Contributor
There was a problem hiding this comment.
Guard against zero or negative CHUNK_SAMPLES.
If OMNIVOICE_STREAM_CHUNK is 0 or negative, sent_samples never advances and this loop hangs the websocket request indefinitely.
Proposed fix
-CHUNK_SAMPLES = int(os.environ.get("OMNIVOICE_STREAM_CHUNK", "4800"))
+try:
+ CHUNK_SAMPLES = max(1, int(os.environ.get("OMNIVOICE_STREAM_CHUNK", "4800")))
+except ValueError:
+ CHUNK_SAMPLES = 4800Also applies to: 185-193
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/api/routers/tts_stream.py` at line 35, Guard against zero/negative
chunk sizes by validating OMNIVOICE_STREAM_CHUNK when initializing
CHUNK_SAMPLES: parse the env var, convert to int, and if value <= 0 fallback to
the default (4800) or a minimum positive value, e.g., max(1, parsed) so
CHUNK_SAMPLES is always > 0; also ensure the send loop that advances
sent_samples (the websocket streaming loop referencing sent_samples and
CHUNK_SAMPLES) will always make progress when CHUNK_SAMPLES is used.
Comment on lines
+473
to
+478
| let _ = Command::new("reg") | ||
| .args(["delete", "HKCU\\Software\\Microsoft\\Windows\\CurrentVersion\\Run", | ||
| "/v", "OmniVoicePill", "/f"]) | ||
| .status(); | ||
| log::info!("Pill autostart disabled via registry"); | ||
| return Ok(()); |
Contributor
There was a problem hiding this comment.
Return an error if Windows registry deletion fails.
Lines 473-476 ignore both process-exec errors and non-zero exit status, so the command can report success even when autostart remains enabled.
Suggested patch
#[cfg(target_os = "windows")]
{
use std::process::Command;
- let _ = Command::new("reg")
+ let status = Command::new("reg")
.args(["delete", "HKCU\\Software\\Microsoft\\Windows\\CurrentVersion\\Run",
"/v", "OmniVoicePill", "/f"])
- .status();
+ .status()
+ .map_err(|e| format!("reg delete: {e}"))?;
+ if !status.success() {
+ return Err("Failed to delete registry key".into());
+ }
log::info!("Pill autostart disabled via registry");
return Ok(());
}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@frontend/src-tauri/src/commands.rs` around lines 473 - 478, The
registry-delete Command::new(...) call currently ignores both the Result and the
ExitStatus so failures are swallowed; capture the result of .status(), handle
the Err from process spawn and non-success exit statuses from the returned
ExitStatus, log an error (e.g., via log::error!) with the command, status and
any stderr info, and return an Err from the surrounding function instead of
unconditionally returning Ok(()). Locate the Command::new("reg") invocation in
commands.rs and replace the ignored call with error-handling that maps
process::ExitStatus failures into a meaningful error (using your crate's error
type/path) so callers see failure when autostart deletion fails.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Scalar-powered API documentation, GitHub community health files, and README improvements.
Changes
Backend
/docs— replaces default Swagger UI with modern, interactive Scalar reference/v1/audioendpoints (openai_compat router)scalar-fastapiadded to dependencies + lockfileCommunity Health (GitHub tabs)
README
docker pullinstructionslocalhost:3900/docs)Frontend
Testing
Type
Checklist
Summary by CodeRabbit
New Features
Documentation
Chores