Skip to content

(CODUX-182) Fix speech request scoping and local ONNX audio decoding - #2680

Merged
numnx merged 1 commit into
feature/CODUX-182-speech-input-and-memory-browserfrom
task/feature-codux-182-speech-input-and-memory-browse-t08-codex-mraqmj6l
Jul 7, 2026
Merged

(CODUX-182) Fix speech request scoping and local ONNX audio decoding#2680
numnx merged 1 commit into
feature/CODUX-182-speech-input-and-memory-browserfrom
task/feature-codux-182-speech-input-and-memory-browse-t08-codex-mraqmj6l

Conversation

@numnx

@numnx numnx commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

🧩 Task: Fix speech request scoping and local ONNX audio decoding

Automated task PR opened by Code UX for T08.

📋 Summary

Sprint: Sprint 182 · Speech Input and Memory Browser · Task: T08

Add speech-to-text input across the dashboard chat thread composer with local ONNX transcription and external API fallback, package it safely for Electron/npm runtimes, and modernize the Memory page model browser with grouped embedding/TTS models plus custom Hugging Face model links.

🤖 Model & Provider

Provider Codex CLI
Model gpt-5.5

⏱️ Timing

Started 2026-07-07 14:21:54 UTC
Finished 2026-07-07 14:29:02 UTC
Duration 7m 8s

📊 Token Usage

Metric Value
Input tokens 120,259
Cached input tokens 3,147,008
Output tokens 15,455
Total tokens 3,282,722
Invocations 1
Tool calls 69
Active time 6m 38s

Included usage estimate (subscription/local login): $2.64

✅ QA Review

⏳ QA review runs after this PR is opened. Check the task card in the dashboard for the outcome once it lands.

📝 Full Task Prompt
## Objective
Fix the speech input pipeline so dashboard recordings resolve project-scoped speech settings and the local ONNX path consumes decoded audio instead of raw upload bytes.

## Scope
- Edit `dashboard/src/v2/lib/speech-api.ts`
- Edit `dashboard/src/v2/components/speech/SpeechInputButton.tsx`
- Edit `dashboard/src/v2/ChatPage.tsx` if needed to supply the active project scope
- Edit `src/services/speech-transcription-service.ts`
- Edit `src/services/speech-audio-utils.ts` if a shared decoder/helper belongs there
- Add or update tests in the existing speech client, component, and backend service suites

## Implementation Requirements
1. Add optional `projectId` and `sprintId` fields to the speech upload client input and include them in the multipart form so the backend can resolve project-scoped speech settings.
2. Thread the active project context from the chat composer into the shared speech button/client without changing the existing read-only, append, or disabled behavior.
3. Replace the local ONNX byte-to-float stub with actual audio decoding from the uploaded WAV/PCM payload, and keep the local path aligned with the recorder output format.
4. Preserve the existing external API fallback behavior, request timeout handling, and structured error responses.
5. Add tests proving project-scoped metadata is sent and the local transcription path uses decoded audio rather than raw container bytes.

## Constraints
- Do not change the speech settings schema or recorder UX.
- Do not add new runtime dependencies.
- Do not weaken the existing fallback, permission, or error handling behavior.

## Verification
- Run the focused speech client, speech component, and speech backend tests that cover this path.
- Run `pnpm run lint`.
- Confirm the dashboard upload request now includes project scope and the local ONNX path works against decoded audio input.
🌿 Branch Info

Base: feature/CODUX-182-speech-input-and-memory-browser
Head: task/feature-codux-182-speech-input-and-memory-browse-t08-codex-mraqmj6l


🤖 Generated by Code UX

@numnx
numnx merged commit 8c97c88 into feature/CODUX-182-speech-input-and-memory-browser Jul 7, 2026
6 checks passed
@numnx
numnx deleted the task/feature-codux-182-speech-input-and-memory-browse-t08-codex-mraqmj6l branch July 7, 2026 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant