Feature/server multipart transcriptions - #25
Merged
0xShug0 merged 3 commits intoJul 3, 2026
Conversation
Owner
|
@dkruyt Thank you for your contribution! I’ll test the PR, and if everything looks good, I’ll merge it. |
Owner
|
@dkruyt Two small follow-ups before merge:
|
OpenAI's Whisper API (and clients built against it, e.g. Open WebUI) POST the audio as a multipart/form-data "file" part rather than a JSON body with a server-local path. Add a small multipart parser and route transcription requests with a multipart Content-Type through it: the uploaded bytes are spooled to a temp file and passed through the existing JSON-based request builder, so the plain JSON path (audio/audio_path/file as a string path) keeps working unchanged.
Clients that build a voice picker (e.g. llama-swap's playground) have no way to learn which cached_voice_id values a TTS model actually supports, so they fall back to generic placeholder names (alloy, nova, ...) that don't exist as real presets and 500 at inference time. For families that keep voice presets under model_root/embeddings/*.safetensors (pocket_tts today, see assets.cpp), list the available ids. Families without an embeddings directory just report no voices, same as an unknown or missing model id. Also required capturing the raw query string in HttpRequest, which read_http_request was previously discarding entirely.
…t test Addresses review feedback on the multipart-transcriptions PR: the README didn't mention the new multipart upload path or the /v1/audio/voices endpoint, and the multipart parser had no test coverage or CTest target.
dkruyt
force-pushed
the
feature/server-multipart-transcriptions
branch
from
July 3, 2026 18:22
cc55db4 to
1b9fc76
Compare
Contributor
Author
|
Both addressed:
|
Owner
|
@dkruyt merged. Thanks! |
dleiferives
pushed a commit
to dleiferives/audio.cpp
that referenced
this pull request
Jul 25, 2026
…anscriptions Feature/server multipart transcriptions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two small additions to
audiocpp_serverthat make it a better drop-in for OpenAI-API-compatible clients (Open WebUI, llama-swap's playground, etc.):Accept multipart/form-data uploads on
POST /v1/audio/transcriptions. The real OpenAI Whisper API (and every client built against it — Open WebUI included) uploads audio as amultipart/form-datafilepart.audiocpp_servercurrently only accepts a JSON body with a server-local path (audio/audio_path/fileas a string). That's fine for local scripting, but it means the endpoint can't be used as-is behind a client that does real file uploads. This adds a small multipart parser (app/server/multipart.{h,cpp}) and routes requests with amultipart/form-dataContent-Typethrough it: the uploaded bytes are spooled to a temp file and handed to the existing JSON-based request builder, so the original JSON/path request shape keeps working unchanged.Add
GET /v1/audio/voices?model=<id>. Clients that build a voice picker (llama-swap's playground does this) have no way to discover whichcached_voice_idvalues a TTS model actually supports, so they fall back to generic placeholder names (alloy,nova, ...) that don't exist as real presets and 500 at inference time. For families that keep voice presets undermodel_root/embeddings/*.safetensors(pocket_ttstoday — seeassets.cpp), this lists the available ids. Families without an embeddings directory, or an unknown/missingmodelquery param, just report an empty list, matching how the frontend already falls back gracefully. This also required capturing the raw query string onHttpRequest, whichread_http_requestwas previously discarding entirely.Neither change touches the existing JSON-only request shapes — both are purely additive.
Testing
audiocpp_serverwith CUDA (-DENGINE_ENABLE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native -DGGML_NATIVE=ON) and ran it against real weights (Qwen3-ASR-0.6B,kyutai/pocket-tts), on an actual GPU:POST /v1/audio/transcriptionswith a JSON{"model","audio"}body (existing behavior) — unchanged, verified no regression.POST /v1/audio/transcriptionswith a realmultipart/form-dataupload (-F file=@... -F model=..., with and withoutlanguage) — new behavior, correct transcript returned, temp file cleaned up afterward.file/modelmultipart fields — errors out same as other malformed requests (500, consistent with existing error handling style).GET /v1/audio/voices?model=pocket-tts— returns the real cached voice ids (alba,cosette,marius, ...).GET /v1/audio/voices?model=qwen3-asr(no embeddings dir),?model=<unknown>, and nomodelparam — all return{"voices":[]}.POST /v1/audio/speech— unchanged, verified no regression after both changes./v1/audio/voicesinstead of guessing.Notes for reviewers
handle_transcriptionnow dispatches onContent-Type(multipart/form-data→ new path, otherwise → existing JSON path) rather than replacing the JSON handling.http.cpp.model_root/embeddingsexist" rather than hardcodingpocket_tts, so it should pick up other families automatically if they adopt the same layout.