[Feature Request / Bug]: Support non-WAV / browser audio uploads (WebM/Opus, MP3, OGG) on /v1/audio/transcriptions for Open WebUI compatibility

[ampersandru]

Description

Currently, the POST /v1/audio/transcriptions endpoint strictly expects raw WAV PCM audio uploads (RIFF....WAVE header).

When integrating audio.cpp with popular web clients like Open WebUI, transcription fails because modern web browsers (Chrome, Edge, Firefox, Safari) using the HTML5 MediaRecorder API record microphone input in audio/webm;codecs=opus (or audio/ogg). Browsers do not support recording native WAV via MediaRecorder.

As a result, sending microphone recordings from Open WebUI to any ASR model (e.g. parakeet_tdt, granite5asr, whisper) fails with:
Error transcribing chunk: External: 500, message='Internal Server Error', url='http://:8080/v1/audio/transcriptions'

In contrast, clients that buffer and convert audio to WAV beforehand (such as Home Assistant via wyoming_openai or curl -F file=@sample.wav) succeed with exceptional sub-30ms latency.

---

Steps to Reproduce

1. Start audiocpp_server with any ASR model (e.g., parakeet_tdt or granite5asr):

   {
     "models": [
       {
         "id": "parakeet_tdt",
         "family": "parakeet_tdt",
         "path": "/models/parakeet-tdt-0.6b-v3-q8_0.gguf",
         "task": "asr",
         "mode": "offline"
       }
     ]
   }
   

2. In Open WebUI (Admin Settings ➔ Audio):

• Set Speech-to-Text Engine: OpenAI
• Set API Base URL: http://:8080/v1
• Set STT Model: parakeet_tdt

3. Click the microphone button in Open WebUI chat to record speech.
4. Open WebUI fails with HTTP 500.

Root Cause

handle_transcription_multipart in app/server/runtime.cpp checks is_wav_upload_filename and directly parses bytes using engine::audio::read_wav_f32(), rejecting .webm, .mp3, .ogg, and .m4a files.
When browsers send MediaRecorder chunks or blobs (often named audio.webm or blob), the server throws an unhandled exception or returns a 400/500 error instead of decoding the compressed audio stream.

Suggested Fixes & Solutions

Solution 1: Lightweight In-Tree Decoders (Recommended)

Embed header-only C/C++ audio decoders (similar to how ggml or llama.cpp bundle dependencies) so audio.cpp remains self-contained without external runtime dependencies:

miniaudio (Public Domain / MIT, single-file) — handles MP3, FLAC, WAV, and Vorbis decoding natively.
opusfile / libopus — decodes WebM/Ogg Opus streams directly to 16 kHz float32 PCM in memory.

Solution 2: Optional Server-Side Transcoder Bridge

If external tools are acceptable when present, allow audiocpp_server to detect and pipe incoming non-WAV multipart bytes through ffmpeg or libavcodec to convert to 16 kHz mono WAV PCM in memory before running inference.

Solution 3: Graceful Handling of Empty / Incomplete Chunks

When MediaRecorder begins recording, the initial chunk may sometimes be 0 bytes or header-incomplete. Returning {"text": ""} (HTTP 200) instead of throwing an unhandled exception ensures the client does not crash before the full audio is delivered.

[0xShug0]

I see the issue. Let me think about it. This could be part of the longer-term plan.

[ampersandru]

I see the issue. Let me think about it. This could be part of the longer-term plan.

thanks - another possible request, support wyoming protocol for direct compatibility with home assistant. Currently, I have to use https://github.com/roryeckel/wyoming_openai to connect any openai-compatible TTS/STT to home assistant. Having it directly incorporated somehow would be amazing!

[0xShug0]

The question I’m considering is not what to support, but how to support it. The core server should stay lightweight and avoid forcing users to compile unnecessary dependencies, while features that are better suited to the frontend or can be handled there easily should be implemented as extensions in the server. That will likely require some redesign of the server architecture. That’s why MP3 support (https://github.com/0xShug0/audio.cpp/tree/ci/new-feature-mp3) was never merged.

[arty-hlr]

I was trying out audio.cpp with open-webui and came across this issue as well. Maybe an extension would make sense or something optional if you don't want to add it by default to the core server @0xShug0 , and at least an option in the config to support this or that format. The docs can then show that the dependency for the extension needs to be installed to support other audio file types.

[arty-hlr]

In the meantime I wrote a quick proxy to convert to wav, maybe that's also useful for others until the feature is added here :)

https://github.com/arty-hlr/audio_to_wav_cpp

[arty-hlr]

As a side note, it seems that open-webui already converts webm to mp3, so audio.cpp would only need to support mp3 -> wav in this case.

2026-08-27 15:21:58.653 | INFO     | open_webui.routers.audio:transcription:1207 - file.content_type: audio/webm; codecs=opus
2026-08-27 15:21:58.655 | INFO     | open_webui.routers.audio:transcribe:1080 - transcribe: /app/backend/data/cache/audio/transcriptions/fcf757e8-b2cb-488e-8eac-540d64bfc9eb.webm None
2026-08-27 15:21:58.915 | INFO     | open_webui.routers.audio:convert_audio_to_mp3:160 - Converted /app/backend/data/cache/audio/transcriptions/fcf757e8-b2cb-488e-8eac-540d64bfc9eb.webm to /app/backend/data/cache/audio/transcriptions/fcf757e8-b2cb-488e-8eac-540d64bfc9eb.mp3
Chunk paths: ['/app/backend/data/cache/audio/transcriptions/fcf757e8-b2cb-488e-8eac-540d64bfc9eb.mp3']
2026-08-27 15:21:59.270 | INFO     | uvicorn.protocols.http.httptools_impl:send:485 - 192.168.178.22:0 - "POST /api/v1/audio/transcriptions HTTP/1.1" 200

[0xShug0]

In the meantime I wrote a quick proxy to convert to wav, maybe that's also useful for others until the feature is added here :)

https://github.com/arty-hlr/audio_to_wav_cpp

Thanks for the pointer! I'm thinking about the same direction. I feel the easier and clean solution is adding a "middleware" to the server which is opt-in,and decoupled from the server.

[IIIIIllllIIIIIlllll]

A preliminary suggestion: add a compile option to decide whether to enable the decoder, generate a standalone file, and let audiocpp_server check at runtime whether the decoder is present and reference it if so?

[0xShug0]

A preliminary suggestion: add a compile option to decide whether to enable the decoder, generate a standalone file, and let audiocpp_server check at runtime whether the decoder is present and reference it if so?

We definitely need a compile option if we go with the middleware/proxy route. But it should be transparent to the core server. I.e., the server should continue to operate on the same normalized audio input, while the optional middleware handles format detection/decoding before forwarding the request.

[osos]

I use the hviske-5.3 model for Danish transcription. It fully kills my DGX Spark if I throw a MP3 file (53mb) at it through the built-in webui. No error messages, just dead spark ready for power cycle. So some handling of non-wav is required.

[ampersandru]

I tried to connect audio.cpp to my hermes agent and, on the reverse side for TTS, it also complained about .wav. I had it summarize the problem and need:

Description

The TTS endpoint POST /v1/audio/speech currently ignores the response_format parameter and always returns a raw RIFF/WAVE PCM payload, regardless of whether the client requested mp3, ogg, opus, or flac.

This is the output-side counterpart to #313 (which is the input-side: /v1/audio/transcriptions only accepts WAV). Both stem from the same WAV-centric audio core (engine::audio). #313 asks for decoders so the server can read non-WAV; this asks for the matching encoders so the server can write non-WAV.

Why it matters (concrete client: Hermes Agent)

Hermes's built-in OpenAI-compatible TTS provider calls /v1/audio/speech with:

• model, voice, input, and response_format (mp3 for TUI/CLI, ogg/opus for Telegram voice bubbles).

Because the server always returns WAV:

• Telegram path works only because Hermes has a post-hoc _repair_ogg_container step that detects non-Opus bytes in the .ogg and re-encodes to real Opus via ffmpeg.
• TUI/CLI path is a latent mismatch: the file is named .mp3 but contains WAV bytes. It plays in permissive players, but any player/tool that trusts the extension (or streams it as MP3) breaks.

So today, correct behavior depends on the client papering over the server's WAV-only output. The clean fix is for the server to honor response_format.

Steps to Reproduce

1. Start audiocpp_server with any TTS model (e.g. omnivoice).
2. Request speech with an explicit non-WAV format:

   curl -s http://<server>:8080/v1/audio/speech \
     -H "Authorization: Bearer <key>" \
     -H "Content-Type: application/json" \
     -d '{"model":"omnivoice","voice":"scarlett","input":"hello","response_format":"mp3"}' \
     -o out.mp3

3. Inspect out.mp3 → magic bytes are 52 49 46 46 … 57 41 56 45 (RIFF…WAVE), i.e. WAV, not MP3. Same result for response_format: "ogg" / "opus".

Root Cause

The speech handler writes its PCM buffer out through the WAV writer only; there is no encoder dispatch on response_format. (Mirror of #313's handle_transcription_multipart → read_wav_f32() decode-only path.)

Suggested Fixes

Solution 1 (recommended): in-tree encoders, matching #313's decoder approach

• WAV / FLAC — encode in-tree via miniaudio (public domain; it already handles WAV+FLAC writing).
• OGG / Opus — encode via opusfile / libopus (same dependency [Feature Request / Bug]: Support non-WAV / browser audio uploads (WebM/Opus, MP3, OGG) on /v1/audio/transcriptions for Open WebUI compatibility #313 proposes for decoding).
• MP3 — miniaudio decodes MP3 but does not encode it; either bundle a small MP3 encoder (e.g. lame) as an optional build flag, or explicitly document MP3 as unsupported and return a clear error instead of silently emitting WAV.

Solution 2 (fallback): server-side transcode bridge

• If external tools are acceptable, pipe the generated WAV through ffmpeg to the requested codec before responding (this is exactly what Hermes does client-side today — moving it server-side removes the per-client workaround).

Minimum acceptable behavior (if full encoders are out of scope):

• At least set the Content-Type header to match the actual bytes returned (audio/wav when emitting WAV), and/or reject unsupported response_format values with a clear 400 instead of silently returning WAV under an MP3 label. That alone removes the mislabeled-file footgun for clients.

Acceptance Criteria

• response_format: "wav" → RIFF/WAVE (unchanged).
• response_format: "ogg" / "opus" → valid Ogg/Opus stream (passes ffprobe as ogg/opus).
• response_format: "mp3" → valid MP3 or an explicit 400 "format not supported" (not silent WAV).
• Content-Type always matches the actual payload.

[arty-hlr]

@ampersandru Quick side question, how did you manage to make local openai compatible TTS/STT work with Hermes / how does your config.yaml and .env look like for it? It kept ignoring them and falling back on local default when I tried. You can PM me on discord @h3ll3r_ to not spam here if you want.

[ampersandru]

@ampersandru Quick side question, how did you manage to make local openai compatible TTS/STT work with Hermes / how does your config.yaml and .env look like for it? It kept ignoring them and falling back on local default when I tried. You can PM me on discord @h3ll3r_ to not spam here if you want.

I actually just asked hermes to set it to my openai-compatible TTS server, gave it the server url, api key, model, and voice. I checked config.yaml and hermes had updated it to:

tts:
  provider: openai
  edge:
    voice: en-US-AriaNeural
  elevenlabs:
    voice_id: pNInz6obpgDQGcFmaJgB
    model_id: eleven_multilingual_v2
  openai:
    model: omnivoice
    voice: scarlett_johansson
    base_url: https://audio.xxxx.net/v1
    api_key: xxxx

i have it just output to .wav for now, and luckily discord, where I have hermes live, supports .wav uploads. I think hermes just warned about .wav only because a lot of people use Telegram to connect Hermes.

[arty-hlr]

Grmpf, that's what I have as well, I'll have to investigate more why it's not working. And you have STT working with a similar config?

[0xShug0]

@ampersandru @arty-hlr @IIIIIllllIIIIIlllll @osos Okay I have something you can test. Check #330

This adds an opt-in in-process server frontend pipeline. The core server stays WAV/text in and WAV/text out, while selected frontend modules can pre-process requests or post-process responses. This keeps client-specific extensions out of the core server. New format converters, text normalization, request rewrites, or response adapters can be added as separate modules with their own dependencies and explicit contracts. When compiling, you can select exactly which frontend modules to include, so unused extensions and their dependencies are not built into the default server.

The first modules support MP3/FLAC input for ASR and response_format=mp3 output for TTS.

The frontend build logic is now isolated so future extensions are scoped to:

app/server/frontends/

A new frontend module should add:

• its source under app/server/frontends/
• its docs in app/server/frontends/README.md
• its CMake branch/dependencies in app/server/frontends/server_frontends.cmake

The top-level CMakeLists.txt does not need more frontend-specific logic beyond the existing include and configure call.

The doc is https://github.com/0xShug0/audio.cpp/pull/330/changes#diff-081a3c99187d3268b4929ff43bbfd1d3c8d20a92a1c14f9082aa2a34d1bb520f

[osos]

@0xShug0, probably going a week until I can test it out. Thank you.

If I understand the patch right it does not alter the api's. I would in any event suggest that the api's hard validate the input files (in STT mode) against a list of supported format - preferable with the magic bytes kind of validation, and then return a valid failure 403-ish response about the format not supported. Same goes if the requested response (in TTS mode) is formats not supported as output.

[Tech-no-1]

I just created a Windows CPU build based on the "server-frontend-module" branch and tested it with my local Open WebUI, unfortunately I'm still getting the same "500 internal server error" message as before when using the transcription endpoint.

I first modified the build_windows.ps1 to incorporate the new build arguments and then used it for the build (with preset windows-cpu-release and target audiocpp_server). In the CMakeFiles directory, there is the "audiocpp_server_frontends.dir" containing the "audio_decode.cpp.obj", so it should be included in the build. Maybe I'm just doing something wrong.

"-DAUDIOCPP_BUILD_SERVER_FRONTENDS=ON",
"-DAUDIOCPP_SERVER_FRONTEND_MODULES=audio_decode"

In my opinion, the frontend modules should be included by default in the pre compiled release packages and Docker images (or perhaps in variants that include the frontend). While it's nice to be able to choose which modules you want to include before compilation, many people either don't know how to make their own builds, don't want to or don't have everything set up (the build environment). I think there is a significant demand for a good (more or less all in one) STT/TTS solution that also includes a server (it was hard to find). It would be a real shame if it could not be use with external applications like Open WebUI simply because the frontend is missing.

[0xShug0]

@Tech-no-1 Can you confirm whether Open WebUI actually sends MP3/FLAC in your test, instead of WebM/Ogg/Opus? The current test branch only supports MP3/FLAC input and MP3 output.

Codec/frontend dependencies are not core model runtime dependencies. Shipping all of them by default would make every release package carry optional dependencies that only some integrations need. However, following llama.cpp’s approach and bundling a minimal frontend module with lightweight dependencies, such as miniaudio, is probably fine.

The current prebuilts mostly target UI users, where these features can be added more easily in the UI if needed.

[Tech-no-1]

@Tech-no-1 Can you confirm whether Open WebUI actually sends MP3/FLAC in your test, instead of WebM/Ogg/Opus? The current test branch only supports MP3/FLAC input and MP3 output.

Oh, I think I see the problem now. If the output is also supposed to be in MP3, I probaply need to include the 'mp3_encode' module in the build as well. I am now getting the following error during build though:

CMake Error at app/server/frontends/server_frontends.cmake:41 (message):
  AUDIOCPP_SERVER_FRONTEND_MODULES=mp3_encode requires libmp3lame headers and
  library (install libmp3lame-dev or build from an environment that provides
  lame/lame.h)
Call Stack (most recent call first):
  CMakeLists.txt:1794 (audiocpp_configure_server_frontends)

libmp3lame-dev appears to be a Debian package (I'm on Windows) and lame/lame.h is missing from the build environment.

Codec/frontend dependencies are not core model runtime dependencies. Shipping all of them by default would make every release package carry optional dependencies that only some integrations need. However, following llama.cpp’s approach and bundling a minimal frontend module with lightweight dependencies, such as miniaudio, is probably fine.

The current prebuilts mostly target UI users, where these features can be added more easily in the UI if needed.

Yes, I understand that and I'm aware that most users are simply using the UI (which is excellent btw), a minimal frontend for core/basic compatibility with external services via the API endpoints would be nice though.

[0xShug0]

libmp3lame-dev appears to be a Debian package (I'm on Windows)

This is mostly just me being lazy. A better solution might be to use a small vendored library for encoding, but I haven’t started looking into it yet. Many factors to consider/test: size, portability, license, maturity, maintenance burden etc.

You can see the issue here: we may need to add different dependencies that may have duplicated or unnecessary functionality just for certain fronend features. It took me some time to extract the tokenizer from llama.cpp to avoid depending on llama.cpp (and SentencePiece may be next cleanup target), so I'd rather avoid going back in the opposite direction.

Another issue is that it’s really hard to guarantee that random dependencies will behave correctly across different environments and configurations. Some models do use runtime dependencies, but they are scoped to individual models and have safe, generic fallbacks. The maintenance effort should go into the framework, rather than external dependencies.

I’m now leaning more toward separating the frontend module from audio.cpp entirely into its own repo. That would give users more freedom to customize and extend it based on their needs ("plugins"), while keeping the main audio.cpp repo focused and limited to a minimal set of vendored dependencies.

[0xShug0]

Updated the PR: mp3_encode now supports -DAUDIOCPP_LAME_ROOT=<prefix>. CMake now searches conda’s Windows layout too: Library/include and Library/lib. It still requires installation of LAME via vcpkg or conda on Windows.

[Tech-no-1]

Updated the PR: mp3_encode now supports -DAUDIOCPP_LAME_ROOT=<prefix>. CMake now searches conda’s Windows layout too: Library/include and Library/lib. It still requires installation of LAME via vcpkg or conda on Windows.

Okay, so first of all, the Windows build is working now. However, audiocpp_server fails to start because the "libmp3lame.dll" is missing. I simply copied the dll from mp3lame and the server started. I haven't tested whether MP3 output actually works yet, because Open WebUI accepts WAV.

Transcoding still isn't working and returns the "500 internal server error". I can confirm that Open WebUI converts the audio to MP3 first and stores that file in cache.

Is there a way to get a verbose/debug log, because even with --log I get nothing.

[0xShug0]

@Tech-no-1 added temp logs. You should see something like.

[MP3_FRONTEND_DEBUG] http.headers method=POST path=/v1/audio/transcriptions content_length=1255076 transfer_encoding=<none> incremental=false
[MP3_FRONTEND_DEBUG] http.body_ready path=/v1/audio/transcriptions body_bytes=1255076
[MP3_FRONTEND_DEBUG] audio_decode.multipart.parts count=2
[MP3_FRONTEND_DEBUG] audio_decode.multipart.part name=file filename=speech.mp3 bytes=1254753
[MP3_FRONTEND_DEBUG] audio_decode.multipart.format MP3
[MP3_FRONTEND_DEBUG] audio_decode.decoded sample_rate=16000 channels=1 samples=6689280
[MP3_FRONTEND_DEBUG] audio_decode.forward_json body_bytes=77
[MP3_FRONTEND_DEBUG] core.transcription.route json
[MP3_FRONTEND_DEBUG] frontend.core_response status=200 content_type=application/json body_bytes=8151

I don't know how Open WebUI works. ChatGPT said it may be because you sent a chunked MP3 without a Content-Length header.

[Tech-no-1]

Here are the relevant parts of the logs, I've changed the file paths and URLs.

Browser request to Open WebUI:

https://openwebui/api/v1/audio/transcriptions

Content-Disposition: form-data; name="file"; filename="Recording-06.09.2026 23:21.webm"
Content-Type: audio/webm; codecs=opus

<Content/Body>

Open WebUI:

2026-09-06 | INFO | open_webui.routers.audio:transcription:1207 - file.content_type: audio/webm; codecs=opus
2026-09-06 | INFO | open_webui.routers.audio:transcribe:1080 - transcribe: C:\path\to\open_webui\data\cache\audio\transcriptions\c4549af8-c651-413f-8af7-f84c02db1a2c.webm {'language': 'en'}
2026-09-06 | INFO | open_webui.routers.audio:convert_audio_to_mp3:160 - Converted C:\path\to\open_webui\data\cache\audio\transcriptions\c4549af8-c651-413f-8af7-f84c02db1a2c.webm to C:\path\to\open_webui\data\cache\audio\transcriptions\c4549af8-c651-413f-8af7-f84c02db1a2c.mp3
Chunk paths: ['C:\\path\\to\\open_webui\\data\\cache\\audio\\transcriptions\\c4549af8-c651-413f-8af7-f84c02db1a2c.mp3']
2026-09-06 | ERROR | open_webui.routers.audio:_transcribe_openai:707 - 500, message='Internal Server Error', url='http://audiocpp/v1/audio/transcriptions'

Audio.cpp:

[MP3_FRONTEND_DEBUG] http.headers method=POST path=/v1/audio/transcriptions content_length=<none> transfer_encoding=chunked incremental=false prefetched_body_bytes=503
[MP3_FRONTEND_DEBUG] http.body_ready path=/v1/audio/transcriptions body_bytes=0
[MP3_FRONTEND_DEBUG] frontend.enter method=POST path=/v1/audio/transcriptions content_type=multipart/form-data; boundary=bea047a51e9d4a919dfc5572b53190a1 body_bytes=0
[MP3_FRONTEND_DEBUG] frontend.pre.begin module=audio_decode path=/v1/audio/transcriptions body_bytes=0
[MP3_FRONTEND_DEBUG] audio_decode.multipart.enter content_type=multipart/form-data; boundary=bea047a51e9d4a919dfc5572b53190a1 body_bytes=0
[MP3_FRONTEND_DEBUG] audio_decode.multipart.parts count=0 boundary_bytes=32
[MP3_FRONTEND_DEBUG] audio_decode.multipart.no_file_part
[MP3_FRONTEND_DEBUG] frontend.pre.end module=audio_decode path=/v1/audio/transcriptions content_type=multipart/form-data; boundary=bea047a51e9d4a919dfc5572b53190a1 body_bytes=0 short_circuit=false
[MP3_FRONTEND_DEBUG] frontend.pre.begin module=mp3_encode path=/v1/audio/transcriptions body_bytes=0
[MP3_FRONTEND_DEBUG] frontend.pre.end module=mp3_encode path=/v1/audio/transcriptions content_type=multipart/form-data; boundary=bea047a51e9d4a919dfc5572b53190a1 body_bytes=0 short_circuit=false
[MP3_FRONTEND_DEBUG] core.transcription.enter content_type=multipart/form-data; boundary=bea047a51e9d4a919dfc5572b53190a1 body_bytes=0
[MP3_FRONTEND_DEBUG] core.transcription.route multipart
[MP3_FRONTEND_DEBUG] core.multipart.parts count=0 body_bytes=0
[MP3_FRONTEND_DEBUG] core.multipart.missing_file
[MP3_FRONTEND_DEBUG] http.error multipart transcription request requires a non-empty 'file' field

Part from the Open WebUI Traceback:

  File "C:\path\to\open_webui\routers\audio.py", line 925, in transcription_handler
    return await _transcribe_openai(request, file_path, filename, languages, file_dir, id, user)
> File "C:\path\to\open_webui\routers\audio.py", line 700, in _transcribe_openai
    r.raise_for_status()

This is a snippet from Open WebUIs 'audio.py' (transcribe_openai), seems like they include a 'file' field in the chunked request:

async def _transcribe_openai(request, file_path, filename, languages, file_dir, id, user=None):
    """Transcribe audio via an OpenAI-compatible STT endpoint."""
    r = None
    try:
        session = await get_session()
        api_key = await Config.get('audio.stt.openai.api_key')
        api_base_url = await Config.get('audio.stt.openai.api_base_url')
        request_format = (await Config.get('audio.stt.openai.api_request_format') or 'multipart').lower()

(...)

            if request_format == 'json':
                ext = os.path.splitext(filename)[1].lower().lstrip('.') or 'wav'
                async with aiofiles.open(file_path, 'rb') as f:
                    payload['input_audio'] = {
                        'data': base64.b64encode(await f.read()).decode('utf-8'),
                        'format': 'ogg' if ext == 'oga' else ext,
                    }

                r = await session.post(
                    url=f'{api_base_url}/audio/transcriptions',
                    headers={**headers, 'Content-Type': 'application/json'},
                    json=payload,
                    ssl=AIOHTTP_CLIENT_SESSION_SSL,
                )
            else:
               form_data = aiohttp.FormData()
                for key, value in payload.items():
                    form_data.add_field(key, str(value))

                async def audio_chunks():
                    async with aiofiles.open(file_path, 'rb') as audio_file:
                        while chunk := await audio_file.read(AIOHTTP_FILE_STREAM_CHUNK_SIZE):
                            yield chunk

                form_data.add_field('file', audio_chunks(), filename=filename)

                r = await session.post(
                    url=f'{api_base_url}/audio/transcriptions',
                    headers=headers,
                    data=form_data,
                    ssl=AIOHTTP_CLIENT_SESSION_SSL,
                )

Audio.cpp log with JSON request instead of default chunked/multipart:

[MP3_FRONTEND_DEBUG] http.headers method=POST path=/v1/audio/transcriptions content_length=31363 transfer_encoding=<none> incremental=false prefetched_body_bytes=7956
[MP3_FRONTEND_DEBUG] http.body_ready path=/v1/audio/transcriptions body_bytes=31363
[MP3_FRONTEND_DEBUG] frontend.enter method=POST path=/v1/audio/transcriptions content_type=application/json body_bytes=31363
[MP3_FRONTEND_DEBUG] frontend.pre.begin module=audio_decode path=/v1/audio/transcriptions body_bytes=31363
[MP3_FRONTEND_DEBUG] audio_decode.multipart.enter content_type=application/json body_bytes=31363
[MP3_FRONTEND_DEBUG] audio_decode.multipart.no_boundary
[MP3_FRONTEND_DEBUG] frontend.pre.end module=audio_decode path=/v1/audio/transcriptions content_type=application/json body_bytes=31363 short_circuit=false
[MP3_FRONTEND_DEBUG] frontend.pre.begin module=mp3_encode path=/v1/audio/transcriptions body_bytes=31363
[MP3_FRONTEND_DEBUG] frontend.pre.end module=mp3_encode path=/v1/audio/transcriptions content_type=application/json body_bytes=31363 short_circuit=false
[MP3_FRONTEND_DEBUG] core.transcription.enter content_type=application/json body_bytes=31363
[MP3_FRONTEND_DEBUG] core.transcription.route json
[MP3_FRONTEND_DEBUG] http.error transcription request requires audio, audio_path, or file path

[0xShug0]

@Tech-no-1 The branch has been updated to handle Open WebUI no Content-Length headers

curl -sS -i \
  -H 'Transfer-Encoding: chunked' \
  -F 'file=@build/logs/server-frontend-mp3-500-validation/input.mp3;type=audio/mpeg' \
  -F 'model=qwen3-asr-smoke' \
  http://127.0.0.1:18132/v1/audio/transcriptions

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 209
Connection: close

{"text":"Well, I don't wish to see it any more,\" observed Phoebe, turning away her eyes. \"It is certainly very like the old portrait.\"","timing":{"wall_ms":308.942,"audio_duration_ms":7435,"rtf":0.0415524}}

For the json failure, the issue is a schema mismatch: Open WebUI’s JSON transcription request does not match the JSON schema currently accepted by audio.cpp.

[Tech-no-1]

Yes, transcription is working now! (with chunked upload)

[MP3_FRONTEND_DEBUG] frontend.core_response status=200 content_type=application/json body_bytes=119

By the way, in my opinion, Cohere Transcribe is one of the best transcription models currently available (very accurate), it might be worth supporting it at some point.

I also tested TTS with Open WebUI, by default it requests/accepts WAV and transcodes it to MP3 afterwards.

"POST /api/v1/audio/speech HTTP/1.1" 200
open_webui.routers.audio:transcode_audio_to_mp3:211 - Transcoded audio/wav audio to MP3

To test the MP3 output, I explicitly used "response_format": "mp3" in the request JSON and it seems to work as well. Open WebUI did not transcode the responses into MP3.

I think you still need to add the "libmp3lame.dll" to the "package_windows_prebuilt" script (Copy-RequiredDll) so that it gets copied from mp3lame (at least for Windows builds). Otherwise the server won't start if you make a build with the 'mp3_encode' frontend module.

[0xShug0]

BTW Cohere Transcribe has been added.

[0xShug0]

Call it a day. Now that the frontend is a submodule in another repo https://github.com/0xShug0/audio.cpp-server-frontends, it's more decoupled from the main audio.cpp. I also added WebSocket support.

If anyone would like to help maintain that repo, let me know.

The main audio.cpp prebuilt binaries would stay as they are and would not include the frontend module. The front repo prebuilts would include all the modules and dependencies I'm not a CI expert, but I think it should be possible. Runtime LAME can be replaced with vendored lib, as I'm more comfortable letting the submodule pull in random dependencies.

@Tech-no-1 Thanks for the pointer! I added the model to TODOs.

[arty-hlr]

Sorry, just circling back to this now, somehow I wasn't subscribed to the issue. I'll test it this week on mac and let you know.