Row: MODEL-MM-dots3-note-dots3-note-for-causal-lm
W7c-1 is the CHANNELS half of W7c, split out because it is an exact mirror with no oracle
risk, while the sample-rate half is an unavoidable divergence. Keeping them apart stops one
claim borrowing the other's credibility.
What is refused today, and why it is nearly free to lift
The audio path accepts PCM16 mono WAV at 16 kHz. A stereo or multi-channel PCM16 WAV
already at 16 kHz is refused only for want of a mean over channels.
Upstream reduces to mono in two places, both plain means:
- decode side,
vllm/multimodal/media/audio.py:207-208 @ 9035151d6 — np.mean over channels
when mono=True, which is load_audio's default (:220);
- parser side,
vllm/multimodal/audio.py:46-56 and :91-161 @ 9035151d6 —
ChannelReduction.MEAN is the default and AudioSpec.target_channels is 1, which
common/processor.py:523-525 selects for dots3.
So this is a mirror, roughly ten lines, and it stops refusing a whole class of real files.
Scope
Widen DecodeWavPcm16Mono (src/vllm/multimodal/audio_processor.cpp:35-79), or add a
channel-aware sibling beside it, to accept multi-channel PCM16 at the target rate and reduce
by mean. Nothing else changes.
Reachability
Production entry point ApiServer::handle_chat_completions → InstallMultiModalChatSeam
(server_main.cpp:1565) → MakeDots3NoteChatSeam → RouteDots3NoteAudioWav
(mm_chat_dots3note.cpp:210).
The smallest failing test already exists and must invert. The SUBCASE
"a STEREO WAV names the container refusal and W7c" at
tests/vllm/entrypoints/openai/test_api_server_dots3_mm_forward.cpp:1379 asserts the refusal
today. It must go true-before / false-after, which is the ownership test AGENTS.md asks for.
Replace it with a case asserting the stereo request SERVES and that its logprobs equal those
of an independently computed per-sample mean.
In flow, in the same change: three statements about the oracle are false
librosa occurs exactly ONCE at 9035151d6, in a comment at vllm/multimodal/audio.py:31
about get_duration. It is not on the decode or resample path at all. The real chain is
soundfile/libsndfile first, PyAV/ffmpeg as fallback, and libswresample for the resample.
Three places in this tree say otherwise, and two of them are production error strings a
user reads:
src/vllm/entrypoints/openai/mm_chat_dots3note.cpp:233-234 — "Upstream decodes with
librosa through its data parser"
src/vllm/multimodal/dots3_note_processor.cpp:526-528 — "upstream resamples in its data
parser ... with librosa"
.agents/specs/dots3-note.md ## Owed — "accepts whatever librosa can open"
Correct all three. Also stale in that same ## Owed entry: "a windowed-sinc resampler is a
numerically delicate port of its own" — true as difficulty, but it is no longer a blocker,
since #2583 landed Ltx2ResampleWaveform.
Not in scope
The sample-rate arm (W7c-2, a recorded divergence). The container arm, which is leaving this
row entirely — it needs a vendored demuxer, libsndfile alone reports 26 formats, and four
other places already want the same capability.
Row:
MODEL-MM-dots3-note-dots3-note-for-causal-lmW7c-1 is the CHANNELS half of W7c, split out because it is an exact mirror with no oracle
risk, while the sample-rate half is an unavoidable divergence. Keeping them apart stops one
claim borrowing the other's credibility.
What is refused today, and why it is nearly free to lift
The audio path accepts PCM16 mono WAV at 16 kHz. A stereo or multi-channel PCM16 WAV
already at 16 kHz is refused only for want of a mean over channels.
Upstream reduces to mono in two places, both plain means:
vllm/multimodal/media/audio.py:207-208 @ 9035151d6—np.meanover channelswhen
mono=True, which isload_audio's default (:220);vllm/multimodal/audio.py:46-56and:91-161 @ 9035151d6—ChannelReduction.MEANis the default andAudioSpec.target_channelsis 1, whichcommon/processor.py:523-525selects for dots3.So this is a mirror, roughly ten lines, and it stops refusing a whole class of real files.
Scope
Widen
DecodeWavPcm16Mono(src/vllm/multimodal/audio_processor.cpp:35-79), or add achannel-aware sibling beside it, to accept multi-channel PCM16 at the target rate and reduce
by mean. Nothing else changes.
Reachability
Production entry point
ApiServer::handle_chat_completions→InstallMultiModalChatSeam(
server_main.cpp:1565) →MakeDots3NoteChatSeam→RouteDots3NoteAudioWav(
mm_chat_dots3note.cpp:210).The smallest failing test already exists and must invert. The SUBCASE
"a STEREO WAV names the container refusal and W7c" at
tests/vllm/entrypoints/openai/test_api_server_dots3_mm_forward.cpp:1379asserts the refusaltoday. It must go true-before / false-after, which is the ownership test AGENTS.md asks for.
Replace it with a case asserting the stereo request SERVES and that its logprobs equal those
of an independently computed per-sample mean.
In flow, in the same change: three statements about the oracle are false
librosaoccurs exactly ONCE at9035151d6, in a comment atvllm/multimodal/audio.py:31about
get_duration. It is not on the decode or resample path at all. The real chain issoundfile/libsndfile first, PyAV/ffmpeg as fallback, and libswresample for the resample.
Three places in this tree say otherwise, and two of them are production error strings a
user reads:
src/vllm/entrypoints/openai/mm_chat_dots3note.cpp:233-234— "Upstream decodes withlibrosa through its data parser"
src/vllm/multimodal/dots3_note_processor.cpp:526-528— "upstream resamples in its dataparser ... with librosa"
.agents/specs/dots3-note.md## Owed— "accepts whateverlibrosacan open"Correct all three. Also stale in that same
## Owedentry: "a windowed-sinc resampler is anumerically delicate port of its own" — true as difficulty, but it is no longer a blocker,
since #2583 landed
Ltx2ResampleWaveform.Not in scope
The sample-rate arm (W7c-2, a recorded divergence). The container arm, which is leaving this
row entirely — it needs a vendored demuxer, libsndfile alone reports 26 formats, and four
other places already want the same capability.