spec(MODEL-MUSIC-MUSIC3): scope MiniMax-Music3, the first row whose oracle is not vLLM (#672) - #679
Merged
Merged
Conversation
…racle is not vLLM (#672) MiniMaxAI/MiniMax-Music3 is text-to-music: lyrics plus a structured description in, a multi-minute stereo song out. It is absent from the pinned vLLM (no source files; the registry carries only the MiniMax M2/M3 TEXT architectures), absent from vLLM `main`, and absent from vllm-omni. So it is the first row to exercise the fallback-oracle rule added in #647, and its primary oracle is an OPEN diffusers PR. W0 ONLY. Spec committed before implementation, per AGENTS.md. No src/, include/ or tests/ change is in scope here; W1 onward are dispatched to fresh implementers from this spec rather than written in the coordinating session. GEOMETRY MEASURED, NOT INFERRED. Every component config.json read, plus the safetensors headers by HTTP range request -- the same no-payload technique LTX-2.5 L0 used. What that corrected, twice: * transformer is 231 tensors / 1.240B params per shard, ALL F32. The card's "2.4B" is right and the 9.73 GB on disk is fp32 storage, NOT a 4.9B bf16 model. Reading the byte count alone would have sized it at double. * condition_encoder has FOUR tensors -- layer_scale, layer_weight_logits and a proj. It is a learned weighted mix over 8 LLM hidden layers, not the encoder tower its name suggests. Also measured: rvq_depth_decoder 47 tensors / 0.646B BF16, vocoder 121 / 0.054B F32 carrying weight_g/weight_v, so weight-norm must be folded at load. THE LARGEST BRICK IS ALREADY BUILT. language_model is `Qwen3ForCausalLM` at 36L / 4096 / 32 heads / 8 KV / head_dim 128 -- our LANDED architecture exactly -- retrained on a 200 000-entry music vocabulary. Vocabulary size is a config value, not an architecture change. TWO PACKAGINGS, ONE SET OF WEIGHTS, and that had to be established rather than assumed, because the two oracles disagree about which one they serve. SGLang-Omni loads ONLY the native arm: checkpoint.py:35-56 resolves qwen_7B/qwen_7B, flowmatching_vae.pth and dav.pth, and pulls the RVQ decoder out of the Qwen shards by the model.audio_decoder. prefix. diffusers loads the six modular components. convert_minimax_music3_to_diffusers.py@c6da9936 :29-38,47-189 settles it -- load the three native artefacts, rename tensors, save. A pure re-layout, no numerical step. Therefore the diffusers arm is the port target (~28.5 GB resident, no pickle, no trust_remote_code, per-component gateable) AND SGLang-Omni stays a valid e2e cross-check instead of an incomparable second model. W1 verifies that equality against actual tensors rather than inheriting it from this reading. THE DTYPE POLARITY IS UPSTREAM'S. The conversion script defaults --dtype float32 (:267), gives transformer/condition_encoder/vocoder that dtype (:208-211) and forces the RVQ decoder to bf16 regardless (:214) -- matching the measured headers exactly, and SGLang-Omni's README confirms both placements run the acoustic stage in FP32. So fp32 there is a resolved upstream choice, not the too-wide accident AGENTS.md warns a token gate cannot see. It is mirrored, and annotated as this. ONE CONTRADICTION LEFT OPEN ON PURPOSE. The model card and SGLang-Omni's README say 32 kHz stereo; every config says 44100 (vocoder sampling_rate, and condition_encoder output_sampling_rate with hop 512). Both cannot be right and the difference is a resample -- which no tensor-parity check on the latents would catch, so it is exactly the class of defect that survives a green gate. W0 resolves it from source before ANY waveform golden is captured. It is recorded as unresolved rather than guessed. ORACLES PINNED. diffusers advances from UNPINNED to the PR #14456 head c6da9936 -- the head SHA and not the branch name, because huggingface:minimax-music3-integration can be rebased under us and "whatever the branch was that day" is not reproducible. SGLang-Omni gets its OWN record at 748a0b43: it is a third repository with its own cadence and its own registry, and folding it into sglang.md would have hidden that a model can be in one and in neither of the others. Both stay gateable = no -- read, never executed here -- which is the honest state and what W0 exists to change. GATE SHAPE, stated up front because conflating the halves is the failure mode: the LLM half emits discrete RVQ codes and gets a real token-exact gate; the acoustic half is a flow-matching loop with no logits and no sampler, so it gets per-stage tensor parity at fixed seed and reduced dimensions, following H3. A correlation coefficient is NOT a gate there -- Pearson is scale-invariant, so a uniformly scaled latent passes it while sounding wrong. Records: MODEL row ratchet 370 -> 371 (the test that mutates it to expected-1 keeps the bump load-bearing), checklist entry + rollup SPIKE 8 -> 9, the 11 non-pin architectures become 12, roadmap issue table, and a claim file. Gate: scripts/agent-preflight.sh -- all checkers and all suites ok EXCEPT test_cpu_x86_llamacpp_floor, a contention-sensitive harness that failed identically (NO_QUIET_WINDOW, exit 4) on an unmodified tree at base SHA cefacd2 under load average 35. Pre-existing and environmental, verified rather than assumed. Issue: #672, #647 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…atchet collided at 371 Both sides moved the same pin on the same day and BOTH read 371. `main` took it 370 -> 372 for IndexTTS-2.5, which vLLM-Omni registers as two architectures (#634); this branch took it 370 -> 371 for MiniMax-Music3 (#672). Neither side was wrong about its own change and neither number was 373, so an automatic three-way merge keeping either side would have produced a tree that is INTERNALLY CONSISTENT while silently short a real architecture -- the state no count assertion can see, because a count only knows the pin matches the rows it can still find. Resolved by re-deriving every count rather than taking a side, per the AGENTS.md rule on keyed records: scripts/check-agent-record.py MODEL 373, both bump comments kept model-matrix.md 14 non-pin rows / 369 architectures; rollup 373 = 323 INVENTORIED + 50 engaged; SPIKE 9 docs/FEATURES.md theirs' TTS row supersedes the stale one this branch carried; the MUSIC row is keyed separately and additive test_music3_and_indextts_rows_both_survive_their_collision names all three rows, mirroring the assertion #633 added for the identical ENGINE collision at 153. Proven load-bearing rather than asserted: with the pin put back to 371 and the two IndexTTS rows deleted -- the wrong-but-consistent resolution -- the count checker still passes its own arithmetic and the new test FAILS. Tree restored and re-verified green afterwards. One incident worth recording, since the recovery is the interesting part. A mid-merge mutation used /tmp/claude-1000 for its restore point; that path is shared between sessions, the backup was clobbered, and restoring from it left check-agent-record.py EMPTY. Recovered with `git checkout --merge`, which is why the conflict was resolved twice. Backups for a mutation belong in the session-private scratchpad, and git is the only restore point that cannot be clobbered by someone else's run. Issue: #672, #634, #633 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…Y, not a contradiction (#672) W0 owed this from source before any waveform golden could be captured. Resolved, and the answer is that both numbers were right about different points in the pipeline. THE VOCODER NATIVELY EMITS 44100 Hz STEREO, and it is derivable rather than merely declared -- which matters, because a declared sample rate in a config is exactly the kind of value that goes stale without anything noticing. The condition encoder's output_sampling_rate 44100 / output_hop_length 512 set a latent frame rate of 86.133 Hz (condition_embedder_minimax_music3.py:40-41), the decoder applies one ConvTranspose1d per upsampling_ratios entry for 8*8*4*2 = 512x (minimax_music3_vocoder.py:84,92-95), and 86.133 * 512 = 44100. The declared sampling_rate matches its own convolution stack. SGLang-Omni's independent implementation lands on the identical numbers (dav.py:94,115), which is the strongest form of confirmation available here: two codebases that were not copied from each other. DIFFUSERS RETURNS THAT UNRESAMPLED (modular_pipeline.py:32-36, decoders.py:84-92). SGLANG-OMNI'S SERVER RESAMPLES 44100 -> 32000 ON DELIVERY (constants.py:18-19, acoustic.py:55-58,422-431). So "32 kHz stereo" in the model card and in SGLang-Omni's README is neither wrong nor stale: it is the reference SERVER's delivery rate. diffusers' own docs state the split in as many words. DECISION: goldens are captured at 44100 stereo -- the model's native generative rate, resample-free, and what the primary oracle hands the caller. The 32 kHz form is a downstream delivery transform, gated separately if SGLang-Omni byte parity is ever wanted. That transform is NOT free, and the spec says so rather than leaving it to be discovered: acoustic.py:58 passes no lowpass_filter_width, rolloff or resampling_method, so reproducing those bytes means reproducing torchaudio's DEFAULT sinc filter, not converting 44.1 -> 32 by any correct method. And the reason this had to be settled before the first golden, not after: a latent-tensor parity check sits entirely upstream of the resample call and cannot see the difference. A golden captured at the wrong rate would be wrong by a resample while every tensor gate stayed green -- the class of defect that survives a green gate, which is the one this project spends its evidence discipline on. Also corrected: the 24000/960 pair in condition_encoder/config.json is the AR stage's 25 Hz frame rate, unrelated to output. It reads like a third sample rate and is not one. W0 now owes only the oracle stand-up. The §1.1 stop condition is removed because it is discharged; the W5 row names 44100 stereo instead of "the resolved rate". Issue: #672 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
localai-bot
force-pushed
the
row/MODEL-MUSIC-MINIMAX-MUSIC3
branch
from
August 13, 2026 22:18
670b1d4 to
64b080e
Compare
This was referenced Aug 14, 2026
localai-bot
added a commit
that referenced
this pull request
Aug 14, 2026
… the ACTIVE contract (#672) (#723) W0 is done and W1 has landed, so the row's state was stale at `SPIKE` while real loader code sat in the tree. Advancing it is what surfaced two things worth recording rather than quietly fixing. THE ACTIVE CONTRACT IS ENFORCED, AND THE SPEC DID NOT MEET IT. Moving the row to `ACTIVE` turned check-agent-record red on nine missing structured fields -- Scope, Upstream chain, Our baseline, Port map, Tests to port, Gates, Dependencies, Work breakdown, Risks/decisions. A `SPIKE` spec is not held to that; an `ACTIVE` one is, which is the checker doing exactly its job. My first attempt added them as PROSE SECTIONS and stayed red on all nine, including the two the spec already had -- because the checker does not read headings at all, it reads TABLE ROWS whose first cell names the field. Fixed by writing the structured record in the shape `minimax-h3.md` already uses, rather than by arguing with the gate. Worth stating plainly: the failure was mine reading the requirement as prose, and the checker's message was right both times. SECOND, A WORDING FIX THAT COST TWO IMPLEMENTERS. §3 said both pins "go into `.agents/oracles/` in W0", which reads as future work. `sglang-omni.md` has existed since #679, yet two separate agents reported it as still owed -- the same agent twice. A record surface described in the future tense reads as a TODO no matter what the tree holds, so it is now present tense with links, plus a note saying why. That is a small edit that stops a recurring false report. WHAT THE ROW NOW CLAIMS, and nothing beyond it: oracle gateable with 13 committed per-stage goldens (#708), modular loader gated 1413/1413 against the real 27 GB checkpoint with all 1012 tensors accounted and the native arm refused by name (#714). Checklist mark 📋 -> 🚧; rollup SPIKE 9 -> 8, ACTIVE 9 -> 10. The lifecycle change owes STATUS, BENCHMARKS and the spec's `## Now`, and all three say the same thing the row does. BENCHMARKS records every axis PENDING with NONE OWED YET and says why: W0/W1 are correctness scaffolding, the oracle capture was CPU-only because dgx.casa was down, and there is no forward pass to time. A row that has landed code is exactly where a speed claim tends to appear by implication, so the absence is written down rather than left to be inferred. Both public rows were first written over the house-style limits (a 402-char BENCHMARKS cell, and STATUS at 45 oversized cells against a 44 ratchet that may only shrink). Shortened to fit rather than lowering a ratchet to accommodate prose. Issue: #672 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #672. W0 only: spec committed before implementation, per AGENTS.md. No
src/,include/ortests/engine change — W1 onward are dispatched to freshimplementers from this spec.
MiniMaxAI/MiniMax-Music3istext-to-music: lyrics plus a structured description in, a multi-minute stereo
song out. It is absent from the pinned vLLM (no source files; the registry
carries only the MiniMax M2/M3 text architectures), absent from vLLM
main, andabsent from
vllm-omni. It is the first row to exercise the fallback-oracle ruleadded in #647.
Geometry measured, not inferred
Every component
config.jsonread, plus the safetensors headers by HTTP rangerequest — the no-payload technique LTX-2.5 L0 used. It corrected the model card
twice:
transformercondition_encoderlayer_scale,layer_weight_logits,proj— a learned weighted mix over 8 LLM layers, not the encoder tower its name suggestsrvq_depth_decodervocoderweight_g/weight_v⇒ weight-norm must be folded at loadThe largest brick is already built.
language_modelisQwen3ForCausalLMat36L / 4096 / 32 heads / 8 KV / head_dim 128 — our LANDED architecture exactly —
retrained on a 200 000-entry music vocab. Vocabulary size is a config value, not
an architecture change.
Two packagings, one set of weights
This had to be established, not assumed, because the two oracles disagree about
which one they serve. SGLang-Omni loads only the native arm
(
checkpoint.py:35-56:qwen_7B/qwen_7B,flowmatching_vae.pth,dav.pth,pulling the RVQ decoder out of the Qwen shards by the
model.audio_decoder.prefix). diffusers loads six modular components.
convert_minimax_music3_to_diffusers.py@c6da9936:29-38,47-189settles it: loadthe three native artefacts, rename tensors, save. A pure re-layout, no numerical
step. So the diffusers arm is the port target (~28.5 GB resident, no pickle,
no
trust_remote_code, per-component gateable) and SGLang-Omni stays a valide2e cross-check rather than an incomparable second model. W1 verifies that
equality against actual tensors instead of inheriting this reading.
The dtype polarity is upstream's
The conversion script defaults
--dtype float32(:267), givestransformer/condition_encoder/vocoder that dtype (
:208-211) and forces the RVQdecoder to bf16 regardless (
:214) — matching the measured headers exactly, andSGLang-Omni's README confirms both placements run the acoustic stage in FP32. So
fp32 there is a resolved upstream choice, not the too-wide accident AGENTS.md
warns a token gate cannot see. Mirrored, and annotated as such.
One contradiction left open on purpose
The model card and SGLang-Omni's README say 32 kHz; every config says
44100 (
vocodersampling_rate,condition_encoderoutput_sampling_ratewith hop 512). Both cannot be right, and the difference is a resample — which no
tensor-parity check on the latents would catch, so it is exactly the class of
defect that survives a green gate. W0 resolves it from source before any
waveform golden is captured. Recorded as unresolved rather than guessed.
Oracles pinned
diffusersadvances fromUNPINNEDto PR#14456 head
c6da9936—the head SHA, not the branch name, because
huggingface:minimax-music3-integrationcan be rebased under us.748a0b43: a third repository with itsown cadence and registry. Folding it into
sglang.mdwould hide that a modelcan be in one and in neither of the others.
Both stay
gateable = no— read, never executed here. That is the honest stateand what W0 exists to change.
Gate shape, stated up front
The LLM half emits discrete RVQ codes and takes a real token-exact gate. The
acoustic half is a flow-matching loop with no logits and no sampler, so it takes
per-stage tensor parity at fixed seed and reduced dimensions, following H3.
A correlation coefficient is not a gate there: Pearson is scale-invariant, so
a uniformly scaled latent passes it while sounding wrong.
The merge collision, and why it needed a test
Both branches moved the MODEL ratchet on the same day and both read 371:
maintook 370 → 372 for IndexTTS-2.5 (two architectures, #634), this branch took370 → 371 for Music3. Neither number was 373, so an auto-merge keeping either side
leaves a tree that is internally consistent while short a real architecture —
the state no count assertion can see.
Resolved by re-deriving every count.
test_music3_and_indextts_rows_both_survive_their_collisionnames all three rows,mirroring the assertion #633 added for the identical ENGINE collision at 153.
Proven load-bearing: with the pin put back to 371 and the two IndexTTS rows
deleted — the wrong-but-consistent resolution — the count checker still passes its
own arithmetic and the new test FAILS. Tree restored and re-verified green.
Gate
scripts/agent-preflight.sh— all checkers and all suites ok excepttest_cpu_x86_llamacpp_floor, a contention-sensitive harness that failedidentically (
NO_QUIET_WINDOW, exit 4) on an unmodified tree at base SHAcefacd2d0under load average 35. Pre-existing and environmental, verifiedrather than assumed.
check-doc-checkpoint.pyover the PR range: OK.🤖 Generated with Claude Code