community_models: VibeASR ternary I2_S LM decoder and asr family (4/4)

[XsquirrelC]

Stacked on #445 (which is stacked on #440, which is stacked on #438). This branch contains all of those plus one commit; only vibeasr: add ternary I2_S LM decoder graph, loader, and session (5a2f688) is new here.

This is the last piece of the VibeASR.cpp port. #438 added the types and fused ops, #440 the I8_S VAE encoder, #445 the ternary matmul kernel. This one puts the Qwen2 decoder on top and wires up a family, so --task asr --family vibeasr transcribes end to end on CPU:

$ ./build/bin/audiocpp_cli --task asr --family vibeasr \
    --model models/vibeasr/vibeasr-lm-i2_s-embed-q6_k.gguf \
    --model-spec-override model_specs --backend cpu --threads 8 \
    --audio assets/asr_validation/librispeech/librispeech_test_clean_6930-75918-0000.wav --metrics
text_output=Concord returned to its place amidst the tents.
metrics.wall_ms=1284.65
metrics.rtf=0.36652

The decoder needs no new graph code

modules::QwenCausalDecoderModule builds it unchanged. Every projection in it goes through LinearModule, which is a bare ggml_mul_mat on a 2-D flattened activation, so the ternary weight dispatches on type with nothing to opt into — which is what #445 was for. The published checkpoint leaves the embedding table at Q6_K and the output projection at F16, and both go through the framework's normal loading path; only the 196 blk.N.* projection weights need load_i2_s_tensor, which is a dtype/shape check plus store.make_tensor(..., GGML_TYPE_I2_S, ...).

Geometry comes from the LM GGUF's KV block: 28 layers, hidden 1536, intermediate 8960, 12 heads over 2 KV heads, RMSNorm eps 1e-6, RoPE theta 1e6. The checkpoint has no qwen2.attention.key_length, so head_dim falls back to qwen2.rope.dimension_count, cross-checked against head_dim * head_count == embedding_length and against token_embd.weight's actual shape rather than trusted.

Two graphs, both built once per session: a prefill graph sized to the prompt, and a single-step decode graph over a static KV cache. Greedy only — upstream defaults to temperature 0.7 / top-p 0.9 with --greedy as an opt-in, and greedy is what parity is measured against.

Speech features go in through ggml_set_rows

The encoder's two branches both emit 1536-wide features — decoder hidden size — and the reference sums them element-wise. Those rows then replace the <|speech_pad|> slots in the embedded prompt. Doing that overwrite in-graph with ggml_set_rows over the EmbeddingModule output keeps the embedding lookup and the injection in one pass, instead of materializing the table gather on the host.

The prompt itself is Qwen2.5 ChatML matching utils/prompt_builder.h token for token, with two details worth naming:

• The six special tokens are inserted by numeric id (151643–151648), not through the tokenizer, and every text segment is tokenized with parse_special = false. The GGUF vocabulary still carries Qwen2.5's original text for those slots while the embedding rows are the ones VibeVoice trained, so tokenizing the literal text would land on the wrong rows.
• There is deliberately no generation prompt. The model emits its own <|im_start|>assistant\n header, and the session strips that leading triple before decoding, exactly as asr_server.cpp does. Without the strip the transcript comes back as assistant\nConcord returned….

min(pads, frames) is also worth a note: upstream builds ceil(samples / 3200) pad tokens but prefills only as many as the encoder produced frames, so emitting exactly frames pads yields the same sequence.

Packaging

The published package is two GGUFs — 703 MB encoder, 993 MB decoder — plus the tokenizer. A --model directory holding more than one GGUF is rejected by require_selected_source, and a single component GGUF has no embedded spec, so this follows the convention minimax_h3 already documents: --model <one component>.gguf --model-spec-override model_specs. No framework change, and no merged GGUF that would diverge from what upstream publishes.

convert_vibeasr_vae.py is renamed to convert_vibeasr_gguf.py, since the same type-id remap fixes both halves and nothing else. It grew --in-place, --list, and --check.

Parity

Four LibriSpeech clips, greedy on both sides, against VibeASR.cpp's own asr_infer --greedy on the same two GGUFs:

Clip	VibeASR.cpp	this port
test-clean 6930-75918-0000	Concord returned to its place amidst the tents.	identical
test-clean 6930-75918-0001	The english forwarded to the french baskets of flowers, … (43 tokens)	identical
test-other 7902-96591-0001	Don't cry, he said. I was obliged to come.	identical
test-other 7902-96591-0000	I'm from the cut or lying off the coast.	I'm from the cutter lying off the coast.

Three of four match token for token. The fourth diverges because these clips are 16 kHz and the resamplers differ — this port uses audio.cpp's own vibevoice_asr front end (soxr, plus the max_abs > 1 clamp), upstream uses naive linear interpolation and no clamp — which perturbs the encoder features enough to flip one greedy argmax. The reference transcript is I AM FROM THE CUTTER LYING OFF THE COAST, so the port is the one that gets it right, but the point is that the difference is the front end and not the graph. A clip already at 24 kHz skips resampling and does not have this failure mode. Both divergences are called out in the source at the point they happen.

One more faithful-to-a-fault behaviour: output_format=json returns an empty transcript on short single-speaker clips, because the model emits an immediate end-of-turn. I checked upstream on the same input before assuming it was my bug — asr_infer --prompt-format json is also empty. Not papered over, and documented.

Performance

Release, gcc, x86-64 AVX2, 24 vCPU EPYC 7V13. 3.505 s clip resampled to 24 kHz: 26 speech frames, 72-token prompt, 13 generated tokens.

Threads	encoder (both branches)	prefill	decode	wall	RTF
8	758 ms	214 ms	239 ms	1285 ms	0.367
1	4652 ms	1415 ms	941 ms	7081 ms	2.020

The encoder dominates — it runs twice, once per branch, over raw samples rather than tokens. Decode is ~18 ms/token at 8 threads. Peak RSS 2.20 GB against 1.70 GB of weights, because BackendWeightStore stages each tensor before upload; graph arenas are 64 + 256 + 256 MB by default and all three are session options.

Tests

tests/vibeasr/test_vibeasr_asr.cpp, registered as test_vibeasr_asr. It loads through the real registry (ModelLoadRequest with family_hint, not a direct constructor), creates an {Asr, Offline} session, and requires raw equality with the reference transcript — punctuation and casing included. A normalized compare runs only to localize a failure. Exits 125 (SKIP) when the checkpoint is not installed, same as the encoder probe, with the message pointing at the weights and the converter.

test_vibeasr_vae_encoder also loses its hard 16 kHz rejection: the encoder is a raw-waveform stack, so it takes whatever rate the clip carries and only the frame count and reported RTF depend on it. That is what lets the probe run on the LibriSpeech clips at their native rate, which is where the 17-frame parity numbers in the doc come from.

Full ctest: 70/70 pass, 0 failures. python3 tools/check_loader_catalog_sync.py reports runtime loaders, model_specs, and model_manager_v2 in sync.

docs/community_models/vibeasr.md is rewritten for the whole pipeline, and the encoder-only qualifications in docs/community_models/models.md and docs/asr.md are updated.

Upstream

• Model port: https://github.com/microsoft/VibeASR.cpp (src/lm.cpp, src/asr_server.cpp, utils/prompt_builder.h)
• Weights: https://huggingface.co/XsquirrelC/VibeVoice-ASR-BitNet

[XsquirrelC]

Closing in favour of #448. You asked for two PRs in microsoft/VibeASR.cpp#10 — one for the additive ggml changes, one for the model integration — so this four-PR stack has been reorganized into exactly that: #447 (ggml) and #448 (model). Same code, no functional change; sorry for the churn.