vMLX v1.3.34

Post-v1.3.33 fixes driven by user reports and a dedicated-machine test matrix across 5 models.

SSM Deferred Re-derive (Hybrid SSM + Thinking Models)

For thinking models on hybrid SSM architectures (Nemotron, Qwen3.5-VL), the post-generation SSM state was contaminated by thinking tokens and previously skipped entirely — causing 100% SSM companion cache miss on every multi-turn request.

Now the scheduler queues a deferred re-derive that runs during idle time: a separate prefill pass on just the prompt tokens to capture clean SSM state. The current conversation still pays full re-prefill, but the next conversation with the same prompt prefix gets instant KV + SSM cache hits.

Verified on Nemotron-Cascade-2-30B: 23 SSM layers stored per idle period, future requests hit.

Smelt Fixes

• pixel_values kwarg crash: Smelt-loaded model wrappers reject pixel_values entirely — now only passed when non-None
• Panel sessions.ts: --is-mllm suppressed when smelt active (was sending both flags for saved VLM sessions)

Gemma 4 Prefix Cache Fix

_get_n_kv_heads() now scans cfg.text_config as a nested fallback. Gemma 4 VLM stores num_key_value_heads inside model.config.text_config, not on model.config directly — validator was finding num_attention_heads=16 first and forcing 100% cache miss.

Issue #56 Fixes

• JIT + MLLM: mx.compile() now targets language_model.model only for VLM engines, preserving the wrapper's .config attribute
• Error surfacing: Prefill failures now finish_reason="error" with traceback instead of silent stop
• dtype normalization: input_ids → mx.int32, pixel_values → mx.array via _ensure_mx_array() — fixes Mistral 3 / Pixtral batched VLM

Smelt Benchmarks (Clean Machine)

Nemotron-Cascade-2-30B-A3B-JANG_4M, M3 Ultra / 128 GB, dedicated (no parallel models):

--smelt-experts	Active RAM	Decode tok/s	RAM saving
off (baseline)	17,408 MB	89.9	—
50	9,529 MB	66.5	-45%
25	5,590 MB	*	-68%

All configurations coherent, no loops.

Test Matrix

Model	Feature	Result
Nemotron Cascade 30B	SSM re-derive	23 layers stored, future hits ✓
Nemotron Cascade 30B	Multi-turn cache	hits=3, 192 tokens saved ✓
Qwen3.5-4B hybrid SSM	Rope state reset	Zero broadcast errors ✓
Gemma-4-26B VLM	Image red/green	Recognized ✓
gemma-4-e2b-it-4bit	VL 4 colors	red/blue/green/yellow all 2 tokens ✓
Nemotron smelt 50%	RAM + coherence	9.5 GB, 66.5 tok/s ✓

PyPI: https://pypi.org/project/vmlx/1.3.34/