v0.7.0

Two themes. llama-server b10621 compatibility is complete, with every one of 376 pinned options, routes and request fields carrying a terminal state and nothing deferred. And a re-measurement campaign across two Apple Silicon hosts found a dtype-promotion class that had been widening half-precision hidden states to f32 in six places, worth up to 3.26x on the checkpoints it reached.

Embeddings and reranking, DeepSeek-V4, and the Inkling family across text, image, video and audio also land in this release.

New Features

• llama-server b10621 compatibility, complete (epic #1431). The pinned release (commit c1d0e7a0, published 2026-08-25) was reduced to 376 manifest entries across 18 shards under compat/llama-server/b10621/: 169 supported, 22 aliased onto an mlxcel spelling, 35 recorded as a deliberate divergence with a written rationale, 150 not applicable to a runtime that does not execute GGUF. A CI gate fails the build if an entry regresses. Covers HTTP transport, TLS, CORS and multi-key authentication (#1462, #1465), the native completion, embedding, tokenization, template and infill routes (#1467, #1480), the sampler remainder and a GBNF grammar engine (#1487, #1506), prompt cache, batching, context retention and YaRN (#1468, #1503, #1522), router mode (#1495, #1501), unfused LoRA adapters with live scales (#1497, #1520), mp3 and flac transcription streamed per token (#1523), embedding and reranking mode and pooling (#1483), speculative flags mapped to MTP and DFlash (#1488), the Vertex AI predict adapter (#1493), /props, /slots, /metrics, /health and slot persistence (#1492, #1524), idle sleep (#1528, #1530), and assistant prefill, echo and streamed thinking tags (#1526, #1527). Full map in docs/llama-server-compat.md.
• Embeddings through mlxcel embed and POST /v1/embeddings (#1408, #1410-#1416). BERT/XLM-RoBERTa, ModernBERT, SigLIP text, EmbeddingGemma, Qwen3 and Qwen3-VL, bidirectional Llama/Nemotron/LFM2.5, Llama-Nemotron-VL, and ColBERT-style multimodal late-interaction models. Pooling, L2 normalization, Matryoshka dimensions, token and image inputs, bounded worker admission and per-checkpoint length limits are shared by the CLI and server paths, and the server can load an embedding checkpoint alone or beside a chat model with --embedding-model.
• Reranking through mlxcel rerank and the Cohere/Jina-compatible POST /v1/rerank (#1417). One-label BERT/XLM-RoBERTa/ModernBERT cross-encoders, Qwen3 generative rerankers and Qwen3-VL multimodal rerankers.
• DeepSeek-V4 (deepseek_v4) (#523). HyperConnections replace the plain residual stream, attention runs one shared 512-wide KV head across 64 query heads inside a 128-token rotating window with per-head softmax sinks, per-layer compression picks local, pooled or sparse-compressed attention, sparse layers add a HiSA indexer, and MoE routing hashes the first three layers by token id. Validated against mlx-community/DeepSeek-V4-Flash-4bit (43 layers, about 151 GB on disk) with a strict weight-coverage check at load and three real-checkpoint generation gates.
• The Inkling family across four modalities (#1532, #1535, #1540, #1546, #1548). Hybrid sliding/global NoPE attention, learned banded relative-position logits, four f32 short-convolution states per layer, and logsigmoid-normalized routed and shared experts, loading original bf16/f32 weights, native ModelOpt NVFP4 expert planes, and pre-converted affine MLX 4-bit experts. On top of it: HMLP image input with the reference 40x40 tiler, adjacent-frame video under one request-wide 16-pair budget, dMel audio, and a native chained-MTP drafter.
• IQuest-Coder (iquestcoder) and text-only Youtu-LLM (youtu, youtu_llm) (#1594, #1593), both new ModelType arms on existing loaders with no new architecture code.
• Locally typical sampling (typical_p) and top-n-sigma logit filtering (#1482, #1479), applied as row-wise graph transformations before the single fused sampling dispatch so requests stay on the batched path.
• mlxcel inspect --json and mlxcel arch --json (#1605, #1606, #1608). The first emits the memory estimator's byte fields as a stable contract; the second derives a per-family registry of runtime, modality, backend, distributed, drafter and KV facts. make recipes-registry writes a versioned snapshot under recipes/registry/.
• A live runtime settings endpoint (#1516), per-request speculative acceptance on the response (#1588), tool_choice accepts required and a named function (#1319), Qwen-VL video input and Responses-native image parts (#1510, #1310), and LFM2-VL image splitting (#1405).
• Chat Completions reasoning under both common field names (#1308) and OpenAI-shaped reasoning controls driving chat-template thinking (#1307). reasoning_effort, Responses reasoning.effort and compatible extra-body values derive enable_thinking.
• A generation that goes entirely to the reasoning channel says so (#1721). mlxcel generate suppresses the <think> channel by default, so a reasoning model whose generation ends before the channel closes printed a blank line, which is indistinguishable from a broken checkpoint. Both the one-shot flow and the chat REPL now say how many tokens went there. scripts/ab_output_equality.sh is the output half of an A/B built on the same reading.

Improvements

• Half-precision paths keep their dtype. A helper that builds its constants as f32 returns f32 for a half-precision input; the widened activation enters the residual stream at the first layer that uses it, and every later matmul promotes its own weight to match. Nothing is arithmetically wrong, which is why unit tests, output comparisons and perplexity all pass over it. Five bridge activation helpers were doing this (#1709), which took gpt_bigcode-santacoder from 28% to 90% of mlx-lm (51.23 to 164.63 tok/s) and pythia-1b from 31% to 97%; the multimodal and 3D rotary paths dropped the restore upstream performs (#1714), worth up to 2.45x on M1 Ultra and 3.26x on M5 Max; Gemma3n held two dtypes in one checkpoint and now loads uniformly bf16, moving decode from 78% of mlx-vlm to 97%; and Mistral 4, Ministral 3 (#1711) and the GraniteMoeHybrid gated norm (#1723) each carried a local copy.
• Pre-Ampere CUDA gets three arms it did not have. bf16 weights convert to f16 at load (#1542), since no pre-Ampere part has a bf16 ALU. qmm_naive gets a Volta tensor-core path (#1543): prefill goes from 138 to 536 tok/s on qwen3.8-27B-4bit and 326 to 1579 on gemma-4-12B-it-4bit, against scalar-path ceilings of 259 and 583. qmv blocks output rows per warp, dividing activation traffic while leaving weight traffic and per-row arithmetic untouched: gemma-4-12B-it-4bit decode 44.7 to 57.4 tok/s, generated text byte-identical at --temp 0. Separately, qmv accumulates bf16 in float below Ampere at every bit width (#1539), which is 1.87x on qwen3.8-27B-4bit and 1.73x on gemma-4-12B-it-4bit on a V100, and qmm_naive sizes its CTA tile against the device's real shared-memory budget instead of an architecture proxy (#1541).
• apply_token_bias no longer synchronizes the device on every decode step. It read the pre-bias argmax id back to the host to feed two suppression counters, which breaks the async lookahead pipeline. Measured on M1 Ultra as the delta between --ignore-eos and no flag: +65.4% decode time on qwen3-0.6b-4bit, +30.4% on llama-3.2-1b-4bit, +24.0% on qwen3-30b-a3b-4bit, +13.6% on llama-3.1-8b-4bit. Behind MLXCEL_LANG_BIAS_COUNTERS=1 the same deltas are within 1.2% of zero. Only requests using logit_bias, ignore_eos or --lang-bias were affected.
• Constrained decoding keeps the grammar mask packed (#1578). The token bitmask stays in u32 form from the matcher to the GPU. On the Qwen3.8-27B geometry that removes two 248k-element host loops and 97% of the per-step host-to-device copy, on every tick of every constrained sequence.
• The GQA repeat is gone from the fused SDPA paths (#1686), Qwen2-VL takes the text-only path when no image is in play, grouped softcap-GQA decode is on by default, Qwen3 MoE batches the decode forward instead of one forward per row (#1616), and the Mamba2 hybrids drop two M5-gated eval boundaries (#259, #266, #1685).
• Chat templates are compiled once and cached (#1518), response stores are bounded by bytes rather than entry count (#1519), and the JSON body limit is derived from the image budget (#1515) instead of a fixed number that either truncated valid image requests or admitted oversized ones.
• mlxcel serve carries the help sections that only mlxcel-server had (#1628), and a test asserts that every flag named under src/commands appears in some --help (#1131).
• OpenXLA compiles a set of context capacities and routes each request to the smallest that fits (#1302).

Bug Fixes

• Half-precision reductions in the bridge accumulate in f32 (#997). max over a bfloat16 array returned NaN for a finite input on M5 Max, about one call in six. Reading the same logits row back to the host found every element finite on 60 of 60 runs while the device reduction over that row reported non-finite on 28, with no overlap. Measured one variant per process over 150 forwards: 23 non-finite for the plain reduction, 31 reshaped to 1-D, 20 made contiguous, 24 reduced over an axis, 0 cast to f32, so element width is the trigger and contiguity is not.
• Four RoPE scaling defects across shared decoder paths. Qwen3 ignored a configured rope_scaling (#1398), the shared Llama 3 path dropped the scaled frequencies, Gemma 3 skipped the linear factor on global-attention layers, and InternLM doubled positions under dynamic NTK. Phi-3 selects its LongRoPE table by whole-prompt position (#1580) rather than by the current chunk.
• nemotron_nas resolves llama3 RoPE scaling through the shared reader, rotates after the head transpose, and reads eos_token_id from the config (#1687, #1688). The family had never run: before the first fix it emitted a single "I", after it a fluent loop, and only with both does it answer, at 99% of mlx-lm on the same machine.
• recurrent_gemma builds every layer and gets its conv and RoPE layouts right (#1687). It was registered and could not generate text at all.
• Qwen2.5-VL runs its vision tower norm in f32 and restores raster order correctly (#1601, #1582), and Youtu-VL emits patches in merge-block-major order after the windowed tower's inverse permutation was corrected (#1600, #1610).
• MiniCPM-V upscales an image that arrives below the scale resolution (#1684), which had been giving it 16 vision tokens against the reference 64. Inkling expands KV heads for GQA under the per-head mask (#1549).
• Phi and StableLM carry the upstream f32 attention guard (#1710). phi-2-4bit went NaN at layer 29 of 32 in f16 and then emitted ! for every token, reproducible in bare MLX.
• CUDA grouped GEMM no longer tells CUTLASS a Volta part is a Turing part (#1544), and CUDA JIT kernel configuration is serialized (#1566), closing a divide-by-zero.
• A malformed MLXCEL_PAGED_SLAB_BLOCKS falls back to the derived slab size, as its warning always claimed (#1137), and speculative-burst responses report prompt_ms and predicted_ms the way classic decode does (#1592), where predicted_per_second had read 95000 on a 96-token DFlash request.
• An absolute file:// URL resolves under --media-path (#1612), matching b10621, where it had returned HTTP 400 since request media was confined.
• mlxcel_core::is_gpu_available() is renamed default_device_is_gpu(), and gpu_backend_available() answers whether a GPU backend exists (#1421). The old name read as a hardware query but reported whether the MLX default device was currently the GPU.
• Falcon-Mamba applies the B/C/dt RMS norm once (#1574), recurrent decode state is persisted across requests (#1513), KV modes are applied to model-owned caches (#1400), LoRA adapters that do not map onto the loaded model are refused (#1576), XML tool-call arguments are typed by the request's own schema (#1575), and chat-template failures surface instead of being swallowed (#1511, #1394).

CI/CD Improvements

• The Volta sm_70 baseline is recorded and gated in CI (#1556), so the pre-Ampere arms cannot regress unmeasured, and CUDA compute capability is visible to the runtime (#1551).
• CI links an OpenXLA binary so a link-only regression cannot reach main (#1305), and its warning backlog is cleared with warnings denied (#1381).
• The Python client is tested across CPython 3.9 through 3.13 (#1572).
• The real-model integration tests resolve checkpoints again (#1722, #1726). repo_model_dir looked only under models/<name>, so none of the 30 checkpoint names resolved under the current store layout and every real-model test skipped while reporting 2 passed ... finished in 0.00s. Elapsed time is what separates a passing gate from a gate that ran nothing.
• The benchmark harness gained three guards after each of them let a wrong answer through: it refuses to run when the source tree is newer than the benchmark binary, it rejects VLM rows where the image never reached the prompt, and it fails an empty sweep instead of writing an empty CSV and reporting success.

Technical Details

• Both Apple Silicon hosts were re-swept on a fixed pp512/tg128 shape against same-host, same-day mlx-lm 0.31.3 and mlx-vlm 0.6.17 runs. Text decode is a median 99% of mlx-lm on M5 Max over 60 pairs and 100% on M1 Ultra over 110; VLM decode is 105% and 108% of mlx-vlm over 47 pairs each, none below 90%. The earlier 2.78x and 1.79x prefill figures are retired: they ran on a short natural prompt whose token count differed between the two runtimes, and prefill_tok_s is prompt tokens over prefill time, so the tokenization difference landed directly in the ratio.
• The split in the text roster is architectural. Mixture-of-experts checkpoints run at a median 106% of mlx-lm on M5 Max (19 pairs) while dense ones sit at 100% (69 pairs), including qwen3.8-27b-4bit, mixtral-8x7b and the dense Gemma 4 QAT variants.
• Sixty model directories carried a name that disagreed with the checkpoint inside them. docs/model-catalog.md documents the naming rule and docs/model-catalog.tsv records what each name rests on; scripts/checkpoint_fingerprint.py fingerprints the shards a checkpoint actually loads, so a cross-host row can be paired against the weights rather than the directory name. docs_internal/ is retired and its one surviving column moved into the catalog.
• docs/benchmarks.md gains the conditions for judging a change that moves the numbers: gate on disagreement at decided positions in a teacher-forced logit trace rather than byte-identity or perplexity, run the arm with the change reverted and confirm it fails, trace at the width the code runs at, and check that the input reaches the branch that changed.

Dependencies

• hyper 1.11.0 to 1.11.1 and smallvec 1.15.2 to 1.16.0 (#1682), and uuid 1.24.1 to 1.26.0 (#1529).

Breaking Changes

• --models-dir changes meaning (#1495). On mlxcel serve and mlxcel-server, --models-dir and LLAMA_ARG_MODELS_DIR now select b10621 router-mode discovery. The mlxcel store root that spelling used to name is --model-store-root. A command line carrying both logs a migration diagnostic at startup.
• --timeout and LLAMA_ARG_TIMEOUT change meaning (#1432). They are now the HTTP socket read/write timeout with b10621's 3600-second default. The per-request decode watchdog is --decode-timeout / MLXCEL_DECODE_TIMEOUT, with its 600-second default unchanged.
• Unsupported llama-server options now fail at startup (#1463, #1469, #1481, #1489). GGUF model-source semantics, GGML runtime flags and KV quantizers, projector flags and control-vector options are refused with a diagnostic naming the option, where they were previously accepted and ignored.
• Pre-Ampere CUDA output changes (#1539). Float accumulation is strictly more accurate than the emulated bf16 accumulation it replaces, so quality improves, but greedy decode on a Volta host is no longer token-identical to a build from before this change. sm_80 and later are untouched, with byte-identical emitted SASS.
• Gemma3n loads uniformly bf16 instead of keeping only its language MLP at bf16, so its resident weight footprint changes. MLXCEL_KEEP_BF16 is the A/B instrument.
• mlxcel_core::is_gpu_available() is deprecated (#1421) and remains as a shim for one release. Use default_device_is_gpu() or gpu_backend_available().

Known Issues

• Youtu-VL still describes multi-window images wrong (#1618). #1600 and #1610 corrected the token order and the patch order, and the only fixture that is answered correctly is the one that fits its merged grid inside a single attention window. max_num_patches from the preprocessor is still ignored (#1611).
• Compressed-container transcription reaches only the families on the shared decoder (Phi-4 Multimodal, Gemma 3n). Gemma 4 Unified, Qwen3-Omni, Nemotron-H Nano Omni and standalone Whisper use a family-local WAV reader, so mp3 and flac fail during model preparation for those.
• Qwen-VL text decode still falls below mlx-lm as context grows (#1686). The dtype work above closed most of the gap on the image path; the remaining context-dependent term is open.
• mlx-community/afm-4.5b does not load (#1719). It declares model_type: arcee with ArceeForCausalLM, and the binary carries only the AFMoE / Trinity MoE variant under that label.
• The Inkling-VL mixed-prefill scatter test fails intermittently under the full suite and passes in isolation (#1622). Both sides of the comparison are the same call sequence, so a load-dependent numerical error would be a production defect rather than a test defect.
• On Volta, qmv roofline attainment rises from 7.27% to 14.80% of 900 GB/s, short of the 25% #1539 asked for.
• #1065 is open: f32 matmul and single-query SDPA precision correction. #1217 is open: M1 Ultra and M3 Ultra MTP re-measurement, and x86_64 CUDA verification.

What's Changed

• fix(server): give preemption victim selection a total order by @inureyes in #1301
• feat(xla): compile a set of context capacities and route requests between them by @inureyes in #1302
• chore(ci): link an OpenXLA binary in CI so link-only regressions cannot reach main by @inureyes in #1305
• chore: clear the OpenXLA warning backlog and deny all warnings in CI by @inureyes in #1381
• fix(server): make the chat-template tojson filter match json.dumps by @inureyes in #1382
• fix(llama3): apply rope_scaling frequencies on the shared Llama path by @inureyes in #1385
• fix(gemma3): apply the linear rope_scaling factor on global-attention layers by @inureyes in #1386
• fix(internlm): stop doubling RoPE positions under dynamic NTK scaling by @inureyes in #1389
• fix(server): stop the prompt cache donating and adopting KV-less shadow paged entries by @inureyes in #1390
• fix(sampling): filter on the untempered distribution and scale by temperature last by @inureyes in #1391
• fix(server): return None for missing template map keys by @inureyes in #1394
• chore(test): raise dev test stack for Make targets by @inureyes in #1396
• chore(ci): derive cross-repo ref boundaries live by @inureyes in #1397
• fix(core): refuse quantized KV modes on MLA latent caches and perform the announced Turbo4 fallback by @inureyes in #1395
• fix(qwen3): apply configured RoPE scaling by @inureyes in #1398
• update: audit shapeless MLX compile sites by @inureyes in #1401
• fix: apply KV modes to model-owned caches by @inureyes in #1400
• docs: fix docs/README.md intro and index separators by @VedantMadane in #1393
• fix: align SmolVLM split image framing by @inureyes in #1402
• fix: keep Gemma4 Unified vision overlay with audio by @inureyes in #1403
• fix: parse Python repr tool-call values (#1306) by @inureyes in #1404
• feat: add LFM2-VL image splitting by @inureyes in #1405
• fix: disable ThinLTO in test-fast by @inureyes in #1407
• feat(embeddings): pooling, masks, Embedding kind and /v1/embeddings by @inureyes in #1408
• docs: add technical report for PR #1408 by @inureyes in #1409
• feat(siglip): serve the SigLIP text tower through /v1/embeddings by @inureyes in #1410
• feat(modernbert): port the ModernBERT encoder for /v1/embeddings by @inureyes in #1412
• feat(embeddings): EmbeddingGemma and Qwen3-Embedding forward passes by @inureyes in #1413
• feat(bert): port BERT and XLM-RoBERTa encoders for /v1/embeddings by @inureyes in #1411
• feat(embeddings): ColIdefics3 and ColQwen2.5 late-interaction embedders by @inureyes in #1414
• feat(embeddings): bidirectional Llama, Nemotron-3-Embed and LFM2.5 by @inureyes in #1415
• feat(embeddings): Qwen3-VL-Embedding and Llama-Nemotron-VL-Embed by @inureyes in #1416
• feat(server): /v1/rerank with three reranker kinds by @inureyes in #1417
• fix(tests): pin the GPU default device in the shared embedding test guard by @inureyes in #1420
• docs: add the embedding and rerank throughput ladder for GB10 by @inureyes in #1427
• fix: harden side-model request boundaries by @inureyes in #1428
• docs: Refresh embedding and reranking CLI guidance by @inureyes in #1429
• fix: align llama-server b10621 compatibility by @inureyes in #1454
• feat(models): port DeepSeek-V4 as a net-new architecture (#523) by @inureyes in #1455
• test(server): add pinned b10621 compatibility manifest and gates by @inureyes in #1458
• fix(server): accept space-separated negative values on server flags by @inureyes in #1461
• feat(server): align HTTP transport semantics with llama-server b10621 by @inureyes in #1462
• feat(server): apply b10621 RoPE runtime overrides before model construction by @inureyes in #1464
• fix(server): match llama-server b10621 multi-key authentication by @inureyes in #1465
• feat(server): separate the native completion and embedding routes by @inureyes in #1467
• fix(server): translate model-source flags and reject GGUF semantics by @inureyes in #1463
• feat(server): implement b10621 prompt-cache and batching semantics by @inureyes in #1468
• fix(server): reject unsupported GGML runtime flags and KV formats by @inureyes in #1469
• fix(server): align chat-template, reasoning, and output-parsing semantics by @inureyes in #1471
• fix(compat): reconcile manifest ownership and fix trace perplexity by @inureyes in #1474
• fix(server): honor string stop sequences on the MLX serving path by @inureyes in #1475
• feat(server): honor response_fields, stream_options and real prefill timings by @inureyes in #1476
• fix(compat): move the native completion remainder to #1477 by @inureyes in #1478
• feat(server): add b10621 tokenize, template, and infill utilities by @inureyes in #1480
• feat(sampling): add top-n-sigma logit filtering with a row-filter hook by @inureyes in #1479
• fix(server): classify projector flags and confine request media by @inureyes in #1481
• feat(server): align b10621 embedding and reranking mode and pooling by @inureyes in #1483
• feat(sampling): add locally typical sampling (typical_p) by @inureyes in #1482
• fix(server): align the b10621 audio transcription route by @inureyes in #1484
• fix(server): give the image cardinality route test a model that takes images by @inureyes in #1486
• feat(inference): complete b10621 sampling semantics and windows by @inureyes in #1487
• feat(server): map b10621 speculative flags to MTP and DFlash semantics by @inureyes in #1488
• feat(server): explicitly reject b10621 control-vector options by @inureyes in #1489
• feat(server): align b10621 logging, introspection, and preset flags by @inureyes in #1490
• feat(server): implement b10621 resumable streams and completion control by @inureyes in #1491
• feat(server): align props, slots, metrics, health, and slot persistence by @inureyes in #1492
• feat(server): add Vertex AI (GCP) predict route adapter for b10621 by @inureyes in #1493
• feat(server): classify b10621 UI, tools, MCP, CORS-proxy and agent surfaces by @inureyes in #1494
• feat(server): implement b10621 router mode and resolve models-dir semantics by @inureyes in #1495
• feat(server): implement multi-adapter LoRA request and hot-swap semantics by @inureyes in #1497
• fix(server): point the cache-list test at the store-root spelling by @inureyes in #1498
• feat(compat): add the by_design b10621 compatibility-policy state by @inureyes in #1500
• feat(server): give the b10621 router a model cache source and INI presets by @inureyes in #1501
• fix(distributed): stabilize pipeline LRU eviction test by @inureyes in #1502
• feat: map reasoning controls to template kwargs by @inureyes in #1505
• feat: accept Responses-native image input parts (#1310) by @inureyes in #1504
• feat: gate boundary snapshots by model capability by @inureyes in #1507
• feat: add configurable reasoning response alias by @inureyes in #1508
• feat(models): add Qwen-VL video input by @inureyes in #1510
• fix(server): size snapshot cache by model by @inureyes in #1509
• fix: stop swallowing template rejections by @inureyes in #1511
• test: make TP crossover assertion deterministic by @inureyes in #1512
• fix(models): persist recurrent decode state by @inureyes in #1513
• fix(bench): use binary byte units in KV reports by @inureyes in #1514
• fix(audio): share bounded real FFT frontend by @inureyes in #1517
• fix: derive JSON body limit from image budget by @inureyes in #1515
• feat(server): add live runtime settings endpoint by @inureyes in #1516
• perf: cache compiled chat templates by @inureyes in #1518
• fix(server): bound response stores by bytes by @inureyes in #1519
• feat(server): close the b10621 context, batching and YaRN runtime gaps by @inureyes in #1503
• refactor: split batch scheduler modules by @inureyes in #1521
• feat(server): close the b10621 prompt-cache and slot-scheduling gaps by @inureyes in #1522
• feat(server): serve b10621 LoRA adapters unfused with live scales by @inureyes in #1520
• feat(server): decode mp3 and flac and stream ASR per token by @inureyes in #1523
• fix(server): bind slots on first progress and settle the b10621 slot classifications by @inureyes in #1524
• fix(server): report b10621 tokens_cached and accept verbose on the native route by @inureyes in #1525
• feat(server): implement the b10621 sampler remainder and the GBNF grammar engine by @inureyes in #1506
• feat(server): serve the b10621 assistant prefill and echo, and settle reasoning_in_content by @inureyes in #1526
• feat(server): honor the b10621 native generation fields and stream the thinking tags by @inureyes in #1527
• feat(server): sleep the serving worker on idle and wake it on the next request by @inureyes in #1528
• deps(deps): bump uuid from 1.24.1 to 1.26.0 in the minor-and-patch group by @dependabot[bot] in #1529
• fix(server): harden b10621 generation bounds and idle sleep by @inureyes in #1530
• test(deepseek-v4): run the HiSA hierarchy on real weights, add its kill switches (#549, #550) by @inureyes in #1460
• test(deepseek-v4): demote the HiSA decode-scaling test from gate to report by @inureyes in #1533
• feat(models): add Inkling text backbone by @inureyes in #1532
• docs: add Inkling text backbone report by @inureyes in #1534
• feat(vision): add Inkling HMLP image support by @inureyes in #1535
• feat(speculative): add Inkling native MTP by @inureyes in #1540
• fix(server): derive the thinking_mode sentinel from the chat template by @inureyes in #1547
• feat(vision): add Inkling adjacent-frame video support by @inureyes in #1546
• feat(audio): add Inkling dMel input support by @inureyes in #1548
• chore(core): expose CUDA compute capability to the mlxcel runtime by @inureyes in #1551
• docs: add technical report for PR #1547 by @inureyes in #1552
• fix(server): derive primed-open thinking from tokenizer markers by @inureyes in #1554
• refactor: flatten Inkling audio chunk concatenation by @inureyes in #1555
• test(benchmark): record the Volta sm_70 baseline and gate sm_70 in CI by @inureyes in #1556
• perf(cuda/quant): accumulate qmv in float below Ampere for bf16 by @inureyes in #1557
• perf(cuda/quant): size the qmm_naive tile from the shared-memory budget by @inureyes in #1559
• fix(models): accept an Inkling sub-config that carries a field and its alias by @inureyes in #1561
• fix(cuda/moe): tag the pre-Ampere grouped GEMM arm Sm70, not Sm75 by @inureyes in #1562
• perf(cuda): explain the Volta fixed first-token cost, and instrument it by @inureyes in #1565
• docs: quote Python extras installs under zsh (#1222) by @inureyes in #1568
• fix: stabilize nightly verify-test failures (#1256) by @inureyes in #1567
• docs: correct stale "not a server knob" claim in turbo-kv-cache.md by @inureyes in #1571
• test(ci): run Python client CI across a 3.9-3.13 version matrix by @inureyes in #1572
• chore(build): guard webpage-build against missing MkDocs sources by @inureyes in #1570
• chore(execution): parse memory sizes with one shared grammar by @inureyes in #1573
• fix(mamba): apply Falcon-Mamba B/C/dt RMS norm once, no per-call ones by @inureyes in #1574
• fix(server): type XML tool-call arguments by the request schema by @inureyes in #1575
• fix(lora): refuse adapters that do not map onto the model by @inureyes in #1576
• perf(server): pack structured-output masks as u32 bitmasks by @inureyes in #1578
• fix(phi3): select the LongRoPE table by whole-prompt position by @inureyes in #1580
• fix(qwen2_5_vl): normalize the raw HF patch-embed layout at load by @inureyes in #1582
• feat(server): report per-request speculative acceptance by @inureyes in #1588
• feat(models): route text-only Youtu-LLM to its MLA decoder by @inureyes in #1593
• feat(models): add the IQuest-Coder (iquestcoder) route by @inureyes in #1594
• refactor(cli): deduplicate image-loading closures into open_image by @reiden21 in #1591
• docs: replace opaque (A4)/(B11) task codes in code comments by @reiden21 in #1586
• fix(bench): reject --target paths without a file name by @cestercian in #1569
• feat(server): enforce required and named tool_choice on chat (#1319) by @adrysn in #1581
• docs: sync PR template clippy and test commands with CONTRIBUTING.md by @reiden21 in #1583
• chore(cli): unify error-message capitalization across CLI commands by @reiden21 in #1590
• docs: fix garbled comments left by issue-reference scrubbing by @adrysn in #1579
• docs: correct the MLXCEL_DEVICE row in installation.md by @reiden21 in #1585
• docs(bench): clean scrub artifacts in benchmark_results by @reiden21 in #1584
• refactor(loading): remove dead is_special_weight_model_type wrapper by @reiden21 in #1587
• refactor(server): remove unused build_failed_response helper by @reiden21 in #1589
• fix(server): define tools only when the request sends them by @inureyes in #1598
• fix(core): separate GPU backend availability from the default device by @adrysn in #1595
• fix(qwen2_5_vl): f32 tower norm and a correct window inverse by @inureyes in #1601
• feat: add JSON inspect output by @inureyes in #1605
• feat: add arch JSON recipes registry by @inureyes in #1606
• docs: add technical report for PR #1606 by @inureyes in #1607
• fix: include RT-DETRv2 in arch JSON registry by @inureyes in #1608
• fix(youtu_vl): build the window inverse as argsort by @rapsealk in #1603
• fix(youtu_vl): emit patches in merge-block-major order by @inureyes in #1619
• fix(server): resolve absolute file:// URLs under --media-path by @inureyes in #1620
• feat(bench): dispatch speculative_bench MTP past Gemma 4 Unified by @inureyes in #1621
• perf(qwen3_moe): batch the decode forward instead of per-row forwards by @inureyes in #1623
• docs: add recipe request issue form by @inureyes in #1624
• docs: add CMake to the CONTRIBUTING.md minimum development environment by @agatha197 in #1642
• docs(examples): fix dead link, placeholder, and stale in-progress claim in surgery examples by @agatha197 in #1643
• docs: document four missing MLXCEL_* server variables in environment-variables.md by @agatha197 in #1640
• docs(python): document the LLM/AsyncLLM constructor parameters and server_kwargs by @agatha197 in #1646
• docs(python): link the python/examples scripts from the README and client docs by @agatha197 in #1645
• chore(cli): port the Model store and Embeddings/Reranking help sections to mlxcel serve by @agatha197 in #1650
• docs: update the SECURITY.md supported-versions table to the 0.7.x line by @agatha197 in #1639
• docs: describe all six make verify gates in CONTRIBUTING.md by @agatha197 in #1641
• docs: point --distributed-config examples at the checked-in TOML templates by @agatha197 in #1644
• fix(server): stamp the speculative burst's first token at its prefill end so timings splits prompt from decode by @rapsealk in #1609
• refactor: deduplicate the byte-size formatters by @agatha197 in #1651
• test: assert every flag named in src/commands exists in some --help by @rapsealk in #1604
• fix(server): let a malformed MLXCEL_PAGED_SLAB_BLOCKS fall back to the derived slab size by @rapsealk in #1602
• chore(build): surface maintenance targets in make help and accept digit/dot target names by @agatha197 in #1648
• docs(cli): describe the .sock rule instead of the deprecated --port 0 socket spelling by @agatha197 in #1647
• fix(core): name --kv-bits in the bits validation error by @agatha197 in #1652
• test(cli): add parse tests for the infill and batch KV quant flag groups by @agatha197 in #1649
• fix(cuda): serialize JIT kernel configuration to stop a divide-by-zero by @inureyes in #1655
• perf(cuda): convert bf16 weights to f16 at load on pre-Ampere GPUs by @inureyes in #1656
• perf(cuda/quant): add a Volta tensor-core arm to qmm_naive by @inureyes in #1678
• perf(cuda/quant): block output rows per warp in qmv below Ampere by @inureyes in #1681
• deps(deps): bump the minor-and-patch group with 2 updates by @dependabot[bot] in #1682
• bench: refresh the 0.7.0 sweeps and fix the dtype, rotary and rope defects they exposed by @inureyes in #1617
• fix(reductions): accumulate half-precision reductions in f32 by @inureyes in #1718
• fix(core): clear the two clippy errors failing nightly verify by @inureyes in #1720
• fix(tests): resolve checkpoints under the current store roots by @inureyes in #1722
• docs(models): record the two RMS-norm A/B verdicts that changed no code by @inureyes in #1724
• feat(cli): say when a generation went entirely to the reasoning channel by @inureyes in #1721
• perf(granitemoehybrid): run the gated RMSNorm in the input dtype by @inureyes in #1723
• bench: measure the checkpoints neither host had benchmarked, and fix the harness defects that hid it by @inureyes in #1725
• fix(tests): point real-model tests at the names the store now uses by @inureyes in #1726
• bench: close the M5 Max commit gap and correct two false doc claims by @inureyes in #1727
• bench: re-sweep both VLM tables at one commit and guard empty sweeps by @inureyes in #1729
• docs(bench): correct the M5 Max pair count cited from the M1 Ultra page by @inureyes in #1730

New Contributors

• @VedantMadane made their first contribution in #1393
• @reiden21 made their first contribution in #1591
• @cestercian made their first contribution in #1569
• @adrysn made their first contribution in #1581
• @rapsealk made their first contribution in #1603
• @agatha197 made their first contribution in #1642

Full Changelog: v0.6.0...v0.7.0