TensorFold 1.0.10

TensorFold 1.0.10

• Macs serve MLX's mixed-width (oQ) checkpoints for Qwen3.8-27B, GLM-5.3-Flash and Qwen3.8-Flash-Next, such as TensorFold/Qwen3.8-27B-oQ4 and -oQ2, TensorFold/GLM-5.3-Flash-MLX-oQ2-MTP and TensorFold/Qwen3.8-Flash-Next-MLX-oQ2-MTP and -oQ3-MTP. Each module loads at its own MLX width, 2 to 8 bits in groups of 32, 64 or 128, and a uniform checkpoint keeps 1.0.9's kernels and launches. GLM-5.3-Flash on two Macs and with expert parallel still takes 4-bit group-64 checkpoints only, and refuses others by name at load (#755, #756, #773, #774). Thanks to @grantoverton, and to @sethforprivacy for the M5 Ultra runs.
• Mixed-width checkpoints get faster kernels on Macs. On the author's M3 Ultra, the 27B's oQ2 decodes 2.0x (code) and 1.6x (chat) faster and prefills 2.4 to 3.3 times faster on the new 2- to 8-bit simd_qmm tiles, oQ4 1.3x; GLM-5.3-Flash's oQ2 decodes 1.22x and 1.13x faster and its oQ4 prefills 1.33 to 1.37 times faster; Flash Next on M1 to M4 decodes its uniform 6-bit checkpoint 1.06x (code) and 1.19x (chat) faster (#784, #786, #788). Thanks to @grantoverton.
• Follow-ups keep their history. The prompt cache keeps a turn's history while that turn's reply and its background prefill are kept, so a regenerated turn, an edited reply or a client that sends the reply back rendered differently resumes instead of reading the whole prompt again. On an M3 Ultra with first turns of 4,500 to 16,000 tokens, the 27B's follow-ups started in 0.14 to 0.98 s instead of 15 to 63 s, and on a GB10 Nemotron's conversations resumed at least as many tokens as before at every step, with the same replies.
• Nemotron's NVFP4 prompts read faster on CUDA: on a GB10, cold prompts of 2k to 32k tokens prefill 5.6% to 8.0% faster than 1.0.9. The expert kernel's prompt rows take their row tile count as a compile-time constant, and every teacher-forced row and served reply matched the build without it bit for bit (#805). Thanks to @Bizuayeu.
• On CUDA, Nemotron with a Living Weights pocket no longer pays for the layers whose pocket has no neuron in use, and a taught layer's pocket runs over many blocks, with every sum in its order. On a GB10, a copy taught one fact decoded 17% faster with drafts off and 5% to 10% faster drafted, and read 2k to 32k-token prompts 8.5% to 10.3% faster, with the same replies (#787). Thanks to @ThinkOffApp.
• Nemotron on CUDA adds each layer's residual and normalizes it in the shared kernels of cuda/norms.cu. The sums run in another order of the same precision, so a reply can part from 1.0.9's at a near tie; teacher-forced loss over seven public texts is unchanged within noise (#796). Thanks to @jschmied.
• cortheon_fetch reads only URLs the conversation has already shown: search results, links on pages it has read, and URLs in the user's, the system prompt's or a tool's messages. A URL that appears only in the model's replies is refused, so a page that steers the model cannot make it build a URL that carries conversation text. The one exception is one named package's registry JSON on PyPI, npm or crates.io.
• For contributors: a CUDA launch that no variant of a captured Triton set matches exactly takes one that assumes less of its arguments, and the exact match still comes first (#794, #791); tools/zig/triton_aot_pack.py writes a captured Triton set from a capture's manifest or from a Recorder launch dump (#795, #792), thanks to @zweck; a host test checks that the CUDA host forwards each lane-backend entry to the family's entry of the same name (#800), thanks to @Bizuayeu.

Behaviour changes:

• Nemotron on CUDA can give a different reply from 1.0.9 at a near tie, from #796's new sum order. In the release gate on a GB10, 4 of 34 cells on the MLX 4-bit checkpoint and 6 of 34 on NVFP4 kept 1.0.9's tokens; drafted, resumed and concurrent replies equal their plain, fresh and solo runs as before.
• Flash Next's uniform 6-bit checkpoint can decode different tokens on Macs without tensor units (M1 to M4), from #788's faster products. FZ_M3_LANES=1 keeps the lane kernels and their tokens. Tensor units keep every kernel and launch.
• A Flash Next pack is rebuilt once after the upgrade, because the pack's identity now covers config.json.
• cortheon_fetch refuses a URL that appears only in the model's replies, and the cortheon object's pages_read counts only pages read, not refused or failed fetches.

Known issues: 1.0.9's still apply.

Not run for this release:

• No M5-generation Mac and no two-Mac pair ran the release build. The oQ set's M5 Ultra runs are @sethforprivacy's, at the commits its entries name.
• Flash Next and GLM-5.3-Flash were not served from the release build. Their oQ changes rest on the contributors' runs on an M3 Ultra and M5 Ultra and on our kernel checks against float64 on an M3 Ultra.
• No RTX 30-series (sm_86) or Blackwell sm_120 GPU ran 1.0.10. Of the 34 CUDA kernel files, 30 are byte for byte 1.0.9's; the four that changed (#787, #796, #805) are built for sm_86, sm_120 and sm_121 but ran only on a GB10.
• GLM's shared rounds under #769's draft pricing were not measured, as in 1.0.9.
• --cortheon was not part of the served gate. Its unit tests and the server's golden tests with the flag off stand in.

Tested: the release build ran against 1.0.9 on an M3 Ultra (Qwen3.8-27B with its DFlash2 drafter, and Nemotron 3.5 Lightning MLX 4-bit) and on a GB10 (Nemotron MLX 4-bit and NVFP4). Drafted replies equal "draft": false, resumed prompts equal fresh ones, concurrent requests and 1 to 8 streams equal their solo runs, and 26 structured-output cases are valid. On the M3 Ultra both models gave 1.0.9's tokens in every case, with decode and prompt speed level, and 64k to 256k-token prompts answered. On the GB10, decode with drafts off is level with 1.0.9.

Homebrew: brew upgrade tensorfold.