Skip to content

TensorFold 1.3.1

Choose a tag to compare

@ashhart ashhart released this 20 Nov 13:39
· 1239 commits to main since this release

TensorFold 1.3.1

1.3.1 is 1.3.0 plus native Hy4-flash on Metal. Nothing else changes: every other model, flag and default is as in 1.3.0.

New

  • Hy4-flash on Metal: tensorfold-native serve takes Tencent's Hy4-flash (hy_v4_flash) 4- and 8-bit MLX checkpoints (TensorFold/Hy4-flash-MLX-4bit-MTP, TensorFold/Hy4-flash-MLX-8bit-MTP) with DFly2 drafts and contexts to 256K tokens (128K by default). Past 2,048 tokens the compressed indexer recalls 2,048 keys a sparse layer, and a following prompt that starts with the last one's tokens prefills only the rest. Drafted output equals plain decode, and a resumed prompt gives the same tokens as a fresh one (#3941, refs #3797).
  • Kernel contract: Hy4-flash's Metal kernels (indexer, MLA projections and attention, hyper-connections, routing, experts, row windows and Engram) and its lane projection variants register under gates 1 to 5, each group with a planted control. Building the records found and fixed a row-window library that failed to compile on Apple chips before the M5 (#4022).

What changes by default

  • Nothing for existing models: the shared Metal code Hy4-flash adds (an 8-bit register layout, in-kernel group sums, mapped expert gathers) is used only by Hy4-flash, and the other models' kernels are built from the same source text as in 1.3.0.
  • Hy4-flash was measured on an M5 Ultra; other Apple chips build its kernels but were not run with the model.