Skip to content

v0.3.0: image input and MTP

Latest

Choose a tag to compare

@pmichalas pmichalas released this 18 Nov 11:44

Image input and faster decoding for SenseNova-Loop-9B on Apple Silicon.

New in v0.3.0

  • Image input: SenseNova's vision tower in MLX (bfloat16), image processor (about 1 MP per image by default) and 3-axis mRoPE positions through the 56-visit loop. scripts/generate_vision.py --model <build> --image photo.jpg --prompt "...". Text-only prompts take exactly the same path as before (bit-identical logits to v0.2.0).
  • MTP speculative decoding: SenseNova's MTP head drafts tokens that the looped model verifies in one pass, with exact Gated DeltaNet rollback per visit slot. 6-bit build on a MacBook Pro M5 Max: 19.3 → 33.5 tok/s at k=3 (1.74×). Greedy output matches plain decoding except at near-ties of the two best tokens.
  • Float32 norm weights: the 81 RMSNorm weights can be loaded in float32 (model.fp32norms.safetensors), as SenseNova's code computes them; bf16 build top-1 agreement with SenseNova's code 98.6% → 99.1%.

Builds on Hugging Face (updated with vision.safetensors, mtp.safetensors, model.fp32norms.safetensors)

6-bit 8-bit 4-bit bf16
Same top token as SenseNova's code (text) 96.7% 98.6% 84.4% 99.1%
Image prompts: same top token as bf16 98.0% 96.7% 95.4% reference

Apache-2.0. Not affiliated with SenseTime / SenseNova.