Skip to content

1.6.5

Latest

Choose a tag to compare

@github-actions github-actions released this 25 Oct 17:35
  • Support Nemotron-3.5-Super-VL (fix NemotronH config parsing)
  • Reduced VRAM overhead for prefill using cache quantization
  • Lower-latency drafting with hot/sparse vocabulary
  • Faster AVX2 and AVX512-BW inference
  • Faster model loading when offloading experts
  • Support Gemma4 MTP model (Gemma4AssistantForCausalLM)
  • Fix tokenizer regression from 1.6.4 causing prompt caching to fail sometimes
  • Many more fixes and optimizations

Full Changelog: v1.6.4...v1.6.5