Skip to content

0.0.35

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 May 23:54
· 69 commits to master since this release
v0.0.35
c0b20f6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
  • Tensor parallel mode for Qwen3.5/3.6
  • New recurrent state manager avoid dynamics allocation of recurrent states (reduces fragmentation and keeps VRAM overhead constant for Qwen3.5 etc.)
  • Improved checkpointing decision to make better use of available cache space
  • Perform reconstruct-GEMM in slices for large layers, greatly reducing VRAM overhead
  • Fix race condition causing streaming token output to lag behind generation in some situations
  • Support new tensor keys in Mistral 3.5 Medium
  • KL-div and perplexity kernels for eval scripts
  • More tuning
  • Lots of bugfixes

Full Changelog: v0.0.34...v0.0.35

Assets 52
Loading