Skip to content

v0.1.20: CUDA Backend Support

Choose a tag to compare

View all tags
@farukciftler farukciftler released this 29 Jul 15:57
· 7 commits to main since this release
v0.1.20
e968416

What's Changed

  • Added CUDA backend support ( feature) to and
  • Implemented device-aware KV cache budget (8 GB for CUDA)
  • Added precheck validation for GGUF quantization formats supported on CUDA
  • Updated diagnostic reports (, ) to dynamically show active backend

Full Changelog: v0.1.19...v0.1.20

Assets 11
Loading