Skip to content

GPTFast 0.3.0

Choose a tag to compare

View all tags
@MDK8888 MDK8888 released this 21 Jun 01:51
GPTFast-0.3.0
60ab0b8
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
  • GPTQ INT4 quantization available for all HF models
  • Accelerates inference speed by 7.6x-9x
  • Integrates optimized INT4 matrix multiplication kernels from the PyTorch team for all HF models
Assets 2
Loading