Skip to content

GPTFast 0.2.0

Choose a tag to compare

View all tags
@MDK8888 MDK8888 released this 02 Apr 04:16
GPTFast-0.2.0
7653cca
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
  • Inference speeds are now accelerated by 6-8.5x
  • Static key-value caching is now enabled for all Hugging Face models
  • Support for generic sampling functions in addition to argmax
  • Debugged speculative decoding
Assets 2
Loading