Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

optimize generation caching #12

Merged
merged 1 commit into from Nov 3, 2021
Merged

optimize generation caching #12

merged 1 commit into from Nov 3, 2021

Commits on Nov 3, 2021

  1. optimize generation caching 

    Over 10x speedup, adds MLP caching and optimizes attention caching.
Uses changes from https://t.co/BTwo6NKq9H.
    @neverix
    neverix committed Nov 3, 2021
    Configuration menu
    Copy the full SHA
    56590a5 View commit details
    Browse the repository at this point in the history