Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Further improvements to attention backward #170

Merged
merged 4 commits into from
Apr 18, 2024
Merged

Further improvements to attention backward #170

merged 4 commits into from
Apr 18, 2024

Commits on Apr 18, 2024

  1. significantly faster kernel by reusing data in registers

    @ngc92
    ngc92 committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    314661f View commit details
    Browse the repository at this point in the history

  2. another 15% by getting rid of ifs

    @ngc92
    ngc92 committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    6d98af1 View commit details
    Browse the repository at this point in the history

  3. splitting the loop to get rid of the indicator

    @ngc92
    ngc92 committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    09d35a5 View commit details
    Browse the repository at this point in the history

  4. updated main training script

    @ngc92
    ngc92 committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    0d55421 View commit details
    Browse the repository at this point in the history