Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add triton implementation of layer norm #260

Draft
wants to merge 34 commits into
base: main
Choose a base branch
from
Draft

Add triton implementation of layer norm #260

wants to merge 34 commits into from

Commits on Sep 6, 2023

  1. add triton implementation of layer norm

    @epwalsh
    epwalsh committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    25086b6 View commit details
    Browse the repository at this point in the history

  2. add note about source

    @epwalsh
    epwalsh committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    495cd73 View commit details
    Browse the repository at this point in the history

  3. reorganize

    @epwalsh
    epwalsh committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    f8854ac View commit details
    Browse the repository at this point in the history

  4. decorate tests

    @epwalsh
    epwalsh committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    31b4ed9 View commit details
    Browse the repository at this point in the history

  5. add missing params to test

    @epwalsh
    epwalsh committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    aeeb623 View commit details
    Browse the repository at this point in the history

  6. add bfloat16 test

    @epwalsh
    epwalsh committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    6dddf66 View commit details
    Browse the repository at this point in the history

  7. clean up tests

    @epwalsh
    epwalsh committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    978d29a View commit details
    Browse the repository at this point in the history

  8. more dtypes

    @epwalsh
    epwalsh committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    eb8b1d8 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2023

  1. adjust test tolerance

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    4f1e66c View commit details
    Browse the repository at this point in the history

  2. fix

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    d3c565d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into petew/triton

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    f5cf624 View commit details
    Browse the repository at this point in the history

  4. add triton build script

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    7718032 View commit details
    Browse the repository at this point in the history

  5. upload to s3

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    cafa951 View commit details
    Browse the repository at this point in the history

  6. clean up tests

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    ea2ba9c View commit details
    Browse the repository at this point in the history

  7. increase tolerance

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    54d0a14 View commit details
    Browse the repository at this point in the history

  8. add lumi test script

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    ace4266 View commit details
    Browse the repository at this point in the history

  9. change names

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    add5f7a View commit details
    Browse the repository at this point in the history

  10. fix lints

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    a9066e1 View commit details
    Browse the repository at this point in the history

  11. shorten names

    @epwalsh
    epwalsh committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    8344dc5 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. Merge branch 'main' into petew/triton

    @epwalsh
    epwalsh committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    e1f54fc View commit details
    Browse the repository at this point in the history

  2. Merge branch 'petew/layer-norm' into petew/triton

    @epwalsh
    epwalsh committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    4c5c6cd View commit details
    Browse the repository at this point in the history

  3. add TritonLayerNorm class

    @epwalsh
    epwalsh committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    0a06206 View commit details
    Browse the repository at this point in the history

  4. fix merge conflicts

    @epwalsh
    epwalsh committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    84df0f2 View commit details
    Browse the repository at this point in the history

  5. clean up

    @epwalsh
    epwalsh committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    6ce3ed3 View commit details
    Browse the repository at this point in the history

  6. update build script

    @epwalsh
    epwalsh committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    dd5628d View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. Merge branch 'main' into petew/triton

    @epwalsh
    epwalsh committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    7415a72 View commit details
    Browse the repository at this point in the history

  2. add note about clearing out the cache

    @epwalsh
    epwalsh committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    95547f1 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2023

  1. try no affines anywhere

    @epwalsh
    epwalsh committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    570dd77 View commit details
    Browse the repository at this point in the history

  2. add warning

    @epwalsh
    epwalsh committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    ffb697a View commit details
    Browse the repository at this point in the history

  3. auto remove triton cache

    @epwalsh
    epwalsh committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    17d27db View commit details
    Browse the repository at this point in the history

  4. fix cleaning triton cache

    @epwalsh
    epwalsh committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    9b7519c View commit details
    Browse the repository at this point in the history

  5. add triton LN with linear

    @epwalsh
    epwalsh committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    63bb3e5 View commit details
    Browse the repository at this point in the history

  6. fix

    @epwalsh
    epwalsh committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    940ea7b View commit details
    Browse the repository at this point in the history

  7. add back elementwise affine to QK norm

    @epwalsh
    epwalsh committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    2ce967d View commit details
    Browse the repository at this point in the history