Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[AutoTP] Make AutoTP work when num_heads not divisible by number of workers #4011

Merged
merged 55 commits into from
Oct 25, 2023
Merged

[AutoTP] Make AutoTP work when num_heads not divisible by number of workers #4011

merged 55 commits into from
Oct 25, 2023

Commits on Jul 20, 2023

  1. allow number of heads not divisible by number of ranks

    @delock
    delock committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    0706acd View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2023

  1. get num_heads from model config, more robust

    @delock
    delock committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    0bf785f View commit details
    Browse the repository at this point in the history

  2. simplify logic where num_head itself is sharded

    @delock
    delock committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    72b9e1a View commit details
    Browse the repository at this point in the history

  3. name tweaks

    @delock
    delock committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    5ed9a56 View commit details
    Browse the repository at this point in the history

  4. make code more robust where num_attention_heads may not be defined in… 

    … model_config
    @delock
    delock committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    73f499d View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    48322c7 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    f14e290 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gma/uneven_heads

    @loadams
    loadams committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    b62317c View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2023

  1. support num_key_value_heads < num_attention_heads which is used by ll… 

    …ama2
    @delock
    delock committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    12c0628 View commit details
    Browse the repository at this point in the history

  2. add test for 5 ranks

    @delock
    delock committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    8f23d9b View commit details
    Browse the repository at this point in the history

  3. change odd rank # to 3 to avoid test skip

    @delock
    delock committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    9c53bd7 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into gma/uneven_heads

    @tjruwase
    tjruwase committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    413224b View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    78d6667 View commit details
    Browse the repository at this point in the history

  2. add get_shard_size function

    @delock
    delock committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    27fde30 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2023

  1. modify sharding mechanism according to latest auto TP

    @delock
    delock committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    8e1fd27 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    9a6bc12 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2023

  1. fix accuracy issue

    @delock
    delock committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    2dac94f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    885f6a3 View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @molly-smith
    molly-smith committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    7ffd811 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @tjruwase
    tjruwase committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    40659ba View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2023

  1. fix format

    @delock
    delock committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    71f9f40 View commit details
    Browse the repository at this point in the history

  2. skip tests with fusedqkv

    @delock
    delock committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    db9db6b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    72531c0 View commit details
    Browse the repository at this point in the history

  4. remove skip of fusedqkv tests

    @delock
    delock committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    9d5eae3 View commit details
    Browse the repository at this point in the history

  5. skip test fusedqkv with odd number of ranks

    @delock
    delock committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    25e656d View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2023

  1. support model with n_heads in model_config

    @delock
    delock committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    7f6d7f6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gma/uneven_heads

    @molly-smith
    molly-smith committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    e3a5b77 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Aug 26, 2023
    Configuration menu
    Copy the full SHA
    c9ec881 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2023

  1. fix TestInjectionPolicy::test[fp32-t5]

    @delock
    delock committed Aug 27, 2023
    Configuration menu
    Copy the full SHA
    f5be257 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2023

  1. fix uneven_heads on some fusedqkv types (#12) 

    * odd support fusedqkv

* fix format and clear text
    @inkcherry
    inkcherry committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    b671040 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2023

  1. better fix when activation size cannot be divided by number of heads

    @delock
    delock committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    d59ff22 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gma/uneven_heads_rebase

    @delock
    delock committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    6c3c841 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @molly-smith
    molly-smith committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    58e8b24 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. move tp_shard.py under module_inject

    @delock
    delock committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    4c6b7fa View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    18e1c5d View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2023

  1. Add get_num_kv_heads in tp_shard.py

    @delock
    delock committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    8ef01e2 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9a61fc2 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    74870db View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gma/uneven_heads

    @molly-smith
    molly-smith committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    115cc20 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2023

  1. Refine according to comments

    @delock
    delock committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    0781c41 View commit details
    Browse the repository at this point in the history

  2. remove old comment

    @mrwyattii
    mrwyattii committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    194337f View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    47d84ca View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @mrwyattii
    mrwyattii committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    369eb3e View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. fix bug in getting num_kv_heads

    @delock
    delock committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    567fb9a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gma/uneven_heads

    @molly-smith
    molly-smith committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    47c83ca View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @tjruwase
    tjruwase committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    d194ab0 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Oct 7, 2023
    Configuration menu
    Copy the full SHA
    6db5ddd View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2023

  1. Merge branch 'up-master' into gma/uneven_heads

    @delock
    delock committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    698b62a View commit details
    Browse the repository at this point in the history

  2. support uneven sharding of lm_head tensor parallel

    @delock
    delock committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    d75149f View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    248532d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    a9056fd View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    81bd29f View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    693a9fe View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    4c45a5b View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2023

  1. Merge branch 'master' into gma/uneven_heads

    @delock
    delock committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    a7513e1 View commit details
    Browse the repository at this point in the history