Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/came #5491

Closed
wants to merge 21 commits into from
Closed

Feature/came #5491

wants to merge 21 commits into from

Commits on Apr 7, 2024

  1. test CAME under LowLevelZeroOptimizer wrapper

    @chongqichuizi875
    chongqichuizi875 committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    544c6bb View commit details
    Browse the repository at this point in the history

  2. test CAME TP row and col pass

    @chongqichuizi875
    chongqichuizi875 committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    c730970 View commit details
    Browse the repository at this point in the history

  3. test CAME zero pass

    @chongqichuizi875
    chongqichuizi875 committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    dc684c8 View commit details
    Browse the repository at this point in the history

  4. came zero add master and worker param id convert

    @chongqichuizi875
    chongqichuizi875 committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    5a42495 View commit details
    Browse the repository at this point in the history

  5. came zero test pass

    @chongqichuizi875
    chongqichuizi875 committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    036ecb6 View commit details
    Browse the repository at this point in the history

  6. came zero test pass

    @chongqichuizi875
    chongqichuizi875 committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    82b04cb View commit details
    Browse the repository at this point in the history

  7. test distributed came passed

    @chongqichuizi875
    chongqichuizi875 committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    f95d875 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. reform code, Modify some expressions and add comments

    @chongqichuizi875
    chongqichuizi875 committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    291db12 View commit details
    Browse the repository at this point in the history

  2. [feat] Add distributed lamb; minor fixes in DeviceMesh (hpcaitech#5476) 

    * init: add dist lamb; add debiasing for lamb

* dist lamb tester mostly done

* all tests passed

* add comments

* all tests passed. Removed debugging statements

* moved setup_distributed inside plugin. Added dist layout caching

* organize better

---------

Co-authored-by: Edenzzzz <wtan45@wisc.edu>
    @Edenzzzz
    Edenzzzz and Edenzzzz committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    67476a9 View commit details
    Browse the repository at this point in the history

  3. minor fix of test came

    @chongqichuizi875
    chongqichuizi875 committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    0727821 View commit details
    Browse the repository at this point in the history

  4. minor fix of test came

    @chongqichuizi875
    chongqichuizi875 committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    e4606df View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2024

  1. minor fix of dist_came and test

    @chongqichuizi875
    chongqichuizi875 committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    58ec531 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    e7707f0 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. [hotfix] Improve tester precision by removing ZeRO on vanilla lamb (h… 

    …pcaitech#5576)

Co-authored-by: Edenzzzz <wtan45@wisc.edu>
    @Edenzzzz
    Edenzzzz and Edenzzzz committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    447f69e View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2024

  1. minor fix of dist_came and test

    @chongqichuizi875
    chongqichuizi875 committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    924984b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature/came' of github.com:chongqichuizi875/ColossalAI… 

    … into feature/came
    @chongqichuizi875
    chongqichuizi875 committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    7febec7 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'feature/dist-optim' of https://github.com/hpcaitech/Col… 

    …ossalAI into feature/came
    @chongqichuizi875
    chongqichuizi875 committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    ded354b View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2024

  1. rebase dist-optim

    @chongqichuizi875
    chongqichuizi875 committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    bff8c54 View commit details
    Browse the repository at this point in the history

  2. rebase dist-optim

    @chongqichuizi875
    chongqichuizi875 committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    3ea5034 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. fix remaining comments

    @chongqichuizi875
    chongqichuizi875 committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    ec44099 View commit details
    Browse the repository at this point in the history

  2. add test dist came using booster api

    @chongqichuizi875
    chongqichuizi875 committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    93c4a88 View commit details
    Browse the repository at this point in the history