Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

save cpu mem by leveraging FSDP rank0 broadcasting #77

Merged
merged 14 commits into from
Aug 11, 2023
Merged

save cpu mem by leveraging FSDP rank0 broadcasting #77

merged 14 commits into from
Aug 11, 2023

Commits on Aug 1, 2023

  1. save cpu mem by leveraging FSDP rank0 broadcasting

    @lchu-ibm
    lchu-ibm committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    d8a81bb View commit details
    Browse the repository at this point in the history

  2. replace init_empty_weights with torch.device(meta)

    @lchu-ibm
    lchu-ibm committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    c8d4f38 View commit details
    Browse the repository at this point in the history

  3. Revert "replace init_empty_weights with torch.device(meta)" 

    This reverts commit c8d4f38.
    @lchu-ibm
    lchu-ibm committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    101391f View commit details
    Browse the repository at this point in the history

  4. switch to simpler param_init_fn and meta device init

    @lchu-ibm
    lchu-ibm committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    1e64fc9 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2023

  1. add nightly check for using low_cpu_fsdp mode

    @lchu-ibm
    lchu-ibm committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    895dfce View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2023

  1. address meta-llama#87

    @lchu-ibm
    lchu-ibm committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    e216c6f View commit details
    Browse the repository at this point in the history

  2. fix fsdp construction on low_cpu_fsdp

    @lchu-ibm
    lchu-ibm committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    c19c5c6 View commit details
    Browse the repository at this point in the history

  3. fix meta-llama#90

    @lchu-ibm
    lchu-ibm committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    0c51b47 View commit details
    Browse the repository at this point in the history

  4. further fix meta-llama#90

    @lchu-ibm
    lchu-ibm committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    80a4c36 View commit details
    Browse the repository at this point in the history

  5. add doc example about using low_cpu_fsdp

    @lchu-ibm
    lchu-ibm committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    c453b66 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2023

  1. remove unused import

    @lchu-ibm
    lchu-ibm committed Aug 6, 2023
    Configuration menu
    Copy the full SHA
    1cc9df1 View commit details
    Browse the repository at this point in the history

  2. code cleanup to remove all unused imports

    @lchu-ibm
    lchu-ibm committed Aug 6, 2023
    Configuration menu
    Copy the full SHA
    41ffbca View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. minor code optimization

    @lchu-ibm
    lchu-ibm committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    3d1e9cd View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2023

  1. resolve conflicts

    @lchu-ibm
    lchu-ibm committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    feaa344 View commit details
    Browse the repository at this point in the history