Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Floating-point ops counting and reloading #40

Merged
merged 24 commits into from
Sep 15, 2021
Merged

Floating-point ops counting and reloading #40

merged 24 commits into from
Sep 15, 2021

Commits on Jul 22, 2021

  1. initial flo count/logging setup (need to fix model parameter count)

    @TevenLeScao
    TevenLeScao committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    5196f8a View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2021

  1. initial flo count/logging setup (need to fix model parameter count)

    @TevenLeScao
    TevenLeScao committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    a3a4ba4 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2021

  1. 1B3 parameter setup + flos counting

    @TevenLeScao
    TevenLeScao committed Jul 28, 2021
    Configuration menu
    Copy the full SHA
    bdd75f1 View commit details
    Browse the repository at this point in the history

  2. 1B3 parameter setup + flos counting

    @TevenLeScao
    TevenLeScao committed Jul 28, 2021
    Configuration menu
    Copy the full SHA
    c0fc29a View commit details
    Browse the repository at this point in the history

  3. 1B3 parameter setup + flos counting

    @TevenLeScao
    TevenLeScao committed Jul 28, 2021
    Configuration menu
    Copy the full SHA
    aefbe3b View commit details
    Browse the repository at this point in the history

  4. 1B3 parameter setup

    @TevenLeScao
    TevenLeScao committed Jul 28, 2021
    Configuration menu
    Copy the full SHA
    17e0184 View commit details
    Browse the repository at this point in the history

  5. 1B3 parameter setup

    @TevenLeScao
    TevenLeScao committed Jul 28, 2021
    Configuration menu
    Copy the full SHA
    97dd06d View commit details
    Browse the repository at this point in the history

  6. synched with latest 13B script

    @TevenLeScao
    TevenLeScao committed Jul 28, 2021
    Configuration menu
    Copy the full SHA
    64892e2 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2021

  1. synched with latest 13B script

    @TevenLeScao
    TevenLeScao committed Jul 29, 2021
    Configuration menu
    Copy the full SHA
    3c79aac View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2021

  1. pipe transformer docstring

    Shaden Smith authored and TevenLeScao committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    b7b3167 View commit details
    Browse the repository at this point in the history

  2. improve DS integration evaluation + logging

    Shaden Smith authored and TevenLeScao committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    8382141 View commit details
    Browse the repository at this point in the history

  3. use pp engine even for pp=1 (#6)

    @jeffra
    jeffra authored and TevenLeScao committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    06cb18f View commit details
    Browse the repository at this point in the history

  4. removed slurm_examples

    TevenLeScao committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    d581894 View commit details
    Browse the repository at this point in the history

  5. flos re-loading

    @TevenLeScao
    TevenLeScao committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    60794bf View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2021

  1. Merge branch 'main' into training_flos

    @TevenLeScao
    TevenLeScao committed Aug 24, 2021
    Configuration menu
    Copy the full SHA
    c79db1c View commit details
    Browse the repository at this point in the history

  2. Update megatron/training.py 

    Co-authored-by: Thomas Wang <24695242+thomasw21@users.noreply.github.com>
    @TevenLeScao @thomasw21
    TevenLeScao and thomasw21 committed Aug 24, 2021
    Configuration menu
    Copy the full SHA
    fb33f13 View commit details
    Browse the repository at this point in the history

  3. Update megatron/data/gpt_dataset.py 

    Co-authored-by: Thomas Wang <24695242+thomasw21@users.noreply.github.com>
    @TevenLeScao @thomasw21
    TevenLeScao and thomasw21 committed Aug 24, 2021
    Configuration menu
    Copy the full SHA
    dff1479 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2021

  1. Update megatron/utils.py 

    Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
    @TevenLeScao @stas00
    TevenLeScao and stas00 committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    2fa3b5b View commit details
    Browse the repository at this point in the history

  2. Update megatron/utils.py 

    Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
    @TevenLeScao @stas00
    TevenLeScao and stas00 committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    ff7af10 View commit details
    Browse the repository at this point in the history

  3. formatting fix, reserving bug for somewhere else, adding flo-logging … 

    …to TB groups
    @TevenLeScao
    TevenLeScao committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    b9ac381 View commit details
    Browse the repository at this point in the history

  4. indentation bug

    @TevenLeScao
    TevenLeScao committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    f25e25f View commit details
    Browse the repository at this point in the history

  5. fixing possible double counts

    @TevenLeScao
    TevenLeScao committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    e63503d View commit details
    Browse the repository at this point in the history

  6. tweaks

    @TevenLeScao
    TevenLeScao committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    5bdcf81 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2021

  1. warning for double counts

    @TevenLeScao
    TevenLeScao committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    72ad711 View commit details
    Browse the repository at this point in the history