Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support gradient accumulation using Horovod's backward_passes_per_step #11911

Merged
merged 21 commits into from
Feb 19, 2022
Merged

Support gradient accumulation using Horovod's backward_passes_per_step #11911

merged 21 commits into from
Feb 19, 2022

Commits on Feb 14, 2022

  1. Support gradient accumulation using Horovod's backward_passes_per_step

    @krshrimali
    krshrimali committed Feb 14, 2022
    Configuration menu
    Copy the full SHA
    f4b1fa1 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Feb 14, 2022
    Configuration menu
    Copy the full SHA
    aeb85ca View commit details
    Browse the repository at this point in the history

  3. Update CHANGELOG.md 

    Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
    @krshrimali @rohitgr7
    krshrimali and rohitgr7 committed Feb 14, 2022
    Configuration menu
    Copy the full SHA
    0e1b93a View commit details
    Browse the repository at this point in the history

  4. Update CHANGELOG.md 

    Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
    @krshrimali @rohitgr7
    krshrimali and rohitgr7 committed Feb 14, 2022
    Configuration menu
    Copy the full SHA
    7db02ca View commit details
    Browse the repository at this point in the history

  5. Add test and minor fix

    @krshrimali
    krshrimali committed Feb 14, 2022
    Configuration menu
    Copy the full SHA
    0bf0fb9 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'feature/11732_grad_accumulation_horovod' of github.com:… 

    …krshrimali/pytorch-lightning into feature/11732_grad_accumulation_horovod
    @krshrimali
    krshrimali committed Feb 14, 2022
    Configuration menu
    Copy the full SHA
    67a0bce View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2022

  1. Update pytorch_lightning/strategies/horovod.py 

    Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
    @krshrimali @ananthsub
    krshrimali and ananthsub committed Feb 15, 2022
    Configuration menu
    Copy the full SHA
    5a17c2b View commit details
    Browse the repository at this point in the history

  2. Changes per review, pass accumulate_grad_batches as arg

    @krshrimali
    krshrimali committed Feb 15, 2022
    Configuration menu
    Copy the full SHA
    53fd24a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'feature/11732_grad_accumulation_horovod' of github.com:… 

    …krshrimali/pytorch-lightning into feature/11732_grad_accumulation_horovod
    @krshrimali
    krshrimali committed Feb 15, 2022
    Configuration menu
    Copy the full SHA
    82ebd9e View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Feb 15, 2022
    Configuration menu
    Copy the full SHA
    c914e3a View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2022

  1. Add tests to ensure error raised, replace gpus= with accelerator=gpu … 

    …and devices=count
    @krshrimali
    krshrimali committed Feb 16, 2022
    Configuration menu
    Copy the full SHA
    2a4c6fa View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature/11732_grad_accumulation_horovod' of github.com:… 

    …krshrimali/pytorch-lightning into feature/11732_grad_accumulation_horovod
    @krshrimali
    krshrimali committed Feb 16, 2022
    Configuration menu
    Copy the full SHA
    2a792df View commit details
    Browse the repository at this point in the history

  3. Don't add tests for now

    @krshrimali
    krshrimali committed Feb 16, 2022
    Configuration menu
    Copy the full SHA
    0850b0a View commit details
    Browse the repository at this point in the history

  4. Use basic model, and trainer.fit instead of _run_horovod

    @krshrimali
    krshrimali committed Feb 16, 2022
    Configuration menu
    Copy the full SHA
    9fd23ce View commit details
    Browse the repository at this point in the history

  5. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Feb 16, 2022
    Configuration menu
    Copy the full SHA
    04ee625 View commit details
    Browse the repository at this point in the history

  6. Apply suggestions from code review 

    Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
    @krshrimali @carmocca
    krshrimali and carmocca committed Feb 16, 2022
    Configuration menu
    Copy the full SHA
    7c73b0c View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2022

  1. Self review

    @carmocca
    carmocca committed Feb 17, 2022
    Configuration menu
    Copy the full SHA
    8c1e3f1 View commit details
    Browse the repository at this point in the history

  2. Remove duplicated test

    @carmocca
    carmocca committed Feb 17, 2022
    Configuration menu
    Copy the full SHA
    60ba198 View commit details
    Browse the repository at this point in the history

  3. Missed horovod in RunIf

    @carmocca
    carmocca committed Feb 17, 2022
    Configuration menu
    Copy the full SHA
    5786ce8 View commit details
    Browse the repository at this point in the history

  4. Further simplification

    @carmocca
    carmocca committed Feb 17, 2022
    Configuration menu
    Copy the full SHA
    670fbac View commit details
    Browse the repository at this point in the history

Commits on Feb 18, 2022

  1. Resolve merge conflict

    @krshrimali
    krshrimali committed Feb 18, 2022
    Configuration menu
    Copy the full SHA
    372d322 View commit details
    Browse the repository at this point in the history