Fix import from torch.distributed when distributed not available #16658

awaelchli · 2023-02-06T21:51:28Z

What does this PR do?

Before submitting

Was this discussed/approved via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

I made sure I had fun coding 🙃

cc @Borda @tchaton @carmocca @justusschock @awaelchli

for more information, see https://pre-commit.ci

…ugfix/torch-dist-import

github-actions · 2023-02-06T23:20:23Z

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, lightning, 3.8, 1.11)	success	✅
pl-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
pl-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.11)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.12)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (windows-2022, lightning, 3.9, 1.11)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 1.12)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (slow, macOS-11, lightning, 3.8, 1.11)	success	✅
pl-cpu (slow, ubuntu-20.04, lightning, 3.8, 1.11)	success	✅
pl-cpu (slow, windows-2022, lightning, 3.8, 1.11)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13)	success	✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13)	success	✅

These checks are required after the changes to src/lightning/fabric/strategies/fsdp.py, tests/tests_pytorch/utilities/test_imports.py.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
pytorch-lightning (GPUs)	success	✅

These checks are required after the changes to tests/tests_pytorch/utilities/test_imports.py, src/lightning/fabric/strategies/fsdp.py.

🟢 pytorch_lightning: Azure HPU

Check ID	Status
pytorch-lightning (HPUs)	success	✅

These checks are required after the changes to src/lightning/fabric/strategies/fsdp.py, tests/tests_pytorch/utilities/test_imports.py.

🟢 pytorch_lightning: Azure IPU

Check ID	Status
pytorch-lightning (IPUs)	success	✅

These checks are required after the changes to src/lightning/fabric/strategies/fsdp.py, tests/tests_pytorch/utilities/test_imports.py.

🟢 lightning_fabric: CPU workflow

Check ID	Status
fabric-cpu (macOS-11, lightning, 3.8, 1.11)	success	✅
fabric-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
fabric-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.11)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.12)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (windows-2022, lightning, 3.9, 1.11)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 1.12)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
fabric-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (macOS-11, fabric, 3.8, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13)	success	✅
fabric-cpu (windows-2022, fabric, 3.8, 1.13)	success	✅

These checks are required after the changes to src/lightning/fabric/strategies/fsdp.py, tests/tests_fabric/utilities/test_imports.py.

🟢 lightning_fabric: Azure GPU

Check ID	Status
lightning-fabric (GPUs)	success	✅

These checks are required after the changes to src/lightning/fabric/strategies/fsdp.py, tests/tests_fabric/utilities/test_imports.py.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to src/lightning/fabric/strategies/fsdp.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.10)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.10)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.10)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.10)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.10)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.10)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.10)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.10)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.10)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.10)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.10)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.10)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.10)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.10)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.10)	success	✅

These checks are required after the changes to src/lightning/fabric/strategies/fsdp.py.

🟢 link-check

Check ID	Status
markdown-link-check	success	✅

These checks are required after the changes to src/lightning/fabric/CHANGELOG.md, src/lightning/pytorch/CHANGELOG.md.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

codecov · 2023-02-06T23:32:12Z

Codecov Report

Merging #16658 (9abf71e) into master (1b1241c) will decrease coverage by 21%.
The diff coverage is 100%.

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #16658     +/-   ##
=========================================
- Coverage      81%      60%    -21%     
=========================================
  Files         444      439      -5     
  Lines       32385    32310     -75     
=========================================
- Hits        26329    19507   -6822     
- Misses       6056    12803   +6747

) (cherry picked from commit 923a842)

torch dist import

68604f1

github-actions bot added the fabric lightning.fabric.Fabric label Feb 6, 2023

pre-commit-ci bot and others added 2 commits February 6, 2023 21:53

[pre-commit.ci] auto fixes from pre-commit.com hooks

68a673a

for more information, see https://pre-commit.ci

improved test

79e6944

github-actions bot added the pl Generic label for PyTorch Lightning package label Feb 6, 2023

pre-commit-ci bot and others added 4 commits February 6, 2023 23:18

[pre-commit.ci] auto fixes from pre-commit.com hooks

a049a42

for more information, see https://pre-commit.ci

add changelog

0d62765

Merge remote-tracking branch 'origin/bugfix/torch-dist-import' into b…

3badb38

…ugfix/torch-dist-import

comment

9abf71e

awaelchli marked this pull request as ready for review February 6, 2023 23:19

awaelchli requested review from carmocca, justusschock and williamFalcon as code owners February 6, 2023 23:19

awaelchli added priority: 0 High priority task bug Something isn't working labels Feb 6, 2023

awaelchli self-assigned this Feb 6, 2023

awaelchli added this to the v1.9.x milestone Feb 6, 2023

carmocca approved these changes Feb 6, 2023

View reviewed changes

Borda approved these changes Feb 7, 2023

View reviewed changes

mergify bot added the ready PRs ready to be merged label Feb 7, 2023

awaelchli merged commit 923a842 into master Feb 7, 2023

awaelchli deleted the bugfix/torch-dist-import branch February 7, 2023 09:52

Borda pushed a commit that referenced this pull request Feb 10, 2023

Fix import from torch.distributed when distributed not available (#16658

40d89ed

) (cherry picked from commit 923a842)

Borda pushed a commit that referenced this pull request Feb 10, 2023

Fix import from torch.distributed when distributed not available (#16658

5ddf283

) (cherry picked from commit 923a842)

Borda pushed a commit that referenced this pull request Feb 10, 2023

Fix import from torch.distributed when distributed not available (#16658

a40d92d

) (cherry picked from commit 923a842)

lantiga pushed a commit that referenced this pull request Feb 10, 2023

Fix import from torch.distributed when distributed not available (#16658

2b75971

) (cherry picked from commit 923a842)

This was referenced Mar 3, 2023

ImportError: cannot import name 'default_pg_timeout' from 'torch.distributed' #16931

Closed

py-pytorch-lightning: torch~distributed not supported in 1.9.0 spack/spack#35809

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix import from torch.distributed when distributed not available #16658

Fix import from torch.distributed when distributed not available #16658

awaelchli commented Feb 6, 2023 •

edited by github-actions bot

github-actions bot commented Feb 6, 2023 •

edited

codecov bot commented Feb 6, 2023 •

edited

Fix import from torch.distributed when distributed not available #16658

Fix import from torch.distributed when distributed not available #16658

Conversation

awaelchli commented Feb 6, 2023 • edited by github-actions bot

What does this PR do?

Before submitting

PR review

Did you have fun?

github-actions bot commented Feb 6, 2023 • edited

⚡ Required checks status: All passing 🟢

Groups summary

codecov bot commented Feb 6, 2023 • edited

Codecov Report

awaelchli commented Feb 6, 2023 •

edited by github-actions bot

github-actions bot commented Feb 6, 2023 •

edited

codecov bot commented Feb 6, 2023 •

edited