Fix torch-numpy compatibility conflict in tests #20004

awaelchli · 2024-06-21T17:29:28Z

What does this PR do?

PyTorch distributed (an potentially other modules) are not yet compatible with NumPy > 2.0. This PR skips tests that run with incompatible versions. Note, this will only affect tests run on master where numpy is unpinned.

📚 Documentation preview 📚: https://pytorch-lightning--20004.org.readthedocs.build/en/20004/

cc @carmocca @justusschock @awaelchli @Borda

for more information, see https://pre-commit.ci

github-actions · 2024-06-21T17:32:40Z

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, lightning, 3.8, 2.0, oldest)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.1)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.2)	success	✅
pl-cpu (macOS-14, lightning, 3.10, 2.3)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 2.0, oldest)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.1)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.2)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.3)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 2.0, oldest)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.1)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.2)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.3)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 2.0)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 2.0)	success	✅
pl-cpu (windows-2022, pytorch, 3.8, 2.0)	success	✅
pl-cpu (macOS-12, pytorch, 3.11, 2.0)	success	✅
pl-cpu (macOS-12, pytorch, 3.11, 2.1)	success	✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0)	success	✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.1)	success	✅
pl-cpu (windows-2022, pytorch, 3.11, 2.0)	success	✅
pl-cpu (windows-2022, pytorch, 3.11, 2.1)	success	✅

These checks are required after the changes to requirements/fabric/base.txt, src/lightning/fabric/utilities/testing/_runif.py, requirements/pytorch/base.txt.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
pytorch-lightning (GPUs) (testing Lightning \| latest)	success	✅
pytorch-lightning (GPUs) (testing PyTorch \| latest)	success	✅

These checks are required after the changes to requirements/pytorch/base.txt, requirements/fabric/base.txt, src/lightning/fabric/utilities/testing/_runif.py.

🟢 pytorch_lightning: Benchmarks

Check ID	Status
lightning.Benchmarks	success	✅

These checks are required after the changes to requirements/fabric/base.txt, requirements/pytorch/base.txt, src/lightning/fabric/utilities/testing/_runif.py.

🟢 fabric: Docs

Check ID	Status
docs-make (fabric, doctest)	success	✅
docs-make (fabric, html)	success	✅

These checks are required after the changes to src/lightning/fabric/utilities/testing/_runif.py, requirements/fabric/base.txt.

🟢 pytorch_lightning: Docs

Check ID	Status
docs-make (pytorch, doctest)	success	✅
docs-make (pytorch, html)	success	✅

These checks are required after the changes to requirements/pytorch/base.txt.

🟢 pytorch_lightning: Docker

Check ID	Status
build-cuda (3.10, 2.0, 11.8.0)	success	✅
build-cuda (3.10, 2.1, 12.1.0)	success	✅
build-cuda (3.10, 2.2, 12.1.0)	success	✅
build-cuda (3.11, 2.1, 12.1.0)	success	✅
build-cuda (3.11, 2.2, 12.1.0)	success	✅
build-pl (3.10, 2.0, 11.8.0)	success	✅
build-pl (3.10, 2.1, 12.1.0)	success	✅
build-pl (3.10, 2.2, 12.1.0)	success	✅
build-pl (3.11, 2.1, 12.1.0)	success	✅
build-pl (3.11, 2.2, 12.1.0)	success	✅

These checks are required after the changes to requirements/pytorch/base.txt, requirements/fabric/base.txt.

🟢 lightning_fabric: CPU workflow

Check ID	Status
fabric-cpu (macOS-11, lightning, 3.8, 2.0, oldest)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
fabric-cpu (macOS-11, lightning, 3.11, 2.1)	success	✅
fabric-cpu (macOS-11, lightning, 3.11, 2.2)	success	✅
fabric-cpu (macOS-14, lightning, 3.10, 2.3)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 2.0, oldest)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.1)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.2)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.3)	success	✅
fabric-cpu (windows-2022, lightning, 3.8, 2.0, oldest)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
fabric-cpu (windows-2022, lightning, 3.11, 2.1)	success	✅
fabric-cpu (windows-2022, lightning, 3.11, 2.2)	success	✅
fabric-cpu (windows-2022, lightning, 3.11, 2.3)	success	✅
fabric-cpu (macOS-11, fabric, 3.8, 2.0)	success	✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 2.0)	success	✅
fabric-cpu (windows-2022, fabric, 3.8, 2.0)	success	✅
fabric-cpu (macOS-12, fabric, 3.11, 2.0)	success	✅
fabric-cpu (macOS-12, fabric, 3.11, 2.1)	success	✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.0)	success	✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.1)	success	✅
fabric-cpu (windows-2022, fabric, 3.11, 2.0)	success	✅
fabric-cpu (windows-2022, fabric, 3.11, 2.1)	success	✅

These checks are required after the changes to requirements/fabric/base.txt, src/lightning/fabric/utilities/testing/_runif.py, tests/tests_fabric/strategies/test_ddp_integration.py, tests/tests_fabric/utilities/test_distributed.py.

🟢 lightning_fabric: Azure GPU

Check ID	Status
lightning-fabric (GPUs) (testing Fabric \| latest)	success	✅
lightning-fabric (GPUs) (testing Lightning \| latest)	success	✅

These checks are required after the changes to requirements/fabric/base.txt, src/lightning/fabric/utilities/testing/_runif.py, tests/tests_fabric/strategies/test_ddp_integration.py, tests/tests_fabric/utilities/test_distributed.py.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to requirements/fabric/base.txt, requirements/pytorch/base.txt, src/lightning/fabric/utilities/testing/_runif.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.11)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.11)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.11)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.11)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.11)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.11)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.11)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.11)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.11)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.11)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.11)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.11)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.11)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.11)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.11)	success	✅

These checks are required after the changes to src/lightning/fabric/utilities/testing/_runif.py, requirements/fabric/base.txt, requirements/pytorch/base.txt.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

codecov · 2024-06-21T17:46:53Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59%. Comparing base (709a2a9) to head (5edc8d8).

❗ There is a different number of reports uploaded between BASE (709a2a9) and HEAD (5edc8d8). Click for more details.

HEAD has 5 uploads more than BASE
| Flag | BASE (709a2a9) | HEAD (5edc8d8) | |------|------|------| |pytest|30|28| |lightning_fabric|4|10| |python3.10|16|14| |cpu|48|45| |pytorch_lightning|10|9| |pytest-full|24|21| |python3.11|6|19| |pytorch2.1|6|4| |pytorch2.3|3|2|

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #20004     +/-   ##
=========================================
- Coverage      84%      59%    -25%     
=========================================
  Files         426      421      -5     
  Lines       35293    35195     -98     
=========================================
- Hits        29709    20769   -8940     
- Misses       5584    14426   +8842

for more information, see https://pre-commit.ci

…ightning into tests/numpy-compatibility

Skip tests that have torch numpy compatibility conflict

c71966c

github-actions bot added the fabric lightning.fabric.Fabric label Jun 21, 2024

pre-commit-ci bot and others added 2 commits June 21, 2024 17:29

[pre-commit.ci] auto fixes from pre-commit.com hooks

64a1f7f

for more information, see https://pre-commit.ci

unpin

560017e

awaelchli changed the title ~~Skip tests that have torch-numpy compatibility conflict~~ [WIP] Skip tests that have torch-numpy compatibility conflict Jun 21, 2024

github-actions bot added pl Generic label for PyTorch Lightning package dependencies Pull requests that update a dependency file labels Jun 21, 2024

awaelchli marked this pull request as ready for review June 21, 2024 17:32

awaelchli requested review from lantiga, Borda, tchaton and justusschock as code owners June 21, 2024 17:32

awaelchli added the tests label Jun 21, 2024

awaelchli and others added 4 commits June 21, 2024 12:47

deepspeed

be11d71

[pre-commit.ci] auto fixes from pre-commit.com hooks

1088fb6

for more information, see https://pre-commit.ci

bump lightning-utilities

d936367

Merge branch 'tests/numpy-compatibility' of github.com:Lightning-AI/l…

6d8da97

…ightning into tests/numpy-compatibility

awaelchli added this to the 2.3.x milestone Jun 21, 2024

fixme

5edc8d8

awaelchli changed the title ~~[WIP] Skip tests that have torch-numpy compatibility conflict~~ Skip tests that have torch-numpy compatibility conflict Jun 21, 2024

awaelchli changed the title ~~Skip tests that have torch-numpy compatibility conflict~~ Fix torch-numpy compatibility conflict in tests Jun 21, 2024

justusschock approved these changes Jun 21, 2024

View reviewed changes

awaelchli merged commit e330da5 into master Jun 22, 2024
131 checks passed

awaelchli deleted the tests/numpy-compatibility branch June 22, 2024 00:21

awaelchli added a commit that referenced this pull request Jun 27, 2024

Fix torch-numpy compatibility conflict in tests (#20004)

7939cd3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix torch-numpy compatibility conflict in tests #20004

Fix torch-numpy compatibility conflict in tests #20004

awaelchli commented Jun 21, 2024 •

edited by github-actions bot

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

codecov bot commented Jun 21, 2024 •

edited

Loading

Fix torch-numpy compatibility conflict in tests #20004

Fix torch-numpy compatibility conflict in tests #20004

Conversation

awaelchli commented Jun 21, 2024 • edited by github-actions bot Loading

What does this PR do?

github-actions bot commented Jun 21, 2024 • edited Loading

⚡ Required checks status: All passing 🟢

Groups summary

codecov bot commented Jun 21, 2024 • edited Loading

Codecov Report

awaelchli commented Jun 21, 2024 •

edited by github-actions bot

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

codecov bot commented Jun 21, 2024 •

edited

Loading