Split `Precision.init_context` #18734

awaelchli · 2023-10-06T12:50:40Z

What does this PR do?

Fixes #18703

📚 Documentation preview 📚: https://pytorch-lightning--18734.org.readthedocs.build/en/18734/

cc @Borda @justusschock @awaelchli @carmocca

github-actions · 2023-10-06T13:27:43Z

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, lightning, 3.8, 1.12, oldest)	success	✅
pl-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.1)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.12, oldest)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.1)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.12, oldest)	success	✅
pl-cpu (windows-2022, lightning, 3.9, 1.12)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.1)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13)	success	✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13)	success	✅
pl-cpu (macOS-12, pytorch, 3.11, 2.0)	success	✅
pl-cpu (macOS-12, pytorch, 3.11, 2.1)	success	✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0)	success	✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.1)	success	✅
pl-cpu (windows-2022, pytorch, 3.11, 2.0)	success	✅
pl-cpu (windows-2022, pytorch, 3.11, 2.1)	success	✅

These checks are required after the changes to src/lightning/fabric/plugins/precision/bitsandbytes.py, src/lightning/fabric/plugins/precision/deepspeed.py, src/lightning/fabric/plugins/precision/double.py, src/lightning/fabric/plugins/precision/fsdp.py, src/lightning/fabric/plugins/precision/half.py, src/lightning/fabric/plugins/precision/precision.py, src/lightning/fabric/plugins/precision/transformer_engine.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla_fsdp.py, src/lightning/pytorch/plugins/precision/deepspeed.py, src/lightning/pytorch/plugins/precision/double.py, src/lightning/pytorch/plugins/precision/fsdp.py, src/lightning/pytorch/plugins/precision/half.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/trainer/call.py, tests/tests_pytorch/plugins/precision/test_all.py, tests/tests_pytorch/plugins/precision/test_deepspeed_precision.py, tests/tests_pytorch/plugins/precision/test_double.py, tests/tests_pytorch/plugins/precision/test_half.py.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
[pytorch-lightning (GPUs) (testing Lightning	latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=178326&view=logs&jobId=47e66f3c-897a-5428-da11-bf5c7745762e)	success
[pytorch-lightning (GPUs) (testing PyTorch	latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=178326&view=logs&jobId=3f274fac-2e11-54ca-487e-194c91f3ae9f)	success

These checks are required after the changes to src/lightning/pytorch/plugins/precision/deepspeed.py, src/lightning/pytorch/plugins/precision/double.py, src/lightning/pytorch/plugins/precision/fsdp.py, src/lightning/pytorch/plugins/precision/half.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/trainer/call.py, tests/tests_pytorch/plugins/precision/test_all.py, tests/tests_pytorch/plugins/precision/test_deepspeed_precision.py, tests/tests_pytorch/plugins/precision/test_double.py, tests/tests_pytorch/plugins/precision/test_half.py, src/lightning/fabric/plugins/precision/bitsandbytes.py, src/lightning/fabric/plugins/precision/deepspeed.py, src/lightning/fabric/plugins/precision/double.py, src/lightning/fabric/plugins/precision/fsdp.py, src/lightning/fabric/plugins/precision/half.py, src/lightning/fabric/plugins/precision/precision.py, src/lightning/fabric/plugins/precision/transformer_engine.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla_fsdp.py.

🟢 pytorch_lightning: Benchmarks

Check ID	Status
lightning.Benchmarks	success	✅

These checks are required after the changes to src/lightning/fabric/plugins/precision/bitsandbytes.py, src/lightning/fabric/plugins/precision/deepspeed.py, src/lightning/fabric/plugins/precision/double.py, src/lightning/fabric/plugins/precision/fsdp.py, src/lightning/fabric/plugins/precision/half.py, src/lightning/fabric/plugins/precision/precision.py, src/lightning/fabric/plugins/precision/transformer_engine.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla_fsdp.py, src/lightning/pytorch/plugins/precision/deepspeed.py, src/lightning/pytorch/plugins/precision/double.py, src/lightning/pytorch/plugins/precision/fsdp.py, src/lightning/pytorch/plugins/precision/half.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/trainer/call.py.

🟢 fabric: Docs

Check ID	Status
docs-make (fabric, doctest)	success	✅
docs-make (fabric, html)	success	✅

These checks are required after the changes to src/lightning/fabric/plugins/precision/bitsandbytes.py, src/lightning/fabric/plugins/precision/deepspeed.py, src/lightning/fabric/plugins/precision/double.py, src/lightning/fabric/plugins/precision/fsdp.py, src/lightning/fabric/plugins/precision/half.py, src/lightning/fabric/plugins/precision/precision.py, src/lightning/fabric/plugins/precision/transformer_engine.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla_fsdp.py.

🟢 pytorch_lightning: Docs

Check ID	Status
docs-make (pytorch, doctest)	success	✅
docs-make (pytorch, html)	success	✅

These checks are required after the changes to src/lightning/pytorch/plugins/precision/deepspeed.py, src/lightning/pytorch/plugins/precision/double.py, src/lightning/pytorch/plugins/precision/fsdp.py, src/lightning/pytorch/plugins/precision/half.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/trainer/call.py.

🟢 lightning_fabric: CPU workflow

Check ID	Status
fabric-cpu (macOS-11, lightning, 3.8, 1.12, oldest)	success	✅
fabric-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
fabric-cpu (macOS-11, lightning, 3.11, 2.1)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.12, oldest)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.1)	success	✅
fabric-cpu (windows-2022, lightning, 3.8, 1.12, oldest)	success	✅
fabric-cpu (windows-2022, lightning, 3.9, 1.12)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
fabric-cpu (windows-2022, lightning, 3.11, 2.1)	success	✅
fabric-cpu (macOS-11, fabric, 3.8, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13)	success	✅
fabric-cpu (windows-2022, fabric, 3.8, 1.13)	success	✅
fabric-cpu (macOS-12, fabric, 3.11, 2.0)	success	✅
fabric-cpu (macOS-12, fabric, 3.11, 2.1)	success	✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.0)	success	✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.1)	success	✅
fabric-cpu (windows-2022, fabric, 3.11, 2.0)	success	✅
fabric-cpu (windows-2022, fabric, 3.11, 2.1)	success	✅

These checks are required after the changes to src/lightning/fabric/plugins/precision/bitsandbytes.py, src/lightning/fabric/plugins/precision/deepspeed.py, src/lightning/fabric/plugins/precision/double.py, src/lightning/fabric/plugins/precision/fsdp.py, src/lightning/fabric/plugins/precision/half.py, src/lightning/fabric/plugins/precision/precision.py, src/lightning/fabric/plugins/precision/transformer_engine.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla_fsdp.py, tests/tests_fabric/plugins/precision/test_all.py, tests/tests_fabric/plugins/precision/test_bitsandbytes.py, tests/tests_fabric/plugins/precision/test_deepspeed.py, tests/tests_fabric/plugins/precision/test_half.py, tests/tests_fabric/plugins/precision/test_transformer_engine.py.

🟢 lightning_fabric: Azure GPU

Check ID	Status
[lightning-fabric (GPUs) (testing Fabric	latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=178328&view=logs&jobId=fe777007-6e77-5e50-6e71-fc5977ab193a)	success
[lightning-fabric (GPUs) (testing Lightning	latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=178328&view=logs&jobId=47e66f3c-897a-5428-da11-bf5c7745762e)	success

These checks are required after the changes to src/lightning/fabric/plugins/precision/bitsandbytes.py, src/lightning/fabric/plugins/precision/deepspeed.py, src/lightning/fabric/plugins/precision/double.py, src/lightning/fabric/plugins/precision/fsdp.py, src/lightning/fabric/plugins/precision/half.py, src/lightning/fabric/plugins/precision/precision.py, src/lightning/fabric/plugins/precision/transformer_engine.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla_fsdp.py, tests/tests_fabric/plugins/precision/test_all.py, tests/tests_fabric/plugins/precision/test_bitsandbytes.py, tests/tests_fabric/plugins/precision/test_deepspeed.py, tests/tests_fabric/plugins/precision/test_half.py, tests/tests_fabric/plugins/precision/test_transformer_engine.py.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to src/lightning/fabric/plugins/precision/bitsandbytes.py, src/lightning/fabric/plugins/precision/deepspeed.py, src/lightning/fabric/plugins/precision/double.py, src/lightning/fabric/plugins/precision/fsdp.py, src/lightning/fabric/plugins/precision/half.py, src/lightning/fabric/plugins/precision/precision.py, src/lightning/fabric/plugins/precision/transformer_engine.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla_fsdp.py, src/lightning/pytorch/plugins/precision/deepspeed.py, src/lightning/pytorch/plugins/precision/double.py, src/lightning/pytorch/plugins/precision/fsdp.py, src/lightning/pytorch/plugins/precision/half.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/trainer/call.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.11)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.11)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.11)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.11)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.11)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.11)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.11)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.11)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.11)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.11)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.11)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.11)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.11)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.11)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.11)	success	✅

These checks are required after the changes to src/lightning/fabric/plugins/precision/bitsandbytes.py, src/lightning/fabric/plugins/precision/deepspeed.py, src/lightning/fabric/plugins/precision/double.py, src/lightning/fabric/plugins/precision/fsdp.py, src/lightning/fabric/plugins/precision/half.py, src/lightning/fabric/plugins/precision/precision.py, src/lightning/fabric/plugins/precision/transformer_engine.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla_fsdp.py, src/lightning/pytorch/plugins/precision/deepspeed.py, src/lightning/pytorch/plugins/precision/double.py, src/lightning/pytorch/plugins/precision/fsdp.py, src/lightning/pytorch/plugins/precision/half.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/trainer/call.py.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

codecov · 2023-10-06T13:39:09Z

Codecov Report

Merging #18734 (0c54d3c) into master (0777aab) will decrease coverage by 32%.
Report is 3 commits behind head on master.
The diff coverage is 90%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #18734      +/-   ##
==========================================
- Coverage      84%      52%     -32%     
==========================================
  Files         428      431       +3     
  Lines       33486    34216     +730     
==========================================
- Hits        28019    17701   -10318     
- Misses       5467    16515   +11048

src/lightning/fabric/strategies/fsdp.py

for more information, see https://pre-commit.ci

src/lightning/pytorch/strategies/strategy.py

src/lightning/fabric/plugins/precision/bitsandbytes.py

src/lightning/fabric/plugins/precision/fsdp.py

carmocca

Thank you!

Borda

split looks good :)

src/lightning/fabric/plugins/precision/double.py

awaelchli added 3 commits October 6, 2023 14:44

rename to module_init_context

7ec9c80

implement tensor_init_context

9212fc3

call tensor_init_context

4bc1051

github-actions bot added fabric lightning.fabric.Fabric pl Generic label for PyTorch Lightning package labels Oct 6, 2023

awaelchli added this to the 2.1 milestone Oct 6, 2023

update

c2b6f4f

awaelchli changed the title ~~Split Precision.init_context~~ WIP: Split Precision.init_context Oct 6, 2023

awaelchli marked this pull request as ready for review October 6, 2023 13:27

awaelchli requested review from Borda, carmocca, justusschock and williamFalcon as code owners October 6, 2023 13:27

awaelchli added refactor plugin labels Oct 6, 2023

fix attribute error

13f55e7

awaelchli added 2 commits October 6, 2023 15:41

rename test functions

bca55cf

wip

94425a2

carmocca mentioned this pull request Oct 6, 2023

Create context managers before entering any with ExitStack #18716

Merged

awaelchli added 2 commits October 6, 2023 15:59

consistent

1b8ada7

fix

25fd9d8

awaelchli commented Oct 6, 2023

View reviewed changes

src/lightning/fabric/strategies/fsdp.py Show resolved Hide resolved

[pre-commit.ci] auto fixes from pre-commit.com hooks

92b150c

for more information, see https://pre-commit.ci

awaelchli changed the title ~~WIP: Split Precision.init_context~~ Split Precision.init_context Oct 6, 2023

carmocca reviewed Oct 6, 2023

View reviewed changes

src/lightning/pytorch/strategies/strategy.py Show resolved Hide resolved

src/lightning/fabric/plugins/precision/bitsandbytes.py Show resolved Hide resolved

src/lightning/fabric/plugins/precision/fsdp.py Show resolved Hide resolved

call the thing from the other thing

5928797

awaelchli requested a review from carmocca October 9, 2023 14:28

Merge branch 'master' into refactor/precision-module-init

2f6fe13

awaelchli added 2 commits October 9, 2023 16:44

call PrecisionPlugin.module_init_context()

1aea71a

Python 3.8 doesn't support multi-line with-context

0c54d3c

carmocca approved these changes Oct 9, 2023

View reviewed changes

Borda approved these changes Oct 9, 2023

View reviewed changes

src/lightning/fabric/plugins/precision/double.py Show resolved Hide resolved

mergify bot added the ready PRs ready to be merged label Oct 9, 2023

awaelchli merged commit 3775340 into master Oct 9, 2023

awaelchli deleted the refactor/precision-module-init branch October 9, 2023 16:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Split `Precision.init_context` #18734

Split `Precision.init_context` #18734

Uh oh!

awaelchli commented Oct 6, 2023 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Oct 6, 2023 •

edited

Loading

Uh oh!

codecov bot commented Oct 6, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carmocca left a comment

Uh oh!

Borda left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Split Precision.init_context #18734

Split Precision.init_context #18734

Uh oh!

Conversation

awaelchli commented Oct 6, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Uh oh!

github-actions bot commented Oct 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚡ Required checks status: All passing 🟢

Groups summary

Uh oh!

codecov bot commented Oct 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carmocca left a comment

Choose a reason for hiding this comment

Uh oh!

Borda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Split `Precision.init_context` #18734

Split `Precision.init_context` #18734

awaelchli commented Oct 6, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Oct 6, 2023 •

edited

Loading

codecov bot commented Oct 6, 2023 •

edited

Loading