Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Weekly minor patch release 2.1.2 [Rebase & merge] #19001

Merged
merged 27 commits into from
Nov 15, 2023
Merged

Weekly minor patch release 2.1.2 [Rebase & merge] #19001

merged 27 commits into from
Nov 15, 2023

Conversation

Borda
Copy link
Member

@Borda Borda commented Nov 14, 2023 •
edited by github-actions bot

What does this PR do?

regular patch release

Fixes #<issue_number>

Before submitting
  • Was this discussed/agreed via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

📚 Documentation preview 📚: https://pytorch-lightning--19001.org.readthedocs.build/en/19001/

cc @Borda @carmocca @justusschock @awaelchli

Borda and others added 24 commits November 14, 2023 19:01
@Borda
chlog: add new section
8428b79
@awaelchli @Borda
Fix symlink permission error for "last" checkpoint on Windows (#18942)
44897e6 
(cherry picked from commit 3b05d83)
@awaelchli @Borda
Create cache dir if it doesn't exist (#18955)
bb2b7f8 
(cherry picked from commit c4af18b)
@tchaton @Borda
Add the input_dir in the cache_dir to avoid overlapping downloads (#1…
70e5ae5 
…8960)

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
(cherry picked from commit e79ac21)
@awaelchli @Borda
Greedily select files for data processor workers based on size (#18907)
55910e5 
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit 62771f3)
@tchaton @Borda
Add support for deleting chunks (#18959)
94973e7 
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
(cherry picked from commit 529f07f)
@awaelchli @Borda
Cache directory per worker to avoid collisions (#18957)
1a9a6ef 
(cherry picked from commit 8a5d342)
@tchaton @Borda
Bump Lightning Cloud to 0.5.51 (#18962)
fd9f00c 
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit 20f58f6)
@tchaton @Borda
Prevent downloading more chunks than needed (#18964)
c266573 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit 1b3a3fb)
@tchaton @Borda
Bump Lightning Cloud Version to 0.5.52 (#18965)
80b36df 
(cherry picked from commit c524c0b)
@VictorPrins @Borda
Fix broken links on Lightning App docs (#18904)
ef6beb5 
fix broken links on App docs

(cherry picked from commit 07461a1)
@awaelchli @Borda
Move torchmetrics to device when using FSDP (#18954)
f7e0952 
(cherry picked from commit 964364b)
@awaelchli @Borda
Provide the default vocab size for the transformer demo model (#18963)
cd4c47a 
(cherry picked from commit 618e1c8)
@ethanwharris @Borda
App: Force plugin server to use localhost (#18976)
771324c 
(cherry picked from commit ec374ab)
@tchaton @Borda
Add Video/Audio support (#18977)
978f401 
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit 1c86011)
@ethanwharris @Borda
App: Enable bundling addtional files into app source (#18980)
5fe681d 
(cherry picked from commit a9d427c)
@ethanwharris @Borda
App: Limit rate of requests to http queue (#18981)
0c21338 
(cherry picked from commit 9085db4)
@carmocca @Borda
Fix precision default from environment (#18928)
ba03469 
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
(cherry picked from commit 466f772)
@awaelchli @Borda
Fix ModelCheckpoint.CHECKPOINT_NAME_LAST test interaction (#18993)
74574d0 
(cherry picked from commit b4605b4)
@AMHermansen @Borda
Fix typo (#18986)
d9363b5 
(cherry picked from commit ebc7cc3)
@awaelchli @Borda
Avoid importing from lightning.app in lightning.data (#18988)
7f3ec4d 
(cherry picked from commit 532c723)
@awaelchli @Borda
Fix test interactions (#18994)
1c5c61a 
(cherry picked from commit 340961a)
@tchaton @Borda
Add multiple uploaders to the map, optimize (#18989)
8a60a34 
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit 7288302)
@tchaton @Borda
Update auto_encoder.py to accomodate torchvision breaking change (#18996
d3e841d 
)

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit cb23fc2)
@Borda Borda changed the title Weekly minor patch release 2.1.1 [wip] Weekly minor patch release 2.1.2 [wip] Nov 14, 2023
@Borda Borda changed the title Weekly minor patch release 2.1.2 [wip] Weekly minor patch release 2.1.2 [wip] Nov 14, 2023
@github-actions github-actions bot added docs Documentation related fabric lightning.fabric.Fabric app Generic label for Lightning App package pl Generic label for PyTorch Lightning package dependencies Pull requests that update a dependency file package labels Nov 14, 2023
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 14, 2023 •
edited

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow
Check ID Status
pl-cpu (macOS-11, lightning, 3.8, 1.12, oldest) success ✅
pl-cpu (macOS-11, lightning, 3.9, 1.12) success ✅
pl-cpu (macOS-11, lightning, 3.10, 1.13) success ✅
pl-cpu (macOS-11, lightning, 3.10, 2.0) success ✅
pl-cpu (macOS-11, lightning, 3.10, 2.1) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.12, oldest) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.1) success ✅
pl-cpu (windows-2022, lightning, 3.8, 1.12, oldest) success ✅
pl-cpu (windows-2022, lightning, 3.9, 1.12) success ✅
pl-cpu (windows-2022, lightning, 3.10, 1.13) success ✅
pl-cpu (windows-2022, lightning, 3.10, 2.0) success ✅
pl-cpu (windows-2022, lightning, 3.10, 2.1) success ✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13) success ✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13) success ✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13) success ✅
pl-cpu (macOS-12, pytorch, 3.11, 2.0) success ✅
pl-cpu (macOS-12, pytorch, 3.11, 2.1) success ✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0) success ✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.1) success ✅
pl-cpu (windows-2022, pytorch, 3.11, 2.0) success ✅
pl-cpu (windows-2022, pytorch, 3.11, 2.1) success ✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/pytorch/callbacks/model_checkpoint.py, src/lightning/pytorch/callbacks/prediction_writer.py, src/lightning/pytorch/demos/transformer.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, tests/tests_pytorch/checkpointing/test_model_checkpoint.py, tests/tests_pytorch/loggers/test_tensorboard.py, tests/tests_pytorch/strategies/test_fsdp.py.

🟢 pytorch_lightning: Azure GPU
Check ID Status
pytorch-lightning (GPUs) (testing Lightning | latest) success ✅
pytorch-lightning (GPUs) (testing PyTorch | latest) success ✅

These checks are required after the changes to examples/pytorch/basics/autoencoder.py, src/lightning/pytorch/callbacks/model_checkpoint.py, src/lightning/pytorch/callbacks/prediction_writer.py, src/lightning/pytorch/demos/transformer.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, tests/tests_pytorch/checkpointing/test_model_checkpoint.py, tests/tests_pytorch/loggers/test_tensorboard.py, tests/tests_pytorch/strategies/test_fsdp.py, src/lightning/fabric/connector.py, src/lightning/fabric/strategies/fsdp.py.

🟢 pytorch_lightning: Benchmarks
Check ID Status
lightning.Benchmarks success ✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/pytorch/callbacks/model_checkpoint.py, src/lightning/pytorch/callbacks/prediction_writer.py, src/lightning/pytorch/demos/transformer.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py.

🟢 fabric: Docs
Check ID Status
docs-make (fabric, doctest) success ✅
docs-make (fabric, html) success ✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/fsdp.py.

🟢 pytorch_lightning: Docs
Check ID Status
docs-make (pytorch, doctest) success ✅
docs-make (pytorch, html) success ✅

These checks are required after the changes to src/lightning/pytorch/callbacks/model_checkpoint.py, src/lightning/pytorch/callbacks/prediction_writer.py, src/lightning/pytorch/demos/transformer.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, docs/source-pytorch/advanced/post_training_quantization.rst, docs/source-pytorch/starter/style_guide.rst.

🟢 lightning_data: CPU workflow
Check ID Status
data-cpu (macOS-11, lightning, 3.10, 2.1) success ✅
data-cpu (ubuntu-20.04, lightning, 3.10, 2.1) success ✅
data-cpu (windows-2022, lightning, 3.10, 2.1) success ✅

These checks are required after the changes to src/lightning/data/datasets/index.py, src/lightning/data/streaming/cache.py, src/lightning/data/streaming/config.py, src/lightning/data/streaming/constants.py, src/lightning/data/streaming/data_processor.py, src/lightning/data/streaming/dataset.py, src/lightning/data/streaming/functions.py, src/lightning/data/streaming/item_loader.py, src/lightning/data/streaming/reader.py, src/lightning/data/streaming/serializers.py, src/lightning/data/streaming/writer.py, src/lightning/data/utilities/format.py, src/lightning/data/utilities/packing.py, tests/tests_data/streaming/test_cache.py, tests/tests_data/streaming/test_client.py, tests/tests_data/streaming/test_data_processor.py, tests/tests_data/streaming/test_dataset.py, tests/tests_data/streaming/test_reader.py, tests/tests_data/streaming/test_serializer.py, tests/tests_data/streaming/test_writer.py, tests/tests_data/utilities/__init__.py, tests/tests_data/utilities/test_packing.py.

🟢 lightning_fabric: CPU workflow
Check ID Status
fabric-cpu (macOS-11, lightning, 3.8, 1.12, oldest) success ✅
fabric-cpu (macOS-11, lightning, 3.9, 1.12) success ✅
fabric-cpu (macOS-11, lightning, 3.10, 1.13) success ✅
fabric-cpu (macOS-11, lightning, 3.10, 2.0) success ✅
fabric-cpu (macOS-11, lightning, 3.11, 2.1) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.12, oldest) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.12) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.1) success ✅
fabric-cpu (windows-2022, lightning, 3.8, 1.12, oldest) success ✅
fabric-cpu (windows-2022, lightning, 3.9, 1.12) success ✅
fabric-cpu (windows-2022, lightning, 3.10, 1.13) success ✅
fabric-cpu (windows-2022, lightning, 3.10, 2.0) success ✅
fabric-cpu (windows-2022, lightning, 3.11, 2.1) success ✅
fabric-cpu (macOS-11, fabric, 3.8, 1.13) success ✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13) success ✅
fabric-cpu (windows-2022, fabric, 3.8, 1.13) success ✅
fabric-cpu (macOS-12, fabric, 3.11, 2.0) success ✅
fabric-cpu (macOS-12, fabric, 3.11, 2.1) success ✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.0) success ✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.1) success ✅
fabric-cpu (windows-2022, fabric, 3.11, 2.0) success ✅
fabric-cpu (windows-2022, fabric, 3.11, 2.1) success ✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/fsdp.py, tests/tests_fabric/loggers/test_tensorboard.py, tests/tests_fabric/test_connector.py.

🟢 lightning_fabric: Azure GPU
Check ID Status
lightning-fabric (GPUs) (testing Fabric | latest) success ✅
lightning-fabric (GPUs) (testing Lightning | latest) success ✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/fsdp.py, tests/tests_fabric/loggers/test_tensorboard.py, tests/tests_fabric/test_connector.py.

🟢 lightning_app: Tests workflow
Check ID Status
app-pytest (macOS-11, lightning, 3.8, latest) success ✅
app-pytest (macOS-11, lightning, 3.8, oldest) success ✅
app-pytest (macOS-11, app, 3.9, latest) success ✅
app-pytest (macOS-12, app, 3.11, latest) success ✅
app-pytest (ubuntu-20.04, lightning, 3.8, latest) success ✅
app-pytest (ubuntu-20.04, lightning, 3.8, oldest) success ✅
app-pytest (ubuntu-20.04, app, 3.9, latest) success ✅
app-pytest (ubuntu-22.04, app, 3.11, latest) success ✅
app-pytest (windows-2022, lightning, 3.8, latest) success ✅
app-pytest (windows-2022, lightning, 3.8, oldest) success ✅
app-pytest (windows-2022, app, 3.8, latest) success ✅
app-pytest (windows-2022, app, 3.11, latest) success ✅

These checks are required after the changes to src/lightning/app/core/constants.py, src/lightning/app/core/queues.py, src/lightning/app/plugin/plugin.py, tests/tests_app/core/test_queues.py, tests/tests_app/plugin/test_plugin.py, requirements/app/app.txt.

🟢 lightning_app: Examples
Check ID Status
app-examples (macOS-11, lightning, 3.9, latest) success ✅
app-examples (macOS-11, lightning, 3.9, oldest) success ✅
app-examples (macOS-11, app, 3.9, latest) success ✅
app-examples (ubuntu-20.04, lightning, 3.9, latest) success ✅
app-examples (ubuntu-20.04, lightning, 3.9, oldest) success ✅
app-examples (ubuntu-20.04, app, 3.9, latest) success ✅
app-examples (windows-2022, lightning, 3.9, latest) success ✅
app-examples (windows-2022, lightning, 3.9, oldest) success ✅
app-examples (windows-2022, app, 3.9, latest) success ✅

These checks are required after the changes to src/lightning/app/core/constants.py, src/lightning/app/core/queues.py, src/lightning/app/plugin/plugin.py, requirements/app/app.txt.

🟢 lightning_app: Docs
Check ID Status
docs-make (app, doctest) success ✅
docs-make (app, html) success ✅

These checks are required after the changes to src/lightning/app/core/constants.py, src/lightning/app/core/queues.py, src/lightning/app/plugin/plugin.py, docs/source-app/index.rst, requirements/app/app.txt.

🟢 mypy
Check ID Status
mypy success ✅

These checks are required after the changes to requirements/app/app.txt, src/lightning/app/core/constants.py, src/lightning/app/core/queues.py, src/lightning/app/plugin/plugin.py, src/lightning/data/datasets/index.py, src/lightning/data/streaming/cache.py, src/lightning/data/streaming/config.py, src/lightning/data/streaming/constants.py, src/lightning/data/streaming/data_processor.py, src/lightning/data/streaming/dataset.py, src/lightning/data/streaming/functions.py, src/lightning/data/streaming/item_loader.py, src/lightning/data/streaming/reader.py, src/lightning/data/streaming/serializers.py, src/lightning/data/streaming/writer.py, src/lightning/data/utilities/format.py, src/lightning/data/utilities/packing.py, src/lightning/fabric/connector.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/pytorch/callbacks/model_checkpoint.py, src/lightning/pytorch/callbacks/prediction_writer.py, src/lightning/pytorch/demos/transformer.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, src/version.info.

🟢 install
Check ID Status
install-pkg (ubuntu-22.04, app, 3.8) success ✅
install-pkg (ubuntu-22.04, app, 3.11) success ✅
install-pkg (ubuntu-22.04, fabric, 3.8) success ✅
install-pkg (ubuntu-22.04, fabric, 3.11) success ✅
install-pkg (ubuntu-22.04, pytorch, 3.8) success ✅
install-pkg (ubuntu-22.04, pytorch, 3.11) success ✅
install-pkg (ubuntu-22.04, lightning, 3.8) success ✅
install-pkg (ubuntu-22.04, lightning, 3.11) success ✅
install-pkg (ubuntu-22.04, notset, 3.8) success ✅
install-pkg (ubuntu-22.04, notset, 3.11) success ✅
install-pkg (macOS-12, app, 3.8) success ✅
install-pkg (macOS-12, app, 3.11) success ✅
install-pkg (macOS-12, fabric, 3.8) success ✅
install-pkg (macOS-12, fabric, 3.11) success ✅
install-pkg (macOS-12, pytorch, 3.8) success ✅
install-pkg (macOS-12, pytorch, 3.11) success ✅
install-pkg (macOS-12, lightning, 3.8) success ✅
install-pkg (macOS-12, lightning, 3.11) success ✅
install-pkg (macOS-12, notset, 3.8) success ✅
install-pkg (macOS-12, notset, 3.11) success ✅
install-pkg (windows-2022, app, 3.8) success ✅
install-pkg (windows-2022, app, 3.11) success ✅
install-pkg (windows-2022, fabric, 3.8) success ✅
install-pkg (windows-2022, fabric, 3.11) success ✅
install-pkg (windows-2022, pytorch, 3.8) success ✅
install-pkg (windows-2022, pytorch, 3.11) success ✅
install-pkg (windows-2022, lightning, 3.8) success ✅
install-pkg (windows-2022, lightning, 3.11) success ✅
install-pkg (windows-2022, notset, 3.8) success ✅
install-pkg (windows-2022, notset, 3.11) success ✅

These checks are required after the changes to src/lightning/app/core/constants.py, src/lightning/app/core/queues.py, src/lightning/app/plugin/plugin.py, src/lightning/data/datasets/index.py, src/lightning/data/streaming/cache.py, src/lightning/data/streaming/config.py, src/lightning/data/streaming/constants.py, src/lightning/data/streaming/data_processor.py, src/lightning/data/streaming/dataset.py, src/lightning/data/streaming/functions.py, src/lightning/data/streaming/item_loader.py, src/lightning/data/streaming/reader.py, src/lightning/data/streaming/serializers.py, src/lightning/data/streaming/writer.py, src/lightning/data/utilities/format.py, src/lightning/data/utilities/packing.py, src/lightning/fabric/connector.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/pytorch/callbacks/model_checkpoint.py, src/lightning/pytorch/callbacks/prediction_writer.py, src/lightning/pytorch/demos/transformer.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/strategy.py, src/version.info, requirements/app/app.txt.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

@Borda Borda changed the title Weekly minor patch release 2.1.2 [wip] Weekly minor patch release 2.1.2 [Rebase & merge] Nov 14, 2023
@mergify mergify bot added the ready PRs ready to be merged label Nov 14, 2023
awaelchli
awaelchli approved these changes Nov 14, 2023
View reviewed changes
Copy link
Member

@awaelchli awaelchli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
#18992 is missing from the commits

@Borda
Copy link
Member Author

Borda commented Nov 15, 2023

LGTM #18992 is missing from the commits

add, thx for cathing it

@lantiga lantiga merged commit 3368d16 into release/stable Nov 15, 2023
143 checks passed
@lantiga lantiga deleted the releasing/2.1.2 branch November 15, 2023 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lantiga lantiga lantiga approved these changes

@awaelchli awaelchli awaelchli approved these changes

@carmocca carmocca carmocca approved these changes

@rlizzo rlizzo rlizzo approved these changes

@tchaton tchaton Awaiting requested review from tchaton tchaton is a code owner

@ethanwharris ethanwharris Awaiting requested review from ethanwharris ethanwharris is a code owner

@justusschock justusschock Awaiting requested review from justusschock justusschock is a code owner

@williamFalcon williamFalcon Awaiting requested review from williamFalcon williamFalcon is a code owner

@edenlightning edenlightning Awaiting requested review from edenlightning

Assignees
No one assigned
Labels
app Generic label for Lightning App package dependencies Pull requests that update a dependency file docs Documentation related fabric lightning.fabric.Fabric package pl Generic label for PyTorch Lightning package ready PRs ready to be merged
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
9 participants
@Borda @lantiga @awaelchli @carmocca @rlizzo @tchaton @VictorPrins @ethanwharris @AMHermansen