Fix a bug that caused spurious `AttributeError` when multiple `DataLoader` classes are imported #14117

otaj · 2022-08-09T14:00:20Z

What does this PR do?

When using PyTorch Geometric, there is quite a few of DataLoaders, which do not define their own __init__ method and instead override only some methods. While patching the __init__ and then removing this patch, the handling of such cases wasn't careful enough and caused spurious AttributeErrors, that were completely unrelated to user's code. The reason was import of multiple DataLoaders.

This PR fixes it by being more careful about checking if we can remove our patch.

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

Was this discussed/approved via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃
As always 🐱

cc @Borda @justusschock @awaelchli @ninginthecloud @rohitgr7 @otaj @akihironitta

carmocca

Wow!

tests/tests_pytorch/utilities/test_data.py

codecov · 2022-08-10T10:51:36Z

Codecov Report

Merging #14117 (4837b7a) into master (5801484) will increase coverage by 15%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master   #14117     +/-   ##
=========================================
+ Coverage      61%      76%    +15%     
=========================================
  Files         326      326             
  Lines       26561    26604     +43     
=========================================
+ Hits        16227    20220   +3993     
+ Misses      10334     6384   -3950

krshrimali

Love it. Thank you, @otaj 🎉

…ader` classes are imported (#14117)

…ader` classes are imported (#14117) fix

…ader` classes are imported (Lightning-AI#14117)

* update version and changelog for 1.7.2 release * Reset all results on epoch end (#14061) Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> * Skip ddp fork tests on windows (#14121) * Fix device placement when `.cuda()` called without specifying index (#14128) * Convert subprocess test to standalone test (#14101) * Fix entry point test for Python 3.10 (#14154) * Fix flaky test caused by weak reference (#14157) * Fix saving hyperparameters in a composition where parent is not a LM or LDM (#14151) Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com> * Remove DeepSpeed version restriction from Lite (#13967) * Configure the check-group app (#14165) Co-authored-by: Jirka <jirka.borovec@seznam.cz> * Update onnxruntime requirement from <=1.12.0 to <1.13.0 in /requirements (#14083) Updates the requirements on [onnxruntime](https://github.com/microsoft/onnxruntime) to permit the latest version. - [Release notes](https://github.com/microsoft/onnxruntime/releases) - [Changelog](https://github.com/microsoft/onnxruntime/blob/master/docs/ReleaseManagement.md) - [Commits](microsoft/onnxruntime@v0.1.4...v1.12.1) --- updated-dependencies: - dependency-name: onnxruntime dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update gcsfs requirement from <2022.6.0,>=2021.5.0 to >=2021.5.0,<2022.8.0 in /requirements (#14079) Update gcsfs requirement in /requirements Updates the requirements on [gcsfs](https://github.com/fsspec/gcsfs) to permit the latest version. - [Release notes](https://github.com/fsspec/gcsfs/releases) - [Commits](fsspec/gcsfs@2021.05.0...2022.7.1) --- updated-dependencies: - dependency-name: gcsfs dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix a bug that caused spurious `AttributeError` when multiple `DataLoader` classes are imported (#14117) fix * CI: Replace `_` of in GHA workflow filenames with `-` (#13917) * Rename workflow files * Update docs * Fix azure badges * Update the main readme * bad rebase * Update doc * CI: Update Windows version from 2019 to 2022 (#14129) Update windows * CI/CD: Add CUDA version to docker image tags (#13831) * append cuda version to tags * revertme: push to hub * Update docker readme * Build base-conda-py3.9-torch1.12-cuda11.3.1 * Use new images in conda tests * revertme: push to hub * Revert "revertme: push to hub" This reverts commit 0f7d534. * Revert "revertme: push to hub" This reverts commit 46a05fc. * Run conda if workflow edited * Run gpu testing if workflow edited * Use new tags in release/Dockerfile * Build base-cuda and PL release images with all combinations * Update release docker * Update conda from py3.9-torch1.12 to py3.10-torch.1.12 * Fix ubuntu version * Revert conda * revertme: push to hub * Don't build Python 3.10 for now... * Fix pl release builder * updating version contribute to the error? docker/buildx#456 * Update actions' versions * Update slack user to notify * Don't use 11.6.0 to avoid bagua incompatibility * Don't use 11.1, and use 11.1.1 * Update .github/workflows/ci-pytorch_test-conda.yml Co-authored-by: Luca Medeiros <67411094+luca-medeiros@users.noreply.github.com> * Update trigger * Ignore artfacts from tutorials * Trim docker images to distribute * Add an image for tutorials * Update conda image 3.8x1.10 * Try different conda variants * No need to set cuda for conda jobs * Update who to notify ipu failure * Don't push * update filenaem Co-authored-by: Luca Medeiros <67411094+luca-medeiros@users.noreply.github.com> * Avoid entry_points deprecation warning (#14052) Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> * Configure the check-group app (#14165) Co-authored-by: Jirka <jirka.borovec@seznam.cz> * Profile batch transfer and gradient clipping hooks (#14069) Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com> * Avoid false positive warning about using `sync_dist` when using torchmetrics (#14143) Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com> * Avoid raising the sampler warning if num_replicas=1 (#14097) Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com> Co-authored-by: otaj <6065855+otaj@users.noreply.github.com> * Remove skipping logic in favor of path filtering (#14170) * Support checkpoint save and load with Stochastic Weight Averaging (#9938) Co-authored-by: thomas chaton <thomas@grid.ai> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com> Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com> Co-authored-by: Kushashwa Ravi Shrimali <kushashwaravishrimali@gmail.com> Co-authored-by: Jirka <jirka.borovec@seznam.cz> Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com> * Use fsdp module to initialize precision scalar for fsdp native (#14092) Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Laverne Henderson <laverne.henderson@coupa.com> Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com> * add more issues types (#14174) * add more issues types * Update .github/ISSUE_TEMPLATE/config.yml Co-authored-by: Mansy <ahmed.mansy156@gmail.com> * typo Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com> Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com> Co-authored-by: Mansy <ahmed.mansy156@gmail.com> Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com> Co-authored-by: Laverne Henderson <laverne.henderson@coupa.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> * CI: clean building docs (#14216) * CI: clean building docs * group * . * CI: docker focus on PL only (#14246) * CI: docker focus on PL only * group * Allowed setting attributes on `DataLoader` and `BatchSampler` when instantiated inside `*_dataloader` hooks (#14212) Co-authored-by: otaj <6065855+otaj@users.noreply.github.com> * Revert "Remove skipping logic in favor of path filtering (#14170)" (#14244) * Update defaults for WandbLogger's run name and project name (#14145) Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com> Co-authored-by: Jirka <jirka.borovec@seznam.cz> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> Co-authored-by: Luca Medeiros <67411094+luca-medeiros@users.noreply.github.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: otaj <6065855+otaj@users.noreply.github.com> Co-authored-by: Adam Reeve <adreeve@gmail.com> Co-authored-by: thomas chaton <thomas@grid.ai> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Kushashwa Ravi Shrimali <kushashwaravishrimali@gmail.com> Co-authored-by: Laverne Henderson <laverne.henderson@coupa.com> Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com> Co-authored-by: Mansy <ahmed.mansy156@gmail.com>

fix inheritance

6188c93

otaj added bug Something isn't working data handling Generic data-related topic labels Aug 9, 2022

otaj added this to the pl:1.7.x milestone Aug 9, 2022

otaj self-assigned this Aug 9, 2022

otaj requested review from Borda, tchaton, carmocca, williamFalcon, awaelchli, justusschock, kaushikb11 and rohitgr7 as code owners August 9, 2022 14:00

github-actions bot added the pl Generic label for PyTorch Lightning package label Aug 9, 2022

changelog

cda6798

carmocca approved these changes Aug 9, 2022

View reviewed changes

tests/tests_pytorch/utilities/test_data.py Outdated Show resolved Hide resolved

otaj added 2 commits August 9, 2022 16:46

apply suggestions

dab2ab2

Merge branch 'master' into bugfix/_wrap_init

da597b4

mergify bot added the has conflicts label Aug 10, 2022

merge master

6ab3248

mergify bot removed the has conflicts label Aug 10, 2022

awaelchli approved these changes Aug 10, 2022

View reviewed changes

mergify bot added the ready PRs ready to be merged label Aug 10, 2022

awaelchli enabled auto-merge (squash) August 10, 2022 10:25

mergify bot added has conflicts and removed ready PRs ready to be merged labels Aug 10, 2022

shuo-zhou mentioned this pull request Aug 10, 2022

Fix problems of tests for Python version 3.7, 3.8 and 3.9 pykale/pykale#330

Merged

5 tasks

krshrimali approved these changes Aug 10, 2022

View reviewed changes

Merge branch 'master' into bugfix/_wrap_init

4837b7a

mergify bot added ready PRs ready to be merged and removed has conflicts ready PRs ready to be merged labels Aug 10, 2022

Borda approved these changes Aug 10, 2022

View reviewed changes

awaelchli merged commit f132d44 into master Aug 10, 2022

awaelchli deleted the bugfix/_wrap_init branch August 10, 2022 16:09

guillaume-be mentioned this pull request Aug 11, 2022

AttributeError when multiple inheriting classes are patched by Lightning #14168

Closed

awaelchli pushed a commit that referenced this pull request Aug 11, 2022

Fix a bug that caused spurious AttributeError when multiple `DataLo…

4b8526a

…ader` classes are imported (#14117)

mberr mentioned this pull request Aug 12, 2022

🍭 🚀 Add optional dep for opt_einsum pykeen/pykeen#1058

Merged

awaelchli added a commit that referenced this pull request Aug 15, 2022

Fix a bug that caused spurious AttributeError when multiple `DataLo…

fcf9c5c

…ader` classes are imported (#14117) fix

jessecambon pushed a commit to jessecambon/lightning that referenced this pull request Aug 16, 2022

Fix a bug that caused spurious AttributeError when multiple `DataLo…

669cd50

…ader` classes are imported (Lightning-AI#14117)

mberr mentioned this pull request Jan 1, 2023

⚡🧪 Bring back lightning tests pykeen/pykeen#1191

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix a bug that caused spurious `AttributeError` when multiple `DataLoader` classes are imported #14117

Fix a bug that caused spurious `AttributeError` when multiple `DataLoader` classes are imported #14117

otaj commented Aug 9, 2022 •

edited by github-actions bot

carmocca left a comment

codecov bot commented Aug 10, 2022 •

edited

krshrimali left a comment

Fix a bug that caused spurious AttributeError when multiple DataLoader classes are imported #14117

Fix a bug that caused spurious AttributeError when multiple DataLoader classes are imported #14117

Conversation

otaj commented Aug 9, 2022 • edited by github-actions bot

What does this PR do?

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

PR review

Did you have fun?

carmocca left a comment

Choose a reason for hiding this comment

codecov bot commented Aug 10, 2022 • edited

Codecov Report

krshrimali left a comment

Choose a reason for hiding this comment

Fix a bug that caused spurious `AttributeError` when multiple `DataLoader` classes are imported #14117

Fix a bug that caused spurious `AttributeError` when multiple `DataLoader` classes are imported #14117

otaj commented Aug 9, 2022 •

edited by github-actions bot

codecov bot commented Aug 10, 2022 •

edited