Add Yield loop example

[awaelchli]

What does this PR do?

Follow up to #9609

Adds the YieldLoop as an example for loop customization.

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

• Was this discussed/approved via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

[tchaton]

LGTM !

[codecov]

Codecov Report

Merging #9983 (25feed6) into master (f95ba20) will increase coverage by 4%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #9983    +/-   ##
=======================================
+ Coverage      89%     92%    +4%     
=======================================
  Files         182     182            
  Lines       16111   16368   +257     
=======================================
+ Hits        14273   15132   +859     
+ Misses       1838    1236   -602     

[awaelchli]

@tchaton I added boxes with tutorial following your style

[SeanNaren]

A heavily advanced feature 😅 but LGTM! I like this suite of custom loops, however it may be worth considering discoverability (add them to docs or something?)

[awaelchli]

A heavily advanced feature 😅 but LGTM! I like this suite of custom loops, however it may be worth considering discoverability (add them to docs or something?)

Yes absolutely. PR is coming. Will add a table to the loop docs with links and short description to these extensive examples.

[rohitgr7]

great example!

[awaelchli]

Getting this error in CI :(

Run # NOTE: do not include coverage report here, see: https://github.com/nedbat/coveragepy/issues/1003
  # NOTE: do not include coverage report here, see: https://github.com/nedbat/coveragepy/issues/1003
  coverage run --source pytorch_lightning -m pytest pytorch_lightning tests -v --durations=50 --junitxml=junit/test-results-Windows-py3.6-latest-stable.xml
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
  env:
    pythonLocation: C:\hostedtoolcache\windows\Python\3.6.8\x64
ImportError while loading conftest 'D:\a\pytorch-lightning\pytorch-lightning\tests\conftest.py'.
tests\conftest.py:24: in <module>
    from pytorch_lightning.plugins.environments.lightning_environment import find_free_network_port
pytorch_lightning\__init__.py:20: in <module>
    from pytorch_lightning.callbacks import Callback  # noqa: E402
pytorch_lightning\callbacks\__init__.py:26: in <module>
    from pytorch_lightning.callbacks.pruning import ModelPruning
pytorch_lightning\callbacks\pruning.py:31: in <module>
    from pytorch_lightning.core.lightning import LightningModule
pytorch_lightning\core\__init__.py:16: in <module>
    from pytorch_lightning.core.lightning import LightningModule
pytorch_lightning\core\lightning.py:39: in <module>
    from pytorch_lightning.trainer.connectors.logger_connector.fx_validator import _FxValidator
pytorch_lightning\trainer\__init__.py:16: in <module>
    from pytorch_lightning.trainer.trainer import Trainer
pytorch_lightning\trainer\trainer.py:30: in <module>
    from pytorch_lightning.accelerators import Accelerator, IPUAccelerator
pytorch_lightning\accelerators\__init__.py:13: in <module>
    from pytorch_lightning.accelerators.accelerator import Accelerator  # noqa: F401
pytorch_lightning\accelerators\accelerator.py:25: in <module>
    from pytorch_lightning.plugins.precision import ApexMixedPrecisionPlugin, NativeMixedPrecisionPlugin, PrecisionPlugin
pytorch_lightning\plugins\__init__.py:8: in <module>
    from pytorch_lightning.plugins.plugins_registry import (  # noqa: F401
pytorch_lightning\plugins\plugins_registry.py:20: in <module>
    from pytorch_lightning.plugins.training_type.training_type_plugin import TrainingTypePlugin
pytorch_lightning\plugins\training_type\__init__.py:1: in <module>
    from pytorch_lightning.plugins.training_type.ddp import DDPPlugin  # noqa: F401
pytorch_lightning\plugins\training_type\ddp.py:60: in <module>
    from torch.distributed.optim import DistributedOptimizer, PostLocalSGDOptimizer, ZeroRedundancyOptimizer
E   ImportError: cannot import name 'DistributedOptimizer'