[Bug] Add SharedCycleIteratorState

[tchaton]

What does this PR do?

The following script will create an infinite loop on master.

from torch.utils.data.sampler import SequentialSampler
from pytorch_lightning.trainer.supporters import CombinedLoader
from torch.utils.data import IterableDataset, DataLoader, Dataset

class MyIterableDataset(IterableDataset):

    def __init__(self, size: int = 10):
        self.size = size

    def __iter__(self):
        self.sampler = SequentialSampler(range(self.size))
        self.iter_sampler = iter(self.sampler)
        return self

    def __next__(self):
        return next(self.iter_sampler)

class MyMapDataset(Dataset):

    def __init__(self, size: int = 10):
        self.size = size

    def __getitem__(self, index):
        return index

    def __len__(self):
        return self.size

loaders = [DataLoader(MyIterableDataset(6)), DataLoader(MyMapDataset(5))]
dataloader = CombinedLoader(loaders, mode="max_size_cycle")
for _ in dataloader:
    print("i")

This PR adds a SharedCycleIteratorState to synchornize the CycleIterator and stop on max_length.

Fixes #<issue_number>

Does your PR introduce any breaking changes? If yes, please list them.

No.

Before submitting

• Was this discussed/approved via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

[codecov]

Codecov Report

Merging #8889 (5e917d1) into master (5b143d0) will decrease coverage by 4%.
The diff coverage is 97%.

@@           Coverage Diff           @@
##           master   #8889    +/-   ##
=======================================
- Coverage      93%     89%    -4%     
=======================================
  Files         169     172     +3     
  Lines       14079   14178    +99     
=======================================
- Hits        13053   12568   -485     
- Misses       1026    1610   +584     

[awaelchli]

Suggested change

	raise MisconfigurationException("Please, call reset once all dataloaders have been added.")
	raise MisconfigurationException("Please call reset once all dataloaders have been added.")

with the comma in there it sounds a bit passive-aggressive 🤣

[awaelchli]

Suggested change

	class SharedCycleIteratorState:
	mode: str = "max_size_cycle"
	class SharedCycleIteratorState:
	"""A state shared between all CylceIterators in a CombinedLoader.
	With a shared state, the iterators can decide to terminate based on the state of all others.
	If the mode is *max_size_cycle*, all iterators need to have finished before the combined loading is considered
	finished, and otherwise any iterator finishing early will lead to all iterators ending early.
	"""
	mode: str = "max_size_cycle"

[awaelchli]

What do you think of these docs? Did I get it right?