Ensure DeepSpeedDataLoader matches torch DataLoader semantics

[tjruwase]

DeepSpeedDataLoader is a wrapper for torch DataLoader and should at a minimum match the semantics. Right now this property is not well tested and discrepancies have already been identified, such as this #167. In my estimation, a fair bit of work is required to achieve this goal, including the following items:

1. Support batching and non-batching modes.

2. Unit tests to validate expected behavior

3. Performance comparison between DeepSpeedDataLoader and torch DataLoader.

4. Some documentation of DeepSpeedDataLoader, including any semantic deviation from torch DataLoader.

[tomekrut]

Hi @tjruwase
I agree that a lot could be done around DeepSpeedDataLoader so adding what I suggested does not fix everything but.... now while writing / thinking... :) for your class https://github.com/microsoft/DeepSpeed/blob/master/deepspeed/pt/deepspeed_dataloader.py

What if you

self._create_dataloader()

from the DeepSpeedDataLoader in its constructor excluding self.data object creation and adding self.dataloader to your def __iter__ def __len__ and def __next__ functions? This way native data loader will handle all the above methods. What do you think?

[tjruwase]

I think your suggestion is along my thoughts as well. The only difference is that using self.data to initiate prefetching could help perf in some cases, which is why it is there. But I can be convinced otherwise. So if you have bandwidth to take a stab at this, that would be greatly appreciated.

[tomekrut]

Hi @tjruwase here is the pull request that I think should address the above... I don't know everything about your repo so well but I think this solution should be ok.
#178