Dev flow.utils.data part3

[Flowingsun007]

dataloader最小实现（part3）

• pr-part1见：#5406
• 原始pr见：#5074
  

---

此pr主要改动：

1.支持flow.utils.data.DistributedSampler

基于DistributedSampler可以完成DDP的数据集构建，使用方式参考pytorch examples:

train_dataset = datasets.ImageFolder(
        traindir,
        transforms.Compose([
            transforms.RandomResizedCrop(224),
            transforms.RandomHorizontalFlip(),
            transforms.ToTensor(),
            normalize,
        ]))

if args.distributed:
    train_sampler = torch.utils.data.distributed.DistributedSampler(train_dataset)
else:
    train_sampler = None

train_loader = torch.utils.data.DataLoader(
    train_dataset, batch_size=args.batch_size, shuffle=(train_sampler is None),
    num_workers=args.workers, pin_memory=True, sampler=train_sampler)

• TODO：补充一个DDP + DistributedSampler的test case

2.支持更多的常用cv类datasets

__all__ = [
    "MNIST", 
    "FashionMNIST", 
    "CIFAR10", 
    "CIFAR100",
    "CocoCaptions",
    "CocoDetection",
    "ImageNet",
    "VOCDetection",
    "VOCSegmentation",
    "DatasetFolder",
    "ImageFolder"
]

3. 支持对齐torchvision的更多transforms方法：

__all__ = [
    "Compose", 
    "ToTensor", 
    "PILToTensor",
    "ConvertImageDtype",
    "ToPILImage",
    "Normalize", 
    "Resize",
    "Scale",
    "CenterCrop",
    "Pad",
    "Lambda",
    "RandomTransforms",
    "RandomApply",
    "RandomOrder",
    "RandomChoice",
    "RandomCrop",
    "RandomHorizontalFlip",
    "RandomVerticalFlip",
    "RandomResizedCrop",
    "RandomSizedCrop",
    "FiveCrop",
    "TenCrop",
    "InterpolationMode"
]

[github-actions]

CI failed, removing label automerge

[github-actions]

CI failed, removing label automerge

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 141.6ms (= 7081.5ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.3ms (= 6413.8ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.10 (= 141.6ms / 128.3ms)

PyTorch resnet50 time: 83.4ms (= 4171.9ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.6ms (= 3731.2ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.12 (= 83.4ms / 74.6ms)

PyTorch resnet50 time: 59.4ms (= 2971.5ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 48.4ms (= 2419.8ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.23 (= 59.4ms / 48.4ms)

PyTorch resnet50 time: 50.2ms (= 2510.5ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 39.2ms (= 1957.7ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.28 (= 50.2ms / 39.2ms)

PyTorch resnet50 time: 39.8ms (= 1989.5ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 37.7ms (= 1886.7ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.05 (= 39.8ms / 37.7ms)