[RFC] New Augmentation techniques in Torchvison

[oke-aditya]

🚀 Feature

Inclusion of new Augmentation techniques in torchvision.transforms.

Motivation

Transforms are important for data augmentation 😅

Proposals

• RandAugment Citations 306 Adding RandAugment implementation #4348
• Cutout Citations 964 Superseded by RandomErasing
• MixUp Citations 1624 Code (Note it is CC-BY-NC-license) Adding Mixup and Cutmix #4379
• CutMix Citations 437 Code ICCV 2019 Adding Mixup and Cutmix #4379
• TrivialAugment ICCV 2021 Integration of TrivialAugment with the current AutoAugment Code #4221
• AugMix Citations 157
• Scale-Jitter Citations 11 - code - benchmarks
• Simple CopyPaste Add SimpleCopyPaste augmentation #5825
• Port SSD & SSDlite Augmentations and Mixup/Cutmix from references to vision
• Detection Transforms [RFC] Abstractions for segmentation / detection transforms #1406 Image augmentation with bounding boxes. #2213 box_rescale function to rescale bounding boxes to a new image shape #3980
• AutoAugment Detection 231 Citations code
• Greedy Search Policy
• FastAutoAugment Citations 135 NeurIPS 2019 Code (1k+ GitHub stars)
• ReMixMatch Citations 165
• FixMatch Citations 258
• Transforms for color spaces augmentations E.g. rgb2hsv, rgb2bgr, rgb2lab, etc [feature request] rgb2lab / rgb2hsv / rgb2gray and other color space conversions (maybe upstream from kornia? or colorsys python core module?) #4029
• Gaussian Noise See Add Gaussian noise transformation #6192

Additional context

To visitors
Kindly give a 👍 if you think any of these would help in your work.

Also if you have any transform in mind please provide few details here!

Linked to #3221

cc @vfdev-5 @fmassa

[hassiahk]

Would be nice to have these Augmentations in torchvision.transforms. 😄

Also found this official code implementation for Cutout.

[datumbox]

@oke-aditya Do we need the cutout given we have Random Erasing which can be configured to have the same more or less effect?

[oke-aditya]

I think the same. I compared both the implementations. RandomErasing is newer than Cutout, also both the augmentations produce almost similar results.

Also, As per docs RandomErasing does not work for for PIL Images. It works only for torch.Tensor. I am not sure if that is intentional or needs some work.

[tflahaul]

Not a transform idea but what about adding an optional 'target_transforms' argument in transforms.RandomApply? That way random imgs transforms and their targets equivalents could be applied at the same time.
The current way of doing so (from what I know) is by writing your own class and using functional transforms. For example :

class AugmentExample:
    def __init__(self, p=0.1):
        self.p = p

    def __call__(self, img, box):
        if self.p < torch.rand(1):
            img = img.flip(-1)
            box = box_hflip(box)
        return img, box

Also having a lot more keypoints/bbox transforms would be really great (ideally any image transform that involves a transformation of the targets should be accompanied by one?).

(Sorry if my english isn't right, I speak baguette.)

[oke-aditya]

@datumbox is closing this issue intended?

As I understand there is dataset and transforms rework. Which would be a major refactor

Do we plan to migrate all transforms to new ones in near future?

(I had a minor look at the proposal which looks fantastic)

[datumbox]

Not at all intended; Github just closed it when I merged the PR. We are certainly not done here :D