[PR Welcome] More data augmentation methods

[irvingzhang0512]

Describe the feature
More data augmentation methods.

Motivation
When training our own models, we need to TRY EVERYTHING...

Additional context

• MMaction2 existing data augmentation methods:
  • random flip.
  • all kinds of crop and resize
  • color jitter(brightness/contrast/saturation/hue)

Plans

• imgaug [Feature] Support Imgaug for augmentations in the data pipeline. #492
• mixup and cutmix [Feature] Support Mixup and Cutmix for Recognizers. #681
• RandAugment by imgaug [Improvement] Set RandAugment as Imgaug default transforms. #585
• PytorchVideo Transforms. [Feature] Support Pytorchvideo Transforms #1008
• albumentations
• videomix

More image augmentation methods could refer to PaddleClas.

[innerlee]

Thanks for the enthusiastic! Here is a practical way:

1. Make a list of candidate augmentations. For each augmentation, find its original paper, together with a reference implementation.
2. Make individual pull requests for each one of the list.

You may use this issue to track progress.

[dreamerlin]

Yeah, Thanks for your proposal. It is better to provide a list of candidate augmentations so that we can also help to implement some of these ! @irvingzhang0512

[irvingzhang0512]

this paper shows that random rotation may help

[innerlee]

In image classification, rotation works to some extent. But color jittering sometimes does not work.

Whether they work or not, it does not affect whether we implement them.

edit: @dreamerlin had some code on color jittering

[innerlee]

Supporting rotation would be great.

I also wanted to have some camera transformations, such as

• push in
• pull out
• pan
• tilt

Do you have interest in implementing them?

[irvingzhang0512]

Supporting rotation would be great.

I also wanted to have some camera transformations, such as

• push in
• pull out
• pan
• tilt

Do you have interest in implementing them?

@innerlee
I'm a little busy until Jan. 20th. If no one implemente these then, I'll have a try. Before that, I'll implement tsm-mobilenet and support imgaug/albumentation in pipeline.

[innerlee]

No hurry.

There is also lots of movie transition fx that can do some fancy temporal "mixup"

[irvingzhang0512]

Find an interesting paper here. This paper proposes a video augmentation strategy called VideoMix.

[innerlee]

haha

[irvingzhang0512]

PytorchVideo transform random_resized_crop support shift mode, which looks like the movie transforms. Maybe we can support this.
@innerlee @dreamerlin