Mixup or CutMix augmentations

[ogencoglu]

Any plan on adding Mixup or CutMix augmentations?

• https://github.com/clovaai/CutMix-PyTorch - https://arxiv.org/abs/1905.04899
• https://github.com/facebookresearch/mixup-cifar10 - https://arxiv.org/abs/1710.09412

[bmabey]

I am looking into contributing both Mixup and CutMix augmentations. Both of these augmentations take multiple images in order to do the augmentation. As far as I can tell all of the existing augmentations in the library only operates on a single image and the entire API is build around a single image being fetched in the datalaoder. Is that correct? To implement these new augmentations we would be operating on the batch level. Do you have any thoughts on how this should be implemented API wise within the library?

[ogencoglu]

Now when I checked the API, you are right. For CutMix, if the mixed image is fixed it can be treated as an input variable (numpy array etc.) but Mixup is more tricky obviously.

[JeffreyWardman]

@bmabey Randomly sampling within the dataloader for multiple images. Would be nice to augment labels as well. These can then be fed in with

AUGMENTATION = MIXUP(p=0.6)  # Second image is 1-p, although ideally this scales for n mixups
aug = albumentations.AUGMENTATION(images=[image1, image2, ...], labels=[[0, 0, 1], [0, 1, 0]]) 
print(aug['labels'])
-->
[0, 0.4, 0.6]

[mathmanu]

Please see the repository https://github.com/naver-ai/relabel_imagenet
It shows impressive accuracy improvements with CutMix

CutMix implementation: https://github.com/naver-ai/relabel_imagenet/blob/main/utils/data_augment.py#L54
It does mixing within the batch.

This may be a simpler alternative to using multiple images. Looking forward to see this in albumentations

[dimimal]

Is anyone working on this? I would be happy to take it and do a PR

[vedal]

The following paper also shows good improvements with MixUp/CutMix
https://arxiv.org/abs/2111.05328

[chAwater]

Ref PR #1147

[mikel-brostrom]

MixUp PR here: #1409

[nopps07]

Any follow up on this?

[rusmux]

Any updates on this? This issue is almost 3 years old

[Aryan-Garg]

Yeah, better to use keras-cv at this point instead of albumentations (for these augmentations)

[ternaus]

MixUp added in #1549