Example on using CutMix Augmentation for Image Classification

[sayannath]

Though there are many augmentation techniques like MixUp and CutOut techniques, there are some issues as well.
It is because in CutOut Augmentation we remove a part(square) from a picture with either a Gaussian Noise or a Black pixel, which results in a decrease of important portions of an image during training. This can lead to a limitation in the case of CNN.
In the case of MixUp Augmentation, the images which are generated are somewhere not natural and confuses the model, especially for the localization task.

In this example how we use the CutMix Augmentation technique to overcome these issues. In this example, I have used the CIFAR10 dataset and the CutMix Augmentation technique performs better and yields better results than the simple one.

[google-cla]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[sayannath]

@googlebot I signed it!

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

@googlebot I signed it!

[sayannath]

@fchollet could you please check and review my PR, if you have time. Thank You!

[fchollet]

Thanks for the PR! I'll get to reviewing it soon. I've been pretty swamped.

[sayannath]

It's totally understandable!! @fchollet

[8bitmp3]

I'll check out your awesome example soon, thanks for your 👍 effort

[sayannath]

Thank you! @8bitmp3

[fchollet]

Thanks for the PR!

[8bitmp3]

Great stuff @sayannath I've added some suggestions, PTAL 👍

Thanks, I learned a lot from the paper and your example.

[8bitmp3]

By the way, hope the first paragraph refactoring suggestion is OK. The first sentence explains the why and the second - what CutMix does (using the words from the original paper by Yun et al.):

_CutMix_ is a data augmentation technique that addresses the issue of
information loss and inefficiency present in regional dropout strategies. 
Instead of removing pixels and filling them with black or grey pixels or
Gaussian noise, you replace the removed regions with a patch from another image,
while the ground truth labels are mixed proportionally to the number of pixels
of combined images. CutMix was proposed in
[CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://arxiv.org/pdf/1905.04899.pdf)
(Yun et al., 2019)

[fchollet]

Thanks for the update! It's looking great. I think we can improve the choice of model.

[fchollet]

Is the PR ready for review at this time?

[sayannath]

Almost there. Improvising the things there.

[fchollet]

Ok, please ping me when you want me to look at it!

[fchollet]

Are you still working on this?

[sayannath]

Are you still working on this?

@fchollet Yes but I have tried with MobileNetV2 but not giving me desired result.

[fchollet]

Ok, resnet seems fine then. Any other changes you're still working on?

[sayannath]

Ok, resnet seems fine then. Any other changes you're still working on?

Training it to some more epochs and I will commit the changes today and will notify you.

[sayannath]

@fchollet you can look into it now, I have incorporated all the changes you have told me to do.

[fchollet]

Thanks for the update! It looks great. Have you tried using larger input images?

Note that I pushed some copyedits, please pull them first.

[fchollet]

Call this IMG_SIZE. IMG_SHAPE would be like (32, 32).

[sayannath]

Ya Sure! Will do that

[fchollet]

Note: in practice you get much better accuracy on CIFAR-10 by upscaling the input images (perhaps this is what you saw better accuracy with ResNet-20...), i.e. by using a (None, None, 3) as input shape and starting the model with a Rescaling(150, 150) layer. Have you tried that?

[sayannath]

No haven't tried that.

[sayannath]

Call this IMG_SIZE. IMG_SHAPE would be like (32, 32).

Incorporated this change. @fchollet

[fchollet]

Looks good, please add the generated files.

[sayannath]

@fchollet I have added the generated files.

[fchollet]

Thanks for the great contribution! 👍

[sayannath]

I learned a lot from this PR. Thanks, @fchollet and @8bitmp3 for guiding me.