Random Resize Crop

[tarunn2799]

Any particular reason why Random Resize Crop isn't added to the augmentation pipeline when you trained SimCLR on CIFAR-10?

I tried adding it to see if it makes a difference, but I'm facing an error. Would appreciate it if you could tell me where I'm going wrong.

class CustomAugment(object):
    def __call__(self, sample):
        sample = self._random_apply(tf.image.flip_left_right, sample, p=0.5)
        sample= self._random_apply(self._randomr_crop, sample, p= 0.7)          
        sample = self._random_apply(self._color_jitter, sample, p=0.5)
        sample = self._random_apply(self._color_drop, sample, p=0.5)
        return sample

    def _randomr_crop(self, x, s= 32):
        r_s= random.randint(7, 16)
        x= tf.image.random_crop(x, size= [r_s, r_s, 3])
        x= tf.image.resize(x, [s, s])
        return x

I added a function for random resize crop. The function works for a single image but I get an error when I pass a batch of images from a tf.data.Dataset : InvalidArgumentError: Incompatible shapes: [4] vs. [3] [Op:GreaterEqual]

[sayakpaul]

Thanks for your interest in our work.

We did not use random resized crops in CIFAR10 because of its inherent low resolution and we noticed that it makes the contrastive learning task more difficult than necessary. In general, it's good to design an augmentation policy that neither makes that task too easy nor too difficult. A good way to check that would be logging the contrastive accuracy i.e. the accuracy at which your unsupervised model is able to tell if a pair of images is a positive pair. I think I had mentioned in my response to your email as well :D

Regarding the error, you are getting it is mainly stemming from the fact that the augmentation policy is implemented as a LambdaLayer which behaves in a different way than tf.data.Dataset's map(). You'll need to have the shapes such that they are always in 4D (batch_size, width, height, nb_channels). This is also why, in the tf.tile() call we pass an extra 1 to account for the batch size.

[tarunn2799]

@sayakpaul Thank you for prompt response, as always. That makes a lot of sense, and I happened to see your recent tweet as well.
Regarding the error,
x= tf.image.random_crop(x, size= [r_s, r_s, 3])
x= tf.image.resize(x, [1, s, s, 3])
I realized that could be the error and I tried passing 1 to the resize function (apologies for not mentioning), but I still get the same error. It'd be super helpful if you could help me with this.
Thank you SO much again for all the help Sayak!

[sayakpaul]

I would recommend doing something like (I haven't tested it yet) -

batch_size = x.shape[0]
x= tf.image.random_crop(x, size= [r_s, r_s, 3])
x= tf.image.resize(x, [batch_size, s, s, 3])

[tarunn2799]

Works like a charm! Thank you!

[sayakpaul]

Cool. Just verify a couple of batches before you pass them on to the training loop.

[tarunn2799]

Yep, that's exactly what I did.