Add support for padding and cropping to tf.keras.layers.experimental.preprocessing.Resizing

[willbattel]

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template

System information

• TensorFlow version (you are using): 2.4.0
• Are you willing to contribute it (Yes/No): No, unfortunately I don't have the time to do so.

Describe the feature and the current behavior/state.
The new layer tf.keras.layers.experimental.preprocessing.Resizing allows for models to be made more portable by handling image resizing within the model itself, as described in the docs here. This layer provides options for interpolation, but cannot be configured to crop or pad the image to maintain aspect-ratio.

In my case, I use the tf.image.resize_with_pad in my tf.data pipeline in order to maintain aspect ratio and letter-box the shorter sides. This cannot be done inside of the model without a custom or Lambda layer.

The layer should ideally also include the functionality from tf.image.resize_with_crop_or_pad.

Will this change the current api? How?
Yes, but purely additive. I suggest adding an additional parameter resize_type to the layer and simply have the default value perform the current behavior. This would not affect current users, but offer the feature to those who chose to enable it.

Who will benefit with this feature?
This would be handy for developers/researchers who want a simple, built-in mechanism for resizing images inside of a model but don't want the current stretching behavior. In particular, this would benefit the portability of models, where we can do more within the model itself.

Any Other info.
N/A

[anilkumarKanasani]

@willbattel , we would like to work on this feature improvement and give the updated code

[gadagashwini]

@willbattel,
Latest Tensorflow v2.8, tf.keras.layers.experimental.preprocessing.Resizing has crop_to_aspect_ratio

tf.keras.layers.Resizing(
    height,
    width,
    interpolation='bilinear',
    crop_to_aspect_ratio=False,
    **kwargs
)

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[willbattel]

@gadagashwini thank you for letting me know. That is great to see.

The other thing we are still interested is a pad_to_aspect_ratio option so that we can include the whole image, without cropping, in its original aspect ratio by adding padding on the short sides- the same way tf.image.resize_with_pad does.