tf.keras.layers.experimental.preprocessing.RandomXXX API is missing "nearest" as an option for fill_mode.

[kechan]

System information

• TensorFlow version (you are using): 2.2.0-dev20200504
• Are you willing to contribute it (Yes/No): Yes (but really depends on the technicality)

Describe the feature and the current behavior/state.
tf.keras.layers.experimental.preprocessing.RandomTranslation's argument "fill_mode" should accept the value of "nearest". This should be available to all relevant data augmentation experimental API that will inherit from Layer. fill_mode of "nearest" is the default for tf.keras.preprocessing.image.ImageDataGenerator.

Will this change the current api? How?
This is about adding an extra fill_mode option for "nearest". It should be there since this is the default for the older Keras API (for data augmentation). This will allow apple to apple comparison to reproduce results.

Who will benefit with this feature?

Any Other info.

[tanzhenyu]

you can set 'nearest' to interpolation

[tanzhenyu]

To be more specific:
-- we have fill_mode which stands for when the output coordinate is valid, but our compute for equivalent input coordinate is out of range, so reflect, wrap, constant are valid modes. For example, in order to fill the value for output coordinate (0, 0), the equivalent value from input coordinate should be (-1, -1), since there's not value at (-1, -1), reflect mode might tell us (-1, -1) is same as (1, 1), so output[0][0] = input[1][1].
-- we have interpolation which stands for when the computed equivalent input coordinate is a float, not int. In this case, nearest and bilinear are valid modes. -- potential extra mode is spline interpolation, which is what scipy supports today. For example, if output[0][0] should be filled with input[0.5][0.5], using nearest it will be same as input[1][1], but using bilinear means it will be interpolated from input[0][0], input[0][1], input[1][0], and input[1][1]

[kechan]

@tanzhenyu I used the old keras data augmentation API for a while now, I am pretty sure I don't want interpolation to "nearest". I really want fill_mode to be "nearest". The effect is that, for example, if you right shift the image, the "void" is filled by copying the edge pixel of the photo over and over until you fill up that void.

Keras actually used this as the default. I may guess that most ppl find it reasonable choice for most image tasks. So leaving this out of the new API is not ideal.

[tanzhenyu]

@tanzhenyu I used the old keras data augmentation API for a while now, I am pretty sure I don't want interpolation to "nearest". I really want fill_mode to be "nearest". The effect is that, for example, if you right shift the image, the "void" is filled by copying the edge pixel of the photo over and over until you fill up that void.

Keras actually used this as the default. I may guess that most ppl find it reasonable choice for most image tasks. So leaving this out of the new API is not ideal.

In that case you may end up with the main content of your image being masked out. reflect in this case would honestly work better.

[kechan]

@tanzhenyu Not sure what you mean by "masked out" exactly. But minor masking out is sometimes what data augmentation intends to do??, e.g. crop out, or randomly obscuring part of object. Also, optimal image augmentation is domain specific, so don't need to say which work better or worse. FYI: I did try "reflect" for my task, and i obtained slightly worse accuracy (although I can't robustly attribute it to this choice.

This feature is after all in Keras for over 2 years and it is the default. If the broader TF community think it ought to not exist, I am fine with that. I will just have to implement my own...

[tanzhenyu]

@tanzhenyu Not sure what you mean by "masked out" exactly. But minor masking out is sometimes what data augmentation intends to do??, e.g. crop out, or randomly obscuring part of object. Also, optimal image augmentation is domain specific, so don't need to say which work better or worse. FYI: I did try "reflect" for my task, and i obtained slightly worse accuracy (although I can't robustly attribute it to this choice.

This feature is after all in Keras for over 2 years and it is the default. If the broader TF community think it ought to not exist, I am fine with that. I will just have to implement my own...

What do you mean by default for Keras?