Fix weird dash lines on ImageProjectiveTransformV2

[Smankusors]

Fixes #41989

So, this is before (look at second row):

and after :

And the script to generate these images is here

Let me know if there's something wrong 😉

[Smankusors]

huh...? It seems that I need to update unit tests at tensorflow/python/keras/layers/preprocessing/image_preprocessing.py?

[tanzhenyu]

huh...? It seems that I need to update unit tests at tensorflow/python/keras/layers/preprocessing/image_preprocessing.py?

Yes

[WindQAQ]

huh...? It seems that I need to update unit tests at tensorflow/python/keras/layers/preprocessing/image_preprocessing.py?

It's at tensorflow/python/keras/layers/preprocessing/image_preprocessing_test.py.

Hi @Smankusors, if you feel difficult doing tests or don't mind that I take over your job, I can also fix it in #41365. Anyway, feel free to reach out me if there is any problem.

[WindQAQ]

I haven't validated if this fix is correct, but my implementation in addons is tested against scipy (well, just plugin in the map_coordinate function from scipy to TF), and gets the identical results.

tensorflow/addons#1721
https://github.com/scipy/scipy/blob/v1.5.2/scipy/ndimage/src/ni_interpolation.c#L43-L119

[tanzhenyu]

I haven't validated if this fix is correct, but my implementation in addons is tested against scipy (well, just plugin in the map_coordinate function from scipy to TF), and gets the identical results.

tensorflow/addons#1721
https://github.com/scipy/scipy/blob/v1.5.2/scipy/ndimage/src/ni_interpolation.c#L43-L119

I think this is very worth fixing -- would you mind taking it here?

[Smankusors]

It's at tensorflow/python/keras/layers/preprocessing/image_preprocessing_test.py.

Hi @Smankusors, if you feel difficult doing tests or don't mind that I take over your job, I can also fix it in #41365. Anyway, feel free to reach out me if there is any problem.

oh yeah damn I pasted a wrong path 😅

hmm I will try do this... Should my transform values be added to the unit tests?

[WindQAQ]

hmm I will try do this... Should my transform values be added to the unit tests?

Yes! You should fix the original one, and probably add some new test cases that will break in current implementation.

[Smankusors]

hmm unsurprisingly this happens on a first test unit (reflect fill, move the image to down by 1 pixel). I also tried apply_affine_transform from keras-preprocessing. It's.... hmm...

(second column is my patch)

It seems the fix is not as simple as I do 😕

[tanzhenyu]

hmm unsurprisingly this happens on a first test unit (reflect fill, move the image to down by 1 pixel). I also tried apply_affine_transform from keras-preprocessing. It's.... hmm...

(second column is my patch)

It seems the fix is not as simple as I do 😕

What's expected_output here?

[Smankusors]

What's expected_output here?

expected output is from the existing test unit

tensorflow/tensorflow/python/keras/layers/preprocessing/image_preprocessing_test.py

Lines 713 to 718 in cdfd111

	expected_output = np.asarray(
	[[0., 1., 2.],
	[0., 1., 2.],
	[3., 4., 5.],
	[6., 7., 8],
	[9., 10., 11]]).reshape((1, 5, 3, 1)).astype(np.float32)

[WindQAQ]

I fix it in #41365. It's because the nearest interpolation will round the coordinate. So for example, for "WRAP" fill mode, out_coord = -0.5, and len = 4, in_coord = 3.5, and will be rounded to 4. Since this value is larger than len - 1, it will be filled with fill_value_, resulting in weird dash lines.

Reflect:

Wrap:

[Smankusors]

I fix it in #41365. It's because the nearest interpolation will round the coordinate. So for example, for "WRAP" fill mode, out_coord = -0.5, and len = 4, in_coord = 3.5, and will be rounded to 4. Since this value is larger than len - 1, it will be filled with fill_value_, resulting in weird dash lines.

nice, I suppose I will close this PR because it doesn't work 😅