tf.image.resize_* gives negative results

[Luonic]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
• TensorFlow installed from (source or binary): source
• TensorFlow version (use command below): v1.8.0-7-g3b85959 1.8.0
• Python version: 3.5.2
• Bazel version (if compiling from source): 0.14.0
• GCC/Compiler version (if compiling from source): 5.4.0
• CUDA/cuDNN version: N/A
• GPU model and memory: N/A
• Exact command to reproduce: N/A

Describe the problem

I am trying to segment images using Pascal VOC 2012. I converted indexed pngs to grayscale, replaced pixels with value 255 with 0 and written them to tfrecord. Then when reading image with tf.data.Dataset i want to resize all images to constant scale at min dim and then randomly crop to square. Problem is that when i am resizing label tensor with tf.image.resize_bicubic some values are negative but in input tensor threre is no negative values.

UPD: Bicubc not only gives negative values but also changes it where it should not
Look at the right image:

Source code / logs

Here is run log:

uint8_label_after_decode[0]
uint8_label_after_decode[0]
uint8_label_after_decode[0]
uint8_label_after_decode[0]
uint8_label_after_resize[-2.43790507]
uint8_label_after_resize[-0.208821028]
uint8_label_after_resize[-3.43527746]
uint8_label_after_resize[-2.6129365]
2018-06-09 19:45:09.765736: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at sparse_xent_op.cc:90 : Invalid argument: Received a label value of -2 which is outside the valid range of [0, 21). 

def resize_image_keep_aspect_ratio(image, max_height, max_width, use_min_ratio, use_nn_interpolation=False):
    def compute_new_dims(height, width, max_height, max_width, use_min_ratio):
        # If use_min_ratio is set to true than image will be resized to max of smaller dim
        height_float = tf.cast(height, tf.float32)
        width_float = tf.cast(width, tf.float32)
        max_height_float = tf.cast(max_height, tf.float32)
        max_width_float = tf.cast(max_width, tf.float32)

        height_ratio = height_float / max_height_float
        widht_ratio = width_float / max_width_float

        if use_min_ratio:
            ratio = tf.minimum(height_ratio, widht_ratio)
        else:
            ratio = tf.maximum(height_ratio, widht_ratio)

        new_height = tf.cast(tf.floor(height_float / ratio), tf.int32)
        new_width = tf.cast(tf.floor(width_float / ratio), tf.int32)

        return (new_height, new_width)

    shape = tf.shape(image)
    height = shape[0]
    width = shape[1]

    new_height_and_width = compute_new_dims(height, width, max_height, max_width, use_min_ratio=use_min_ratio)

    image = tf.expand_dims(image, 0)
    if use_nn_interpolation:
        image = tf.image.resize_nearest_neighbor(image, tf.stack(new_height_and_width), align_corners=True)
    else:
        image = tf.image.resize_bicubic(image, tf.stack(new_height_and_width), align_corners=True)
    image = tf.squeeze(image, [0])
    return image


# Here I am decoding image in tf.data.Dataset
label = tf.image.decode_image(example_parsed['label'], channels=1)
label = tf.Print(label, [tf.reduce_min(label)], 'uint8_label_after_decode')
label = tf.cast(label, tf.float32)
label = resize_image_keep_aspect_ratio(label, image_size[0], image_size[1], use_min_ratio=True, 
  use_nn_interpolation=False)
label = tf.Print(label, [tf.reduce_min(label)], 'uint8_label_after_resize')
image_and_label = tf.concat([image, label], axis=2)
cropped_image_and_label = tf.random_crop(image_and_label, [image_size[0], image_size[1], 4])
image, label = tf.split(cropped_image_and_label, [3, 1], axis=2)
image = tf.cast(image, float_type) * (2.0 / 255.0) - 1.0
label = tf.cast(label, tf.int64)

[karmel]

Hi @Luonic -- I'm having trouble interpreting what exactly the problem is here. bicubic_resize would be expected to change the values in the input tensor, as the same image is represented in fewer pixels. Can you provide a minimal example, using just the resize_* method you think has a bug, to demonstrate the problem?

[Luonic]

@karmel when we resizing image that have no negative values we expect result resized image to not have them too because linear interpolation just interpolates values beetween two non-negative values in source tensor. In my example we see that after resize we have negative values in result tensor but in source tensor all values were positive.
So here is minimal example:

import tensorflow as tf

image = tf.random_uniform((1, 512, 512, 1), minval=0, maxval=255, dtype=tf.float32)
image = tf.Print(image, [tf.reduce_min(image)], 'min of source image')
image = tf.image.resize_bicubic(image, [256, 256], align_corners=True)
image = tf.Print(image, [tf.reduce_min(image)], 'min of result image')

sess = tf.Session()
image.eval(session=sess)

And output is:

min of source image[9.11951065e-05]
min of result image[-47.5840836]

Why do we have -47.5840836 in our resized image?

[karmel]

@cwhipkey - any thoughts on whether this is an expected output for resize_bicubic?

[cwhipkey]

I'm not an expert, but I think that this can happen because of overshooting as described on the wikipedia page for bicubic interpolation: https://en.wikipedia.org/wiki/Bicubic_interpolation#Use_in_computer_graphics .

…

On Wed, Jun 27, 2018 at 2:56 PM Karmel Allison ***@***.***> wrote: @cwhipkey <https://github.com/cwhipkey> - any thoughts on whether this is an expected output for resize_bicubic? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19880 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQw4wamlUWlKViuLxhm8dDhVsTTzQf2Vks5uA_96gaJpZM4UhaSV> .

[karmel]

@shlens -- any chance you have some insight on the expected output of bicubic interpolation in this case?

[shlens]

I agree with @cwhipkey. This is potentially expected behavior with bicubic interpolation and a known limitation. This could be verified by running the same minimal test in OpenCV which has a well-tested version of bicubic interpolation and see if you observe the same result.

[karmel]

Great-- @Luonic , have you checked the results with OpenCV?

[tensorflowbutler]

We are closing this issue for now due to lack of activity. Please comment if this is still an issue for you. Thanks!

[rafariva]

Hi, how do you fix this annoying problem? my image is between [0-255] (min 5, max 248), when interpole bicubic, it change between [no idea, to no idea] (and my min become -3 and max 260),
it loose the relationship, this NOT happen if us NEAREST_NEIGHBOR