error in test_contextual_attention()

[masadcv]

While running the following:
python inpaint_ops.py --imageA examples/style_transfer/bnw_butterfly.png --imageB examples/style_transfer/bike.jpg --imageOut examples/style_transfer/bike_style_out.png

I get the following error:
Traceback (most recent call last):
File "/opt/virtual_tensorflow/g_tf_neuralgym/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 558, in set_shape
unknown_shape)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimension 1 in both shapes must be equal, but are 166 and 64. Shapes are [1,166,250,2] and [1,64,64,2].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "inpaint_ops.py", line 513, in
test_contextual_attention(args)
File "inpaint_ops.py", line 360, in test_contextual_attention
training=False, fuse=False)
File "inpaint_ops.py", line 313, in contextual_attention
offsets.set_shape(int_bs[:3] + [2])
File "/opt/virtual_tensorflow/g_tf_neuralgym/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 561, in set_shape
raise ValueError(str(e))
ValueError: Dimension 1 in both shapes must be equal, but are 166 and 64. Shapes are [1,166,250,2] and [1,64,64,2].

I can see this has been logged before here: #67
But cant find any fix. Can you please let me know how I can fix this?

[JiahuiYu]

What image are you using? A simple fix can be commenting the lines related to offsets. Offsets are only used to visualize.

[masadcv]

I am using the command and images provided in the repository:
python inpaint_ops.py --imageA examples/style_transfer/bnw_butterfly.png --imageB examples/style_transfer/bike.jpg --imageOut examples/style_transfer/bike_style_out.png

--imageA examples/style_transfer/bnw_butterfly.png
--imageB examples/style_transfer/bike.jpg
--imageOut examples/style_transfer/bike_style_out.png

[JiahuiYu]

@devkicks I don't know on your side it returns error - on my side it is fine. Try to remove all offsets related code, and set flow to None.

[masadcv]

Many thanks for checking this. With offset calculation included, I still get the error. I commented out all things related to offset calculation and now it is fine. I am using tensorflow 1.12.0 on Ubuntu 16.

[JiahuiYu]

@devkicks Many thanks for your information and I think others may also find it helpful.

[daa233]

Hi, @devkicks and @JiahuiYu , I have met the same error with tensorflow 1.12.0 and it works fine if I change the tensorflow version to 1.7.0.

[JiahuiYu]

Thanks a lot for your report.

[shenjackson]

So, how to solve this problem when I use tensorflow 1.12.0?

[JiahuiYu]

@shenjackson

Many thanks for checking this. With offset calculation included, I still get the error. I commented out all things related to offset calculation and now it is fine. I am using tensorflow 1.12.0 on Ubuntu 16.

[shenjackson]

I use tensorflow 1.12.0 on CentOS7.... I don't know how to remove all offsets related code, and set flow to None. But I will try my best to solve it = =

[shenjackson]

haha, I just solved this problem.

[JiahuiYu]

@shenjackson If possible, please share your experience so others can have a reference as well. :)

[shenjackson]

OK！ I commented on the relevant code about “flow”，and it can work fine.

like this

"""
print(offsets)
offsets = tf.concat(offsets, axis=0)
offsets.set_shape(int_bs[:3] + [2])
# case1: visualize optical flow: minus current position
h_add = tf.tile(tf.reshape(tf.range(bs[1]), [1, bs[1], 1, 1]), [bs[0], 1, bs[2], 1])
w_add = tf.tile(tf.reshape(tf.range(bs[2]), [1, 1, bs[2], 1]), [bs[0], bs[1], 1, 1])
offsets = offsets - tf.concat([h_add, w_add], axis=3)
# to flow image
flow = flow_to_image_tf(offsets)
# # case2: visualize which pixels are attended
# flow = highlight_flow_tf(offsets * tf.cast(mask, tf.int32))
if rate != 1:
    flow = resize(flow, scale=rate, func=tf.image.resize_nearest_neighbor)
"""
flow = []
return y, flow