deeplab/eval.py got nan iou

[shanyucha]

models/research/deeplab/eval.py

Line 181 in 494cf0b

iou = tf.where(tf.greater(tp + fn, 0.0),

When calculate iou, we should pass the fn_op instead of fn to the calculation, or the iou would get nan.

iou = tf.where(tf.greater(tp + fn, 0.0),
tp_op / (tp + fn_op + fp),
tf.constant(np.NaN))

=>>>>
iou = tf.where(tf.greater(tp_op + fn_op, 0.0),
tp_op / (tp_op + fn_op + fp_op),
tf.constant(np.NaN))

[tensorflowbutler]

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
What is the top-level directory of the model you are using
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

[shanyucha]

I run the eval.py to evaluate training process. I made slight modifications to the official eval.py which does not matter mach.
OS: Ubuntu 14.04
tensorflow version: 1.14
python version: 2.7
cuda version: 10.0
cudnn version: 7.5
GPU memory: 2 kernels and each one is 8G
command line: python deeplab/eval_newest.py
--logtostderr
--eval_split="val"
--model_variant="xception_65"
--atrous_rates=6
--atrous_rates=12
--atrous_rates=18
--output_stride=8
--decoder_output_stride=1
--eval_crop_size=512,512
--dataset="mydata"
--eval_interval_secs=20
--max_number_of_evaluations=10000
--checkpoint_dir="xxxx"
--eval_logdir="xxxxx"
--dataset_dir="xxxxx"

[ankitvora7]

Same issue! I get almost all the miou's as nan values. Is there a solution to this?

[rgangu]

Any update at all to this issue?