Mask RCNN predicted class values and logits do not match

[amir-rahnama]

System information

• What is the top-level directory of the model you are using: mask_rcnn_inception_v2_coco_2018_01_28
• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.10
• TensorFlow installed from (source or binary): Binary
• TensorFlow version (use command below): 1.13.1
• Bazel version (if compiling from source): N/A
• CUDA/cuDNN version: N/A
• GPU model and memory: N/A
• Exact command to reproduce:

Describe the problem

I am using the "mask_rcnn_inception_v2_coco_2018_01_28" model downloaded from Zoo model (https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md).

I am trying to access the tensors that are holding the logits for each class for the image that is fed to the model. I found out that SecondStageBoxPredictor/Reshape_1 is holding those values, however when comparing them with detection_classes, I realized that the values do not match. I want to first find out where are the correct logits and then document this as it is useful to have these documented correctly.

Code

with tf.Session(graph=tf.Graph()) as sess:
    tf.saved_model.loader.load(sess, ['serve'], GRAPH_PB_PATH)
    graph = tf.get_default_graph()

    image = Image.open(IMAGE_PATH)
    (im_width, im_height) = image.size
    image_np = np.array(image.getdata()).reshape((im_height, im_width, 3)).astype(np.uint8)
    image_np_expanded = np.expand_dims(image_np, axis=0)
    
    image_tensor = tf.get_default_graph().get_tensor_by_name('image_tensor:0')

    tensor_dict = {}
    
    tensor_dict['SecondStageBoxPredictor/Reshape_1:0'] = graph.get_tensor_by_name('SecondStageBoxPredictor/Reshape_1:0')
    tensor_dict['detection_classes:0'] = graph.get_tensor_by_name('detection_classes:0')
    
    output_dict = sess.run(tensor_dict,
                             feed_dict={image_tensor: image_np_expanded})

If you compare the values like the following, they do not match:

logits = []
for i in range(0, 92):
    logits.append(np.argmax(output_dict['SecondStageBoxPredictor/Reshape_1:0'][i]))

print(logits)
print(output_dict['detection_classes:0'])

[amir-rahnama]

Btw, the same goes with using the following tensors, the logits do not match still:

tensor_dict['Reshape_12'] = graph.get_tensor_by_name('Reshape_12:0')
tensor_dict['scale_logits'] = graph.get_tensor_by_name('scale_logits:0')
tensor_dict['convert_scores'] = graph.get_tensor_by_name('convert_scores:0')
tensor_dict['BatchMultiClassNonMaxSuppression_1/map/TensorArrayStack_2/TensorArrayGatherV3'] = graph.get_tensor_by_name('BatchMultiClassNonMaxSuppression_1/map/TensorArrayStack_2/TensorArrayGatherV3:0')

for i in range(0, 92):
        np.argmax(output_dict['Reshape_12'][i])
        np.argmax(output_dict['scale_logits'][0][i])
        np.argmax(output_dict['convert_scores'][0][i])
        np.argmax(output_dict['BatchMultiClassNonMaxSuppression_1/map/TensorArrayStack_2/TensorArrayGatherV3'])

These do not match the output values presented by

print(output_dict['detection_classes:0'])