Run Converted .pb model by using OpenCV's DNN Module

[kaanaykutkabakci]

Hello, i have generated a .h5 model by using https://github.com/matterport/Mask_RCNN code. Then, i have converted the .h5 model to .pb by using https://github.com/bendangnuksung/mrcnn_serving_ready code.

After that, to generate .pbtxt file, i am trying to use
https://github.com/opencv/opencv/blob/master/samples/dnn/tf_text_graph_mask_rcnn.py code. I am getting following error:

Traceback (most recent call last):
File "tf_text_graph_mask_rcnn.py", line 86, in
assert(graph_def.node[0].op == 'Placeholder')
IndexError: list index out of range

I am getting same error, even if i create sorted_graph by using following code:

import tensorflow as tf
from tensorflow.tools.graph_transforms import TransformGraph
with tf.gfile.FastGFile('C:/Users/Dell/Desktop/learnopencv-master/learnopencv-master/Mask-RCNN/IG/mask_frozen_graph.pb ', 'rb') as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())
    graph_def = TransformGraph(graph_def, ['image_tensor'], ['detection_boxes', 'detection_classes', 'detection_scores', 
    'num_detections'], ['sort_by_execution_order'])
     with tf.gfile.FastGFile('sorted_mask_inference_graph.pb', 'wb') as f:

    f.write(graph_def.SerializeToString())

After i have tried all of these, i have tested with OpenCV's dnn module the generated .pb file with a previosly working .pbtxt file i am getting following error:

File "mask_rcnn.py", line 108, in
net = cv.dnn.readNetFromTensorflow(modelWeights, textGraph);
cv2.error: OpenCV(4.1.2) C:\projects\opencv-python\opencv\modules\dnn\src\tensorflow\tf_importer.cpp:1583: error: (-2:Unspecified error) Input layer not found: Preprocessor/mul/x in function 'cv::dnn::dnn4_v20190902::`anonymous-namespace'::TFImporter::populateNet'

graph.pbtxt is working .pbtxt file,
mask_frozen_graph.pb is converted from mask_rcnn_signal_0040.h5 file,
NucleusConfig.txt is configuration file that is used for training MatterPort's code,
pipeline__.config is trying to use for generate .pbtxt with tf_text_graph_mask_rcnn.py,

at the atteched Drive link: https://drive.google.com/open?id=1kNDP09h8ykHWcYSIs_g-1f3-khfXD544

I am waiting for your help. I want to run MatterPort's model with OpenCV dnn module. I do not want to use Tensorflow OD API, because I'm not sure to get as successful as the matterport's model and also I am not sure how should i create training folder structure.

[kaanaykutkabakci]

Is not there any solution? @dkurt

[dkurt]

For now OpenCV does not support Matterport's Mask R-CNN so this issue is labeled as "feature" because requires deep investigation.

[kaanaykutkabakci]

For now OpenCV does not support Matterport's Mask R-CNN so this issue is labeled as "feature" because requires deep investigation.

Thank you for your interest @dkurt.

[parthasgouda]

@dkurt Is there any solution yet to the matterport maskrcnn model in opencv dnn? Will appreciate any help in this regard. Thanks

[dkurt]

@parthasgouda, no progress for now - the task requires to do research how Matterpor's Mask R-CNN matches existing TensorFlow implementation path.

[kaanaykutkabakci]

Hello @parthasgouda . I am using Tensorflow Object Detection API to construct custom Mask R-CNN model. It works just as well as MatterPort. I can even say that it is better than MatterPort in some cases.

[madrugou]

I performed the test with the mask_rcnn_inception_v2_coco_2018_01_28 model from TensorFlow's object detection API.

The model provided by Tensorflow worked fine, however when I try to load a trained model with custom data I face several read errors, is there any procedure to perform the step by step inference of a frozen_inference_graph.pb with custom data?