dnn can't load mask-rcnn onnx model, error(-212:Parsing error)

[GilbertPan97]

System Information

Environment:

• OpenCV: 4.6.0
• Platform: ubuntu 20.04
• pytorch 1.11.0

Detailed description

Export pytorch model to onnx code:

# export the model
dy_axes = {'input': {0: 'batch', 2: 'height', 3: 'width'}}
input_label = ['in_img']
output_label = ['out_box', 'out_accept', 'out_score', 'out_mask']
torch.onnx.export(pytorch_model,                # model being run
                  img,                          # model input (or a tuple for multiple inputs)
                  onnx_model_name,              # where to save the model (can be a file or file-like object)
                  export_params=True,           # store the trained parameter weights inside the model file
                  opset_version=11,             # the ONNX version to export the model to
                  do_constant_folding=True,     # whether to execute constant folding for optimization
                  input_names=input_label,      # the model's input names
                  output_names=output_label,    # the model's output names
                  dynamic_axes={})              # variable length axes

Report error:

terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.6.0) /home/fanuc/fanuc/SDK_ws/opencv-4.6.0/modules/dnn/src/graph_simplifier.cpp:76: error: (-212:Parsing error) Input node with name onnx::Div_766 not found in function 'getInputNodeId'

The report show Div_766 not found, however, I review my model on netron, Div_766 exists (as shown in fig). Moreover, I test my onnx model on python-onnx, it can work correctly.

Here is my mask-rcnn onnx model:
https://1drv.ms/u/s!ArseFaneAcYbwV2H1spjyCEcOsUZ?e=Cmu3Xj

Steps to reproduce

int main(int argc, char** argv) {
	// load model
	cv::dnn::Net net = cv::dnn::readNetFromONNX(bin_model);  // load mask-rcnn onnx model
}

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
• I updated to the latest OpenCV version and the issue is still there
• There is reproducer code and related data files (videos, images, onnx, etc)

[fengyuentau]

It won't work even if your issue is solved: DNN does not support ONNX models with dynamic input shape (dynamic height & width). Try with exporting without dynamic_axes and simplifying the model before-hand with onnx-simplifier, and you should have a great chance loading and inferring the model with OpenCV DNN without issues.

[GilbertPan97]

It won't work even if your issue is solved: DNN does not support ONNX models with dynamic input shape (dynamic height & width). Try with exporting without dynamic_axes and simplifying the model before-hand with onnx-simplifier, and you should have a great chance loading and inferring the model with OpenCV DNN without issues.

My onnx model is statically imported. The dynamic_axes is not defined when export ONNX model. I'll give it a try based on your comments, thanks a lot.

[GilbertPan97]

It won't work even if your issue is solved: DNN does not support ONNX models with dynamic input shape (dynamic height & width). Try with exporting without dynamic_axes and simplifying the model before-hand with onnx-simplifier, and you should have a great chance loading and inferring the model with OpenCV DNN without issues.

I simplified the onnx model according to your comments. However, when I loaded the simplified model, the function threw a new error as follows:

terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.6.0) /home/fanuc/fanuc/SDK_ws/opencv-4.6.0/modules/dnn/src/onnx/onnx_graph_simplifier.cpp:842: error: (-210:Unsupported format or combination of formats) Unsupported data type: BOOL in function 'getMatFromTensor'

I notice that a related problem has been raised in Issue, which has not been solved. Does this mean the mask-rcnn onnx model cannot be load with cv::dnn::readNetFromONNX

[fengyuentau]

onnx-simplifier cannot work with your model.

Does this mean the mask-rcnn onnx model cannot be load with cv::dnn::readNetFromONNX

I am afraid the answer is yes for now. However, the tensorflow one seems to work with dnn: https://github.com/opencv/opencv/wiki/Deep-Learning-in-OpenCV#tensorflow-1. You could take a look at this and meanwhile I will look into your problem.

[GilbertPan97]

onnx-simplifier cannot work with your model.

Does this mean the mask-rcnn onnx model cannot be load with cv::dnn::readNetFromONNX

I am afraid the answer is yes for now. However, the tensorflow one seems to work with dnn: https://github.com/opencv/opencv/wiki/Deep-Learning-in-OpenCV#tensorflow-1. You could take a look at this and meanwhile I will look into your problem.

Maybe my file got corrupted while uploading to onedrive, you can try this, both onnx model and simplified model are in the compressed folder:
https://drive.google.com/file/d/1d3kxAOaBKI036LrBynkhaE0yx9R3w9Or/view?usp=share_link

[fengyuentau]

Thanks for the simplified model! Although there seems to be only one issue when trying to read the model with opencv dnn, which is Unsupported data type: BOOL, I still see there is at least one operator (NonMaxSuppression) that opencv dnn does not support.

Could you share the pytorch mask-rcnn repo to me if possible? I would suggest removing the post-processing stage when exporting to ONNX model to get rid of non-supported operators for a quick fix.

[GilbertPan97]

Thanks for the simplified model! Although there seems to be only one issue when trying to read the model with opencv dnn, which is Unsupported data type: BOOL, I still see there is at least one operator (NonMaxSuppression) that opencv dnn does not support.

Could you share the pytorch mask-rcnn repo to me if possible? I would suggest removing the post-processing stage when exporting to ONNX model to get rid of non-supported operators for a quick fix.

Thanks for you reply. I have publicized the pytorch mask-rcnn repo, which are available on https://github.com/GilbertPan97/MaskRCNN-Pytorch, and the dataset can be download on https://drive.google.com/file/d/14Ebfl0ojDB7xJqbqCk_1KELBtILrNCKU/view?usp=sharing

You can decompress the dataset to the root directory and run the training script