Detectron2 with OpenVino

[ivanpanshin]

I know that one can deploy a detectron2 model using a converter to Caffe2. But you also added an option to convert a model to ONNX. But, there a note saying "Note that the exported model contains custom ops only available in caffe2, therefore it cannot be directly executed by other runtime". So, my question is: does it mean that I cannot use model converted to ONNX with OpenVino?

[ArutyunovG]

Yes, you cannot use ONNX model directly with OpenVINO. you need to patch both Model Optimzier and Inference Engine.

[ppwwyyxx]

We in general cannot provide answers for how to use detectron2 + X for all the possible X in the world. But anyone who has experience on that is welcome to comment.

[ivanpanshin]

Yes, you cannot use ONNX model directly with OpenVINO. you need to patch both Model Optimzier and Inference Engine.

Any links to see how it's done? I mean, patching Model optimizer and Inference Engine

[ArutyunovG]

Unfortunately not aware of such links. At my workplace this is even sort of a separate internal job for deployment on Intel devices. We first patch Model Optimizer with some new IR layers then reimplement them on the dedicated OpenVINO plugin.

As a side note, this inconsistency in detection op implementation details is likely to stop if ONNX supports detection ops like GenerateProposals, BBoxTransform. BoxWithNMSLimit and ROIAlign (of all kinds: v1, v2 and TF's CropAndResize). In near future this seems quite unlikely to happen, because those ops can probably be implemented as ONNX-functions. It is also worh noting, that research experimentation and production-oriented tools will anyway have some time-gap in between.

[ivanpanshin]

Oh, man. In terms of Detectron2 vs MMdetection I always thought "MMdetection for competitions, Detectron - for production", but now it seems that detectron's production advantages are limited by just converting models to Caffe2.