Create standalone application without python, c++ only, on windows 10

[stereomatchingkiss]

os : win10 64bits
libtorch: 1.7.1 cuda10.2

Could I create standalone application by this project? According to the deployment tutorial, I need to build the torchvision v0.8, else the model cannot be loaded. According to this post, Using torchvision C++ API without installing python, we still need to install python on the local machine if we want to use torchvision from c++. Thanks

[zhiqwang]

Hi @stereomatchingkiss ,

The main (and seems to be the only) ops we are using from torchvision is the NMS, and that is indeed a frequent problem. Actually I didn't have enough experience in CPP, if you can split NMS out and register it into a standalone app, I think it's a good way.

https://github.com/zhiqwang/yolov5-rt-stack/blob/f4bb74ca14496883bf5252c9699864eba675f90d/test/tracing/test_tracing.cpp#L4-L5

[zhiqwang]

BTW, I think that you could try pytorch 1.8.1 (with libtorch 1.8.1 and torchvision 0.9.1 are attached), the interfaces are refactored, maybe it could get rid of python's dependency.

You could check this Removing python headers and Win32 Python2 inits in C++ for more details.

UPDATE: there is an issue in torchvision about this: pytorch/vision#3965 .

[stereomatchingkiss]

Hi @stereomatchingkiss ,

The main (andseems to be the only) ops we are using from torchvision is the NMS, and that is indeed a frequent problem. Actually I didn't have enough experience in CPP, if you can split NMS out and register it into a standalone app, I think it's a good way.

https://github.com/zhiqwang/yolov5-rt-stack/blob/f4bb74ca14496883bf5252c9699864eba675f90d/test/tracing/test_tracing.cpp#L4-L5

Could you tell me where do you apply the nms? I would like to try to export the functions without this function. As a application developers, it would be nice to create a standalone app by libtorch/onnx only.

I believe many application developers would like to have an option to move out the dependencies of the torchvision, we can use the nms functions of opencv instead.

Thanks.

[zhiqwang]

I believe many application developers would like to have an option to move out the dependencies of the torchvision, we can use the nms functions of opencv instead.

That's a more prctical idea! However, the nms has been embedded in the torchscript model. You should separate PostProcess module from the current model structure, and rewrite a C++ version of PostProcess. That will be a little tricky.

BTW, did you try yasenh's libtorch-yolov5, actually they are using the opencv's NMS.

[stereomatchingkiss]

I believe many application developers would like to have an option to move out the dependencies of the torchvision, we can use the nms functions of opencv instead.

That's a more prctical idea! However, the nms has been embedded in the torchscript model. You should separate PostProcess module from the current model structure, and rewrite a C++ version of PostProcess. That will be a little tricky.

BTW, did you try yasenh's libtorch-yolov5, actually they are using the opencv's NMS.

Yap, tried, and work. Biggest advantages of this project, is it use script but not trace, this could save us a lot of memory. If I figure out how to separate it out and export the model successful, will open a pull request. It is quite painful to build torchivision on different platforms, I like c++, but it is really a big pain to build those libraries on different platforms, almost everyone have their own issues to solve.

[zhiqwang]

Biggest advantages of this project, is it use script but not trace, this could save us a lot of memory. If I figure out how to separate it out and export the model successful, will open a pull request. It is quite painful to build torchivision on different platforms.

We welcome for this feature, and I can help to separete the nms out if you encounter any problems. Or you can pull request first, let's work for this together :)

[zhiqwang]

Hi @stereomatchingkiss ,

Because we now have support for Windows C++ with ONNX Runtime and TensorRT. I believe this issue can be resolved with these two ways, and as such I'm closing this issue but let us know if you have further questions.

Context

• https://github.com/zhiqwang/yolov5-rt-stack/tree/main/deployment/onnxruntime
• https://github.com/zhiqwang/yolov5-rt-stack/tree/main/deployment/tensorrt