New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Has anyone implemented it with tensorrt? #84

Open

SanghyunPark01 opened this issue Mar 2, 2024 · 1 comment

SanghyunPark01 commented Mar 2, 2024

I'd like to use something implemented with tensorrt(c++) for this. Does anyone else implement it?

aRibra commented Mar 7, 2024 •

edited

I'd like to use something implemented with tensorrt(c++) for this. Does anyone else implement it?

You need to convert the model first to be able to build the TenosrRT engines. Different conversion paths are available in the TensorRT framework. For more info please refer to documentation in the following link: https://docs.nvidia.com/deeplearning/tensorrt/quick-start-guide/index.html#conversion

IMO the easiest conversion path is through exporting the model to ONNX graph, then, using exported ONNX file to build the TensorRT engines.

Others are having problems exporting the model to ONNX using torch.onnx.export. (related to the following issue: #79)

Maybe using ONNX with opset 20 will work as suggested here:
pytorch/pytorch#100790 (comment)

If you successfully build the TensorRT engine, then you can load it and use it in C++ or Python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment