❓ [Question] How to differentiate a Torch-TensorRT model from a pure TorchScript model?

[tiandi111]

❓ Question

I'm developing a C++ inference server to deploy Torch-TensorRT models and TorchScript models. Since the Torch-TensorRT compilation process is done AOT, Is there a way to know wether the given .pt model file is a Torch-TensorRT model or a pure TorchScript model?

Thanks!

[narendasan]

You can search through the graph (my_script_model.graph) and look for operations from the TensorRT namespace. But really you shouldn't need to do anything special. Just loading the torch-tensorrt runtime in your inference server will add support for Torch-TensorRT models but will not interfere with standard TorchScript models.

[tiandi111]

You can search through the graph (my_script_model.graph) and look for operations from the TensorRT namespace. But really you shouldn't need to do anything special. Just loading the torch-tensorrt runtime in your inference server will add support for Torch-TensorRT models but will not interfere with standard TorchScript models.

Thank you very much!