What files to modify to support TensorRT engine models

[alsozatch]

Hi, I'm using TensorRT to quantize Yolov8 models, so we get something like yolov8n.engine model files. They can typically be used in exactly the same way as yolov8n.pt files when using Ultralytics, but with SAHI it does not work. I'm looking for where I need to make changes for .engine to work in SAHI, and looking into the file structure and I notice there are some code files in utils/ like yolov8.py and yolov8onnx.py, so perhaps I need to add a yolov8engine.py file.

Just trying to understand the codebase and would appreciate if you could tell me where specifically I need to make changes for SAHI to work with a .engine. Thanks.

[Kanan99]

Hi

Did you find any potential solution to support the TRT engine within SAHI inference?

Thanks in advance :)

[Kanan99]

I actually was looking for V5. However, there shouldn't be that much difference between the implementations.

I'll check and let you know.

[alsozatch]

It actually does work after some tinkering with Sahi and a few non-obvious errors. The bottom line for changes is that the .engine files do not store any of the metadata that you need to load/inference with the model in YOLO. You must record them separately and load them into SAHI. I had it working with harcoded values and now looking at where in the SAHI code can I add some loading of another config file. May pull request if I get it to a good spot

[fcakyon]

Hey @alsozatch, It would be fantastic if you could come up with a PR that doesn't change the old behaviour dramatically, I would gladly review it 🚀

[alsozatch]

It needs to be able to accept a config file. Metadata including imgsz and category mapping, which are already passed through in separate fields, but also some additional fields not currently accounted for in the code.

Rather than add additional fields, my current implementation just uses that config_path field that's already passed through. The config_path is no longer used to add to **kwargs though.

The config file is required, unlike the other engine formats. But otherwise it works in exactly the same manner as other models. I have a script to write a config file given a yolov8 PyTorch model file as well.

[alsozatch]

Hey @alsozatch, It would be fantastic if you could come up with a PR that doesn't change the old behaviour dramatically, I would gladly review it 🚀

#1046

[alsozatch]

#1046