Convert Yolov4

[Sayyam-Jain]

Hi, I have a yolov4 model, that I want to run on TensorRT INT8. I read the documentation but having a hard time following it as an English speaker. Can you please guide me on how do I convert the model and prepared dataset for ProgramEntrance.py script. I have dataset in Yolo format.

Thanks

[ZhangZhiPku]

We feel so sorry about the unreadable documentation, here are the tips for converting yolo4:
Fristly, load your model via ppq.api:

from ppq.api import load_onnx_graph
graph = load_onnx_graph(onnx_import_file='Yolo4.onnx')

Then, prepare your dataset as a list of torch.Tensor:

# You should load calibration imgs from disk.
calibration_dataset = [torch.zeros(size=[1, 3, 640, 640])]
collate_fn          = lambda x: x.cuda()

After calibration dataset once loaded, convert your model via:

from ppq.api import quantize_native_model, ENABLE_CUDA_KERNEL, export_ppq_graph
from ppq import TargetPlatform, graphwise_error_analyse

with ENABLE_CUDA_KERNEL():
    quantized = quantize_native_model(
        model=graph, calib_dataloader=calibration_dataset, calib_steps=8,
        collate_fn=collate_fn, input_shape=[1,3,640,640], 
        platform=TargetPlatform.TRT_INT8)

    graphwise_error_analyse(
        graph=quantized, running_device='cuda', 
        dataloader=calibration_dataset, collate_fn=collate_fn)
    
    export_ppq_graph(
        graph=quantized, platform=TargetPlatform.TRT_INT8, 
        graph_save_to='Quantized.onnx', config_save_to='Quantized.json')

You will get a file named Quantized.onnx and a file named Quantized.json eventually, use script here for generating the executable TensorRT engine.

Notice that engine is not portable, that is to say once binary engine is generated, you are not supposed to move it to elsewhere. The only way for transporting a quantized model is sending Quantized.onnx and Quantized.json and generating the engine again.