Inconsistent Results between TensorRT and PyTorch when Converting Torch Models using Conv1d, Conv2d, and Conv3d Operators

[hongliyu0716]

Description

I have encountered an issue while converting Torch models to TensorRT, where the output results from TensorRT differ from those obtained in PyTorch. This inconsistency specifically arises when using the Conv1d, Conv2d, and Conv3d operators.

Environment

TensorRT Version: 8.6.1.6

NVIDIA GPU: NVIDIA GeForce RTX 3090

NVIDIA Driver Version: 525.116.03

CUDA Version: 12.0

CUDNN Version: 8.9.0

Operating System: Ubuntu 18.04

Python Version: 3.9.18

PyTorch Version: 2.1.1

torch2trt Version: 0.4.0

Steps To Reproduce

Commands or scripts:

1. Conv3d

from torch2trt import torch2trt
import torch
from torch.nn import Module

model = torch.nn.Conv3d(2,3,3,2,(1, 2, 3),1,1,True,'circular',).cuda()
input_data = torch.randn([2, 2, 4, 4, 4], dtype=torch.float32).cuda()
model_trt = torch2trt(model, [input_data])
y = model(input_data)
y_trt = model_trt(input_data)

# check the output against PyTorch
print(torch.max(torch.abs(y - y_trt)))

The output is tensor(1.1362, device='cuda:0', grad_fn=<MaxBackward1>)

2. Conv2d

import torch
from torch.nn import Module
from torch2trt import torch2trt

model = torch.nn.Conv2d(12,6,(3, 2),(1, 2),(1, 1),(1, 1),3,False,'circular',).eval().cuda()
input_data = torch.randn([2, 12, 5, 7], dtype=torch.float32).cuda()
model_trt = torch2trt(model, [input_data])
output = model(input_data)
output_trt = model_trt(input_data)

print(torch.max(torch.abs(output - output_trt)))

The output is tensor(1.1816, device='cuda:0', grad_fn=<MaxBackward1>)
3. Conv1d

import torch
from torch.nn import Module
from torch2trt import torch2trt

model = torch.nn.Conv1d(2,3,3,2,(1,),1,1,True,'reflect',).eval().cuda()
input_data = torch.randn([2, 2, 4], dtype=torch.float32).cuda()
model_trt = torch2trt(model, [input_data])
output = model(input_data)
output_trt = model_trt(input_data)

print(torch.max(torch.abs(output - output_trt)))

The output is tensor(0.8214, device='cuda:0', grad_fn=<MaxBackward1>)

I would greatly appreciate any insights or guidance on resolving this issue.

[zerollzeng]

@hongliyu0716 I noticed you filed several similar issue, are you doing some study on the DL framework accuracy?

Also here are some suggestions that can save the time for us to triage/debug your issue:

1. TensorRT doesn't guarantee to provide bit-wise accuracy with other framework.
2. Would be great if you can compare the accuracy between TRT and onnxruntime, this can be done quickly with polygraphy. e.g. polygraphy run model.onnx --trt --onnxrt --atol 1e-4 --rtol 1e-4. Because sometimes the issue happens when export the onnx.

[hongliyu0716]

@hongliyu0716 I noticed you filed several similar issue, are you doing some study on the DL framework accuracy?

Also here are some suggestions that can save the time for us to triage/debug your issue:

1. TensorRT doesn't guarantee to provide bit-wise accuracy with other framework.
2. Would be great if you can compare the accuracy between TRT and onnxruntime, this can be done quickly with polygraphy. e.g. polygraphy run model.onnx --trt --onnxrt --atol 1e-4 --rtol 1e-4. Because sometimes the issue happens when export the onnx.

Thank you for your response and suggestions. I have followed your advice and executed the polygraphy tool with the command polygraphy run model.onnx --trt --onnxrt --atol 1e-4 --rtol 1e-4. The results of the execution indicate that there are no issues, which is encouraging. BTW, I would like to inquire whether this bug could potentially be caused by the torch2trt tool.

[zerollzeng]

Maybe, because the torch2trt is not maintained by us. our onnx-tensorrt is well maintained and open-source. so 3rd-party framework -> onnx -> TRT is suggestion pipeline, otherwise the user need to figure out whether the issue was cause by parser.

Could you please use polygraphy to check your other new-opened issue? if it's good then please close. Thanks!

[hongliyu0716]

Thank you for your response, I'll close these issues.