Abnormal fp16 inference results of TensorRT v10.0 when running engine converted from onnx of NAFNet

[HSDai]

Description

I tried to convert onnx to trt in FP16, the infering results are abnormal compared with FP32.

fp16:

fp32:

Environment

TensorRT Version:10.0

NVIDIA GPU:

NVIDIA Driver Version:

CUDA Version:

CUDNN Version:

Operating System:

Python Version (if applicable):

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

Model link:
onnx.zip

trt10_fp16.zip
trt10_fp32.zip

Steps To Reproduce

./trtexec --onnx=color_consistency_nafnet.onnx --saveEngine=nafnetcc75_t4_float16_v10.trtmodel --inputIOFormats=fp32:chw --outputIOFormats=fp32:chw --device=3 --minShapes=input:1x64x64x3 --optShapes=input:1x1024x1024x3 --maxShapes=input:1x1920x1920x3 --fp16

Commands or scripts:

Have you tried the latest release?:

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):

[lix19937]

To compare the diff between trt and ort.

polygraphy run model_sim.onnx --trt --onnxrt  --fp16 \
        --trt-min-shapes input:[1,64,64,3]   \
        --trt-opt-shapes input:[1,1024,1024,3] \
        --trt-max-shapes input:[1,1920,1920,3]     

[HSDai]

polygraphy 

Here is the polygraphy log. It seems like there are some issues with fp16.
fp16.log
fp32.log

The installation of the latest version of tensorrt failed, then I tired to use the following version.
If need to use the latest version of tensorrt , I'll give it another try.

[HSDai]

polygraphy log with "mark all" for all layers:
fp16_load_data.log

[zerollzeng]

Don't use 'mark all', it will break all tensorrt layer fusion.

[zerollzeng]

check polygraphy run color_consistency_nafnet.onnx --trt --fp16 --onnxrt --input-shapes input:[1,1024,1024,3] first.

BTW your nvidia driver version is pretty low, I would recommend to update to newer version.

[zerollzeng]

You can feed the real input, from the output image I feel some overflows are happened.

[HSDai]

check polygraphy run color_consistency_nafnet.onnx --trt --fp16 --onnxrt --input-shapes input:[1,1024,1024,3] first.

BTW your nvidia driver version is pretty low, I would recommend to update to newer version.

I tested TensorRT in a new environment, and the nvidia driver version is a bit higher.

Here is the log with real input.
fp16_1024.log

[lix19937]

You can use trtexec --verbose to save a log, and then upload here.

[HSDai]

You can use trtexec --verbose to save a log, and then upload here.

here is the log.
trtexec_fp16_verbose.log

[lix19937]

[06/12/2024-10:17:39] [W] [TRT] Detected layernorm nodes in FP16.

[06/12/2024-10:17:39] [W] [TRT] Running layernorm after self-attention in FP16 may cause overflow. Exporting the model to the latest available ONNX opset (later than opset 17) to use the INormalizationLayer, or forcing layernorm layers to run in FP32 precision can help with preserving accuracy.

You try to set layernorm run with fp32.

[HSDai]

Detected layernorm nodes in FP16

Thank you, that's helpful!
I will close this.