-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Description
Description
I tried to export a GAN model to tensorrt to get accelerated. But the engine file accuracy was lower than onnx model. While use fp32, the accuracy is correct. Then I follow the polygraphy tutorial to debug accuracy.
The debug result shows after the second conv node, fp16 failed. I use graph surgeon to export the subgraph onnx.
But in python, it seems like there is no approach to solve this problem. Could I set some layers to fp32, some layers to fp16? For example, conv1 set to fp16, conv2 set to fp32. Thanks for your help.
Environment
TensorRT Version: 8.5.1.7
NVIDIA GPU: 2080ti
NVIDIA Driver Version:
CUDA Version: 10.2
CUDNN Version: 8.6
Operating System: ubuntu20.04
Python Version (if applicable): 3.9
Tensorflow Version (if applicable):
PyTorch Version (if applicable): 1.9
Baremetal or Container (if so, version):