Description
my model is trained in pytorch
then quantized using pytorch-quantization in tensorrt/tools
then exported to onnx
then build engine from onnx
the error in title occured when i parsed onnx model to build engine in my jetson xaiver nx
(i tested this pipeline in gpu tensorrt 8.5, and no error occured)
this is the log context:
[TensorRT] VERBOSE: Eliminating concatenation node_of_outputs_coords
[TensorRT] VERBOSE: Generating copy for 15813 to outputs_coords because input does not support striding.
[TensorRT] VERBOSE: Generating copy for 15815 to outputs_coords because input does not support striding.
[TensorRT] VERBOSE: Generating copy for 15817 to outputs_coords because input does not support striding.
[TensorRT] VERBOSE: Generating copy for 15819 to outputs_coords because input does not support striding.
[TensorRT] VERBOSE: Generating copy for 15821 to outputs_coords because input does not support striding.
[TensorRT] VERBOSE: Generating copy for 15823 to outputs_coords because input does not support striding.
[TensorRT] VERBOSE: After concat removal: 3085 layers
[TensorRT] VERBOSE: Graph construction and optimization completed in 220.024 seconds.
[TensorRT] INFO: ---------- Layers Running on DLA ----------
[TensorRT] INFO: ---------- Layers Running on GPU ----------
[TensorRT] INFO: [GpuLayer] node_of_1001_quantize_scale_node
[TensorRT] INFO: [GpuLayer] node_of_inputs
...##(too much layers output, more than 3000 layers, )
[TensorRT] INFO: [GpuLayer] node_of_14036
[TensorRT] INFO: [GpuLayer] node_of_15819
[TensorRT] INFO: [GpuLayer] node_of_13155
[TensorRT] INFO: [GpuLayer] node_of_15817
[TensorRT] INFO: [GpuLayer] node_of_12274
[TensorRT] INFO: [GpuLayer] node_of_15815
[TensorRT] INFO: [GpuLayer] 15813 copy
[TensorRT] INFO: [GpuLayer] 15815 copy
[TensorRT] INFO: [GpuLayer] 15817 copy
[TensorRT] INFO: [GpuLayer] 15819 copy
[TensorRT] INFO: [GpuLayer] 15821 copy
[TensorRT] INFO: [GpuLayer] 15823 copy
[TensorRT] VERBOSE: Using cublas a tactic source
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +227, GPU +209, now: CPU 978, GPU 4493 (MiB)
[TensorRT] VERBOSE: Using cuDNN as a tactic source
[TensorRT] INFO: [MemUsageChange] Init cuDNN: CPU +307, GPU +306, now: CPU 1285, GPU 4799 (MiB)
[TensorRT] WARNING: Detected invalid timing cache, setup a local cache instead
[TensorRT] VERBOSE: Constructing optimization profile number 0 [1/1].
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1285, GPU 4811 (MiB)
[TensorRT] ERROR: 2: [shuffleBuilder.cpp::addSupportedFormats::50] Error Code 2: Internal Error (Assertion formats.nbInputs() == 1 || formats.nbInputs() == 2 failed.)
[TensorRT] ERROR: 2: [builder.cpp::buildSerializedNetwork::417] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed.)
this error seems to be an internal error reported from tensorrt,
i searched this error in google, yet found no meaningful information,
i want to know which layer causes this error, then i can use some ops to substitute, but i cannot understand what does this error mean, this is where I need help
Environment
TensorRT Version: 8.0.1.6
NVIDIA GPU: jetson
NVIDIA Driver Version: jetpack 5.0
CUDA Version: 10.2
Operating System: Ubuntu 18.04.5 LTS
Python Version (if applicable): 3.6.9
PyTorch Version (if applicable): 1.11.0a0+17540c5
Baremetal or Container (if so, version):
Relevant Files
Steps To Reproduce
Description
my model is trained in pytorch
then quantized using pytorch-quantization in tensorrt/tools
then exported to onnx
then build engine from onnx
the error in title occured when i parsed onnx model to build engine in my jetson xaiver nx
(i tested this pipeline in gpu tensorrt 8.5, and no error occured)
this is the log context:
this error seems to be an internal error reported from tensorrt,
i searched this error in google, yet found no meaningful information,
i want to know which layer causes this error, then i can use some ops to substitute, but i cannot understand what does this error mean, this is where I need help
Environment
TensorRT Version: 8.0.1.6
NVIDIA GPU: jetson
NVIDIA Driver Version: jetpack 5.0
CUDA Version: 10.2
Operating System: Ubuntu 18.04.5 LTS
Python Version (if applicable): 3.6.9
PyTorch Version (if applicable): 1.11.0a0+17540c5
Baremetal or Container (if so, version):
Relevant Files
Steps To Reproduce