[05/30/2024-10:56:44] [TRT] [E] 4: kOPT values for profile 0 violate shape constraints: IShuffleLayer /Reshape: reshaping failed for tensor: /Concat_output_0 reshape would change volume 230976 to 76992

[hello-lx]

Description

Hello, I have a problem and maybe it happens even at the moment of converting the weights.pt after training YOLACK (https://github.com/dbolya/yolact) to the TensorRT model.

I tried to convert model to trt model yolack (not yolack++) on GPU, but it fails with the error below.

[05/30/2024-10:56:44] [TRT] [E] 4: kOPT values for profile 0 violate shape constraints: IShuffleLayer /Reshape: reshaping failed for tensor: /Concat_output_0 reshape would change volume 230976 to 76992
[05/30/2024-10:56:44] [TRT] [E] 4: [shapeCompiler.cpp::nvinfer1::builder::DynamicSlotBuilder::evaluateShapeChecks::1276] Error Code 4: Internal Error (kOPT values for profile 0 violate shape constraints: IShuffleLayer /Reshape: reshaping failed for tensor: /Concat_output_0 reshape would change volume 230976 to 76992)

Environment

TensorRT Version: 8.6.1

NVIDIA GPU: RTX 4050

NVIDIA Driver Version:

CUDA Version: 11.8

CUDNN Version:

Operating System: Windows 10

Python Version (if applicable): 3.9.19

PyTorch Version (if applicable): 2.2.2

[lix19937]

would change volume 230976 to 76992

Your network forward graph has dynamic ops, and onnx export process has error, make some dynamic as const value.

[hello-lx]

This is the code I used to convert the pytorch model to onnx.
`
import torch
import os
from yolact import Yolact
from utils.functions import SavePath
from data import set_cfg

device = 'cuda' if torch.cuda.is_available() else 'cpu'
trained_model = 'models/yolact_plus_resnet50_54_800000.pth'
model_path = SavePath.from_str(trained_model)
config = model_path.model_name + '_config'
set_cfg(config)
net = Yolact()
net.load_weights(trained_model, device=device)
net.eval()
net.to(device)

output_onnx = os.path.splitext(trained_model)[0] + '.onnx'

inputs = torch.randn(3, 3, 550, 550).to(device)
print('convert', output_onnx, 'begin')
input_names, output_names = ['image'], ["loc", "conf", "mask", "prior", "proto"]
torch.onnx.export(net, inputs, output_onnx, verbose=False, opset_version=12, input_names=input_names,
                  output_names=output_names, dynamic_axes={
        input_names[0]: {0: 'batch_size'},
        output_names[0]: {0: 'batch_size'},
        output_names[1]: {0: 'batch_size'},
        output_names[2]: {0: 'batch_size'},
        output_names[3]: {0: 'batch_size'},
        output_names[4]: {0: 'map_width', 1: 'map_height'},
    }, operator_export_type=torch.onnx.OperatorExportTypes.ONNX_ATEN_FALLBACK)
print('convert', output_onnx, 'to onnx finish!!!')

`

This is the code I used to convert the onnx model to tensorrt.
`
import tensorrt as trt
logger = trt.Logger(trt.Logger.ERROR)

def export1(onnx_name, workspace=4, dynamic=False, half=False):
    logger = trt.Logger(trt.Logger.INFO)
    builder = trt.Builder(logger)
    config = builder.create_builder_config()
    config.max_workspace_size = workspace * 1 << 30
    flag = (1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))
    network = builder.create_network(flag)
    parser = trt.OnnxParser(network, logger)
        
    onnx = onnx_name + '.onnx'
    if not parser.parse_from_file(str(onnx)):
        raise RuntimeError(f'failed to load ONNX file: {onnx}')
    
    inputs = [(network.get_input(i).name, network.get_input(i).shape) for i in range(network.num_inputs)]
    outputs = [(network.get_output(i).name, network.get_output(i).shape) for i in range(network.num_outputs)]
    print(inputs)
    print(outputs)
    
    if dynamic:
        # if im.shape[0] <= 1:
        #     logger.warning(f"{prefix} WARNING ⚠️ --dynamic model requires maximum --batch-size argument")

        profile = builder.create_optimization_profile()

        for item in inputs + outputs:
            profile.set_shape(item[0], item[1], item[2], item[3])

        config.add_optimization_profile(profile)

    if builder.platform_has_fast_fp16 and half:
        config.set_flag(trt.BuilderFlag.FP16)
        # build engine 文件的写入这里的f是前面定义的engine文件

    f = onnx_name + '.engine'
    with builder.build_engine(network, config) as engine, open(f, 'wb') as t:
        # 序列化model
        t.write(engine.serialize())

    return f, None

`

I did not find any abnormalities during the process of converting the model. Can you provide some suggestion? Thank you very much

[lix19937]

Pay attention the log of torch.onnx.export process warnings . And close the operator_export_type=torch.onnx.OperatorExportTypes.ONNX_ATEN_FALLBACK @hello-lx

[hello-lx]

convert models/yolact_plus_resnet50_54_800000_2.onnx begin
D:\XSpace\CPP\DCNv2_Plugin\yolact-onnx-deploy-main\convert-onnx\yolact.py:272: TracerWarning: torch.Tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
self.priors = torch.Tensor(prior_data).view(-1, 4).detach().to(device)
D:\XSpace\CPP\DCNv2_Plugin\yolact-onnx-deploy-main\convert-onnx\yolact.py:272: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
self.priors = torch.Tensor(prior_data).view(-1, 4).detach().to(device)
Traceback (most recent call last):
File "D:\XSpace\CPP\DCNv2_Plugin\yolact-onnx-deploy-main\convert-onnx\convert_onnx.py", line 40, in
torch.onnx.export(net, inputs, output_onnx, verbose=False, opset_version=12, input_names=['image'],
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\onnx\utils.py", line 516, in export
_export(
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\onnx\utils.py", line 1613, in _export
graph, params_dict, torch_out = _model_to_graph(
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\onnx\utils.py", line 1135, in _model_to_graph
graph, params, torch_out, module = _create_jit_graph(model, args)
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\onnx\utils.py", line 1011, in _create_jit_graph
graph, torch_out = _trace_and_get_graph_from_model(model, args)
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\onnx\utils.py", line 915, in _trace_and_get_graph_from_model
trace_graph, torch_out, inputs_states = torch.jit._get_trace_graph(
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\jit_trace.py", line 1296, in _get_trace_graph
outs = ONNXTracedModule(
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\jit_trace.py", line 138, in forward
graph, out = torch._C._create_graph_by_tracing(
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\jit_trace.py", line 129, in wrapper
outs.append(self.inner(*trace_inputs))
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\XStudio\Anaconda\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 1501, in _slow_forward
result = self.forward(*input, **kwargs)
File "D:\XSpace\CPP\DCNv2_Plugin\yolact-onnx-deploy-main\convert-onnx\yolact.py", line 721, in forward
loc = loc.view(s)
RuntimeError: shape '[57744, 4]' is invalid for input of size 692928

Process finished with exit code 1

[lix19937]

RuntimeError: shape '[57744, 4]' is invalid for input of size 692928

Process finished with exit code 1

error arise !