Stand-alone pad operation fails with: Assertion failed: inputs.at(1).is_weights() #439

copaah · 2020-03-17T17:14:32Z

Description

My current workflow is pytorch -> onnx -> tensorrt and I encounter an issue with the nn.ConstantPad2D operation that results in the following error:

While parsing node number 23 [Pad -> "24"]:
--- Begin node ---
input: "input"
input: "22"
input: "23"
output: "24"
op_type: "Pad"
attribute {
  name: "mode"
  s: "constant"
  type: STRING
}

--- End node ---
ERROR: /mypath/onnx-tensorrt/builtin_op_importers.cpp:2106 In function importPad:
[8] Assertion failed: inputs.at(1).is_weights()
[03/12/2020-09:06:29] [E] Failed to parse onnx file
[03/12/2020-09:06:29] [E] Parsing model failed
[03/12/2020-09:06:29] [E] Engine creation failed
[03/12/2020-09:06:29] [E] Engine set up failed

Environment

OS: Ubuntu 18.04
torch: 1.4.0
onnx: 1.6.0
tensorrt: 7.0.0
cuda: 10.0
python: 2.7

Steps To Reproduce

# github_repro_example.py
# -----------
import onnx
import argparse
import torch
import torch.nn as nn

class MinimalModel(nn.Module):
    def __init__(self):
        super(MinimalModel, self).__init__()
        self.constant_zero_pad = nn.ConstantPad2d((1, 0, 0, 0), 0)

    def forward(self, input_tensor):
        return self.constant_zero_pad(input_tensor)

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description='PSMNet')
    parser.add_argument('output_onnx')
    args = parser.parse_args()

    minimal_model = MinimalModel()
    minimal_model = nn.DataParallel(minimal_model)
    minimal_model.cuda()

    # Random deep feature
    input_tensor = torch.rand((1, 32, 128, 128))
    # Check model can do a forward pass
    minimal_model(input_tensor)
    # Export to onnx
    torch.onnx.export(
        minimal_model.module,
        (input_tensor),
        args.output_onnx,
        export_params=True, verbose=True, training=False, opset_version=11
    )

    original_model = onnx.load(args.output_onnx)
    onnx.checker.check_model(original_model)

Run with:

python2 github_repro_example.py ./test.onnx

Run it through tensorrt

trtexec --explicitBatch --onnx ./test.onnx --verbose

Which will result in the above error.

Related issues

onnx/onnx-tensorrt#378
onnx/onnx-tensorrt#411

The text was updated successfully, but these errors were encountered:

rmccorm4 · 2020-03-26T00:15:02Z

Hi @copaah ,

As for this error:

ERROR: /mypath/onnx-tensorrt/builtin_op_importers.cpp:2106 In function importPad:
[8] Assertion failed: inputs.at(1).is_weights()
[03/12/2020-09:06:29] [E] Failed to parse onnx file
[03/12/2020-09:06:29] [E] Parsing model failed
[03/12/2020-09:06:29] [E] Engine creation failed
[03/12/2020-09:06:29] [E] Engine set up failed

PyTorch generated kind of a funky ONNX graph for this simple model. I don't know if this is an issue to be fixed on their part, or on the ONNX parser's part, @kevinch-nv might be able to answer that.

As a workaround to your issue, you can try using onnx-simplifier on the onnx model and then parsing that, which worked for me.

# Start TRT 7 container
nvidia-docker run -it -v $PWD:/mnt nvcr.io/nvidia/tensorrt:20.02-py3

# Install TensorRT OSS components
wget https://raw.githubusercontent.com/rmccorm4/tensorrt-utils/master/OSS/build_OSS.sh
source build_OSS.sh

# Install dependencies
python3 -m pip install torch==1.4 onnx==1.6 onnx-simplifier
# Create repro model
python3 repro.py model.onnx
# Run onnx-simplifier on repro model
python3 -m onnxsim model.onnx model.simple.onnx

# Parse simplified model
trtexec --explicitBatch --onnx=model.simple.onnx
...
&&&& PASSED TensorRT.trtexec # trtexec --explicitBatch --onnx=model.simple.onnx

rmccorm4 · 2020-03-26T00:16:44Z

Above original and simplified onnx models here: onnx-models.zip

kevinch-nv · 2020-03-26T00:42:41Z

For padding, the ONNX-TRT parser expects the padded values to be initializers (i.e constants) in the ONNX graph. I checked @rmccorm4's zip package and the nodes that were contributing to the pad dimensions were constant-folded into an initializer by the onnx-simplifier.

rmccorm4 · 2020-03-26T01:18:47Z

@kevinch-nv thanks for the insight. I added torch.onnx.export(..., do_constant_folding=True, keep_initializers_as_inputs=True), and I get a slightly different model. but still fails with the same error. Any idea why?

Is there any parameter to torch.onnx.export that will correctly create the padded values as initializers?

torch.onnx.export(..., do_constant_folding=True) model: model_constant_fold.zip

torch.onnx.export(..., do_constant_folding=True, keep_initializers_as_inputs=True) model: model_keep_inits.zip

kevinch-nv · 2020-03-26T06:21:03Z

Perhaps the constant-folding functionality of the current torch2onnx export doesn't support this particular structure yet. Pre-opset 11 the pads input of the padding node was an attribute instead of an input, can you try exporting to opset-10 and inspecting the resulting graph?

rmccorm4 · 2020-03-27T00:49:40Z

Good call @kevinch-nv !

With opset 10, whether you use default args, or set do_constant_folding=True, we get a simple graph containing only the Pad op which is parseable by TensorRT.

I raised an issue in Pytorch to see what they think: pytorch/pytorch#35516

kevinch-nv · 2020-08-26T06:48:09Z

@copaah are you able to import your model with TensorRT now?

hhhharold · 2020-09-29T08:15:49Z

same issue, anyone has solved this?

vilmara · 2021-01-20T16:27:00Z

Hi @copaah ,

As for this error:

ERROR: /mypath/onnx-tensorrt/builtin_op_importers.cpp:2106 In function importPad:
[8] Assertion failed: inputs.at(1).is_weights()
[03/12/2020-09:06:29] [E] Failed to parse onnx file
[03/12/2020-09:06:29] [E] Parsing model failed
[03/12/2020-09:06:29] [E] Engine creation failed
[03/12/2020-09:06:29] [E] Engine set up failed

PyTorch generated kind of a funky ONNX graph for this simple model. I don't know if this is an issue to be fixed on their part, or on the ONNX parser's part, @kevinch-nv might be able to answer that.

As a workaround to your issue, you can try using onnx-simplifier on the onnx model and then parsing that, which worked for me.

# Start TRT 7 container
nvidia-docker run -it -v $PWD:/mnt nvcr.io/nvidia/tensorrt:20.02-py3

# Install TensorRT OSS components
wget https://raw.githubusercontent.com/rmccorm4/tensorrt-utils/master/OSS/build_OSS.sh
source build_OSS.sh

# Install dependencies
python3 -m pip install torch==1.4 onnx==1.6 onnx-simplifier
# Create repro model
python3 repro.py model.onnx
# Run onnx-simplifier on repro model
python3 -m onnxsim model.onnx model.simple.onnx

# Parse simplified model
trtexec --explicitBatch --onnx=model.simple.onnx
...
&&&& PASSED TensorRT.trtexec # trtexec --explicitBatch --onnx=model.simple.onnx

Hi @rmccorm4, after applying your instructions mentioned on issues #386 and #439 I got new error, some idea how to fix it?:

[01/20/2021-16:17:48] [W] [TRT] /workspace/TensorRT/parsers/onnx/onnx2trt_utils.cpp:227: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[01/20/2021-16:17:48] [W] [TRT] /workspace/TensorRT/parsers/onnx/onnx2trt_utils.cpp:253: One or more weights outside the range of INT32 was clamped
[01/20/2021-16:17:48] [W] [TRT] /workspace/TensorRT/parsers/onnx/onnx2trt_utils.cpp:253: One or more weights outside the range of INT32 was clamped
[01/20/2021-16:17:48] [W] [TRT] /workspace/TensorRT/parsers/onnx/onnx2trt_utils.cpp:253: One or more weights outside the range of INT32 was clamped
[01/20/2021-16:17:48] [I] [TRT] No importer registered for op: Round. Attempting to import as plugin.
[01/20/2021-16:17:48] [I] [TRT] Searching for plugin: Round, plugin_version: 1, plugin_namespace:
[01/20/2021-16:17:48] [E] [TRT] INVALID_ARGUMENT: getPluginCreator could not find plugin Round version 1
[01/20/2021-16:17:48] [E] [TRT] /workspace/TensorRT/parsers/onnx/ModelImporter.cpp:705: While parsing node number 7 [Loop -> "unused_loop_output___143"]:
[01/20/2021-16:17:48] [E] [TRT] /workspace/TensorRT/parsers/onnx/ModelImporter.cpp:706: --- Begin node ---
[01/20/2021-16:17:48] [E] [TRT] /workspace/TensorRT/parsers/onnx/ModelImporter.cpp:707: input: "trip_count__68"
....
[01/20/2021-16:17:48] [E] [TRT] /workspace/TensorRT/parsers/onnx/ModelImporter.cpp:708: --- End node ---
[01/20/2021-16:17:48] [E] [TRT] /workspace/TensorRT/parsers/onnx/ModelImporter.cpp:711: ERROR: /workspace/TensorRT/parsers/onnx/builtin_op_importers.cpp:3807 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[01/20/2021-16:17:48] [E] Failed to parse onnx file
[01/20/2021-16:17:48] [E] Parsing model failed
[01/20/2021-16:17:48] [E] Engine creation failed
[01/20/2021-16:17:48] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # trtexec --onnx=/faster_rcnn_inceptionv2_coco_updated_model_opset11_onnx1.6_nocostfold.onnx --explicitBatch

opeide · 2021-03-09T14:44:30Z

Half a year later, issue still persists.

dedoogong · 2021-04-11T01:34:28Z

same here

wkl2013DeepVision · 2021-05-13T09:22:01Z

Half a year later, issue still persists.

same here, wast so many time !, can any one who solved it?

ttyio · 2021-05-27T07:24:25Z

@opeide @dedoogong @wkl2013DeepVision

Could you try

      polygraphy surgeon sanitize --fold-constants

We have document here https://github.com/onnx/onnx-tensorrt/blob/master/docs/faq.md#common-assertion-errors thanks

romain87400 · 2021-06-09T09:09:06Z

@ttyio
I'm try
polygraphy surgeon sanitize model.onnx --fold-constants --output model_folded.onnx
I'm come back on the first error [8] Assertion failed: mode == "constant" && value == 0.
When i use onnxsim to corrected [8] Assertion failed: inputs.at(1).is_weights() i come back on this error [8] Assertion failed: mode == "constant" && value == 0.
Do you have a solution on this problem ?
I work on Nvidia Xavier NX, Jetpack 4.4 with tensorRT 7.1.3.

ttyio · 2021-06-10T01:06:35Z

Sorry @romain87400 , this command can help to fold constants to solve some Assertion failed: inputs.at(1).is_weights() failure, but we still only support constant 0 padding.

romain87400 · 2021-06-10T07:48:05Z

@ttyio Thanks for your answer.
So, currently, we cannot converted model with constant 0 padding.
But in futur, you will think correct this use case ?
Or maybe, you have an idear to get around this problem ?
I'm readed that we must converted models to onnx with opset11, but when i do that i fall on [8] Assertion failed: inputs.at(1).is_weights()

Sorry I'm a beginner on the subject.
Have a nice day.

ttyio · 2021-06-10T07:51:35Z

Hello @romain87400 , I am not sure if you can adjust your model, finding this error node, change to pad with 0s, and fine-tune your model.

the inputs.at(1).is_weights() assertion failure usually can be fixed by polygraphy surgeon sanitize --fold-constants unless it is really an activation in your model.

phamdat09 · 2021-08-19T16:14:02Z

@ttyio Any update for fixing it ?? Thanks

ttyio · 2021-08-20T02:51:14Z

@phamdat09 non constant padding will be supported in next release (in around 2 months), thanks

phamdat09 · 2021-08-20T08:49:43Z

@ttyio Thanks for your info. But in my case, I think it is constant padding.
I got an error like it
[ERROR] [TRT] /home/bigbigboy/Documents/Test/TensortRT_Dynamic/TensorRT/parsers/onnx/ModelImporter.cpp:727: input: "496"
input: "554"
input: "556"
output: "557"
name: "Pad_151"
op_type: "Pad"
attribute {
name: "mode"
s: "constant"
type: STRING
}

[ERROR] [TRT] /home/bigbigboy/Documents/Test/TensortRT_Dynamic/TensorRT/parsers/onnx/ModelImporter.cpp:728: --- End node ---
[ERROR] [TRT] /home/bigbigboy/Documents/Test/TensortRT_Dynamic/TensorRT/parsers/onnx/ModelImporter.cpp:731: ERROR: /home/bigbigboy/Documents/Test/TensortRT_Dynamic/TensorRT/parsers/onnx/builtin_op_importers.cpp:2985 In function importPad:
[8] Assertion failed: inputs.at(1).is_weights() && "The input pads is required to be an initializer."`

HaohaoNJU · 2021-10-13T09:30:54Z

Perhaps the constant-folding functionality of the current torch2onnx export doesn't support this particular structure yet. Pre-opset 11 the pads input of the padding node was an attribute instead of an input, can you try exporting to opset-10 and inspecting the resulting graph?

Works for me , THX ~

kimdwkimdw · 2021-11-23T07:28:50Z

from 8.0
https://github.com/onnx/onnx-tensorrt/blob/release/8.0/builtin_op_importers.cpp

It will be fixed in 8.2, maybe?

https://github.com/onnx/onnx-tensorrt/blob/master/builtin_op_importers.cpp#L3100-L3120

nvpohanh · 2022-06-15T11:20:04Z

@copaah Could you try TRT 8.2/8.4 and see if the issue still exists? If it does, we will debug it. Thanks

nvpohanh · 2022-07-01T05:41:47Z

Closing for now due to >14 days without activity. Please feel free to reopen if the issue still exists. Thanks

donrax · 2022-07-20T08:55:12Z

This is perhaps related to #847

rmccorm4 added Export: torch.onnx https://pytorch.org/docs/stable/onnx.html ONNX Release: 7.x labels Mar 26, 2020

This was referenced Mar 27, 2020

torch.onnx.export Pad ops with ONNX opset 11 pytorch/pytorch#35516

Closed

tensort7 load onnx resize ops error #386

Closed

rmccorm4 mentioned this issue Jun 5, 2020

Stand-alone pad operation fails with: Assertion failed: inputs.at(1).is_weights() onnx/onnx-tensorrt#411

Closed

ttyio added the triaged Issue has been triaged by maintainers label Apr 26, 2021

nvpohanh closed this as completed Jul 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stand-alone pad operation fails with: Assertion failed: inputs.at(1).is_weights() #439

Stand-alone pad operation fails with: Assertion failed: inputs.at(1).is_weights() #439

copaah commented Mar 17, 2020 •

edited by rmccorm4

Loading

rmccorm4 commented Mar 26, 2020

rmccorm4 commented Mar 26, 2020

kevinch-nv commented Mar 26, 2020

rmccorm4 commented Mar 26, 2020 •

edited

Loading

kevinch-nv commented Mar 26, 2020

rmccorm4 commented Mar 27, 2020 •

edited

Loading

kevinch-nv commented Aug 26, 2020

hhhharold commented Sep 29, 2020

vilmara commented Jan 20, 2021

opeide commented Mar 9, 2021

dedoogong commented Apr 11, 2021

wkl2013DeepVision commented May 13, 2021

ttyio commented May 27, 2021

romain87400 commented Jun 9, 2021

ttyio commented Jun 10, 2021

romain87400 commented Jun 10, 2021

ttyio commented Jun 10, 2021

phamdat09 commented Aug 19, 2021

ttyio commented Aug 20, 2021

phamdat09 commented Aug 20, 2021 •

edited

Loading

HaohaoNJU commented Oct 13, 2021

kimdwkimdw commented Nov 23, 2021

nvpohanh commented Jun 15, 2022

nvpohanh commented Jul 1, 2022

donrax commented Jul 20, 2022

Stand-alone pad operation fails with: Assertion failed: inputs.at(1).is_weights() #439

Stand-alone pad operation fails with: Assertion failed: inputs.at(1).is_weights() #439

Comments

copaah commented Mar 17, 2020 • edited by rmccorm4 Loading

Description

Environment

Steps To Reproduce

Related issues

rmccorm4 commented Mar 26, 2020

rmccorm4 commented Mar 26, 2020

kevinch-nv commented Mar 26, 2020

rmccorm4 commented Mar 26, 2020 • edited Loading

kevinch-nv commented Mar 26, 2020

rmccorm4 commented Mar 27, 2020 • edited Loading

kevinch-nv commented Aug 26, 2020

hhhharold commented Sep 29, 2020

vilmara commented Jan 20, 2021

opeide commented Mar 9, 2021

dedoogong commented Apr 11, 2021

wkl2013DeepVision commented May 13, 2021

ttyio commented May 27, 2021

romain87400 commented Jun 9, 2021

ttyio commented Jun 10, 2021

romain87400 commented Jun 10, 2021

ttyio commented Jun 10, 2021

phamdat09 commented Aug 19, 2021

ttyio commented Aug 20, 2021

phamdat09 commented Aug 20, 2021 • edited Loading

HaohaoNJU commented Oct 13, 2021

kimdwkimdw commented Nov 23, 2021

nvpohanh commented Jun 15, 2022

nvpohanh commented Jul 1, 2022

donrax commented Jul 20, 2022

copaah commented Mar 17, 2020 •

edited by rmccorm4

Loading

rmccorm4 commented Mar 26, 2020 •

edited

Loading

rmccorm4 commented Mar 27, 2020 •

edited

Loading

phamdat09 commented Aug 20, 2021 •

edited

Loading