TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!

[hchings]

Ask a Question

Question

I'm trying to convert a Huggingface's CANINE model to onnx.

example = {'input_ids': tensor([[57344,    48,    48,    48,    48,    48,    48, 57345],
                                [57344,    48,    48,    48,    48,    48,    48, 57345]]),
                    'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0],
                                              [0, 0, 0, 0, 0, 0, 0, 0]]),
                    'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1],
                                              [1, 1, 1, 1, 1, 1, 1, 1]])}}

torch.onnx.export(model,                   # model being run
                  example,                 # model input (or a tuple for multiple inputs)
                  "./onnx/canine.onnx",    # where to save the model (can be a file or file-like object)
                  opset_version=13,        # the ONNX version to export the model to
                  do_constant_folding=True,
                  input_names = ['input_ids', 'attention_mask', 'token_type_ids'],   # the model's input names
                  output_names = ['last_hidden_state'], # the model's output names
                  dynamic_axes={'input_ids': {0: 'batch', 1: 'sequence'},      # variable length axes
                                'attention_mask': {0: 'batch', 1: 'sequence'},
                                'token_type_ids': {0: 'batch', 1: 'sequence'},    
                                'last_hidden_state' : {0 : 'batch_size'}})

After the model is exported, when I try to do inference with different sequence length (e.g., 9), I got the error:

[E:onnxruntime:, sequential_executor.cc:369 Execute] Non-zero status code returned while running Concat node. Name:'Concat_1714' Status Message: concat.cc:159 PrepareForCompute Non concat axis dimensions must match: Axis 1 has mismatched dimensions of 1536 and 9

I suspect it's due to the few TracerWarnings during export [complete TracerWarnings and Exported graph logs] :

src/transformers/models/canine/modeling_canine.py:605: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!

The problem code snippet in model that caused those warnings:

chunk_end = min(from_seq_length, chunk_start + self.attend_from_chunk_width)

My questions:

1. Which part I did wrong that caused ONNX dynamic axis not working? Are TracerWarnings the root causes?
2. How can I fix the above snippets to resolve TracerWarnings?
3. Is there a working example of CANINE onnx export you could share? Thanks

Further information

• Relevant Area: onnx export

• Versions:

  • python 3.10.8
  • torch 1.12.1
  • onnx 1.11.0

• Is this issue related to a specific model?
  Model name: CANINE
  Model opset: 13