[Split] The split values of Split are changed when I use the post_process_transpose_perm option.

[On-JungWoan]

Issue Type

Others

onnx2tf version number

1.8.15

onnx version number

1.13.1

tensorflow version number

2.12.0

Download URL for ONNX

https://drive.google.com/file/d/1V0y-vElMeTIBf8Ft3Y-gN2KafIXCL2hM/view?usp=sharing

Parameter Replacement JSON

replace.json

{
    "format_version": 1,
    "operations": [
      
      {
        "op_name": "Reshape_191",
        "param_target": "outputs",
        "param_name": "value",
        "post_process_transpose_perm": [0,2,3,1]
      },    
      {
        "op_name": "Split_224",
        "param_target": "attributes",
        "param_name": "axis",
        "values": 3
      },

      {
        "op_name": "Reshape_238",
        "param_target": "outputs",
        "param_name": "value_l_",
        "post_process_transpose_perm": [0,2,3,1]
      },
      {
        "op_name": "Reshape_264",
        "param_target": "outputs",
        "param_name": "value_l_.3",
        "post_process_transpose_perm": [0,2,3,1]
      },
      {
        "op_name": "Reshape_290",
        "param_target": "outputs",
        "param_name": "value_l_.7",
        "post_process_transpose_perm": [0,2,3,1]
      },
      {
        "op_name": "Reshape_316",
        "param_target": "outputs",
        "param_name": "value_l_.11",
        "post_process_transpose_perm": [0,2,3,1]
      },






      {
        "op_name": "Reshape_397",
        "param_target": "outputs",
        "param_name": "value.3",
        "post_process_transpose_perm": [0,2,3,1]
      },    
      {
        "op_name": "Split_430",
        "param_target": "attributes",
        "param_name": "axis",
        "values": 3
      },

      {
        "op_name": "Reshape_444",
        "param_target": "outputs",
        "param_name": "value_l_.15",
        "post_process_transpose_perm": [0,2,3,1]
      },
      {
        "op_name": "Reshape_470",
        "param_target": "outputs",
        "param_name": "value_l_.19",
        "post_process_transpose_perm": [0,2,3,1]
      },
      {
        "op_name": "Reshape_496",
        "param_target": "outputs",
        "param_name": "value_l_.23",
        "post_process_transpose_perm": [0,2,3,1]
      },
      {
        "op_name": "Reshape_522",
        "param_target": "outputs",
        "param_name": "value_l_.27",
        "post_process_transpose_perm": [0,2,3,1]
      }



      ,
        {
            "op_name": "Reshape_603",
            "param_target": "outputs",
            "param_name": "value.7",
            "post_process_transpose_perm": [0,2,3,1]
        },    
        {
            "op_name": "Split_636",
            "param_target": "attributes",
            "param_name": "axis",
            "values": 3
        },

        {
            "op_name": "Reshape_650",
            "param_target": "outputs",
            "param_name": "value_l_.31",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_676",
            "param_target": "outputs",
            "param_name": "value_l_.35",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_702",
            "param_target": "outputs",
            "param_name": "value_l_.39",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_728",
            "param_target": "outputs",
            "param_name": "value_l_.43",
            "post_process_transpose_perm": [0,2,3,1]
        }



      ,
        {
            "op_name": "Reshape_809",
            "param_target": "outputs",
            "param_name": "value.11",
            "post_process_transpose_perm": [0,2,3,1]
        },    
        {
            "op_name": "Split_842",
            "param_target": "attributes",
            "param_name": "axis",
            "values": 3
        },


        {
            "op_name": "Reshape_856",
            "param_target": "outputs",
            "param_name": "value_l_.47",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_882",
            "param_target": "outputs",
            "param_name": "value_l_.51",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_908",
            "param_target": "outputs",
            "param_name": "value_l_.55",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_934",
            "param_target": "outputs",
            "param_name": "value_l_.59",
            "post_process_transpose_perm": [0,2,3,1]
        }




      ,
        {
            "op_name": "Reshape_1015",
            "param_target": "outputs",
            "param_name": "value.15",
            "post_process_transpose_perm": [0,2,3,1]
        },    
        {
            "op_name": "Split_1048",
            "param_target": "attributes",
            "param_name": "axis",
            "values": 3
        },


        {
            "op_name": "Reshape_1062",
            "param_target": "outputs",
            "param_name": "value_l_.63",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_1088",
            "param_target": "outputs",
            "param_name": "value_l_.67",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_1114",
            "param_target": "outputs",
            "param_name": "value_l_.71",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_1140",
            "param_target": "outputs",
            "param_name": "value_l_.75",
            "post_process_transpose_perm": [0,2,3,1]
        }




      ,
        {
            "op_name": "Reshape_1221",
            "param_target": "outputs",
            "param_name": "value.19",
            "post_process_transpose_perm": [0,2,3,1]
        },    
        {
            "op_name": "Split_1254",
            "param_target": "attributes",
            "param_name": "axis",
            "values": 3
        },


        {
            "op_name": "Reshape_1268",
            "param_target": "outputs",
            "param_name": "value_l_.79",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_1294",
            "param_target": "outputs",
            "param_name": "value_l_.83",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_1320",
            "param_target": "outputs",
            "param_name": "value_l_.87",
            "post_process_transpose_perm": [0,2,3,1]
        },
        {
            "op_name": "Reshape_1346",
            "param_target": "outputs",
            "param_name": "value_l_.91",
            "post_process_transpose_perm": [0,2,3,1]
        },











      
      {
        "op_name": "Mul_181",
        "param_target": "inputs",
        "param_name": "onnx::Gather_123",
        "pre_process_transpose_perm": [0,3,2,1]
      },
      {
        "op_name": "Add_359",
        "param_target": "inputs",
        "param_name": "src.1",
        "pre_process_transpose_perm": [0,2,1]
      },

      {
        "op_name": "Add_388",
        "param_target": "inputs",
        "param_name": "pos",
        "pre_process_transpose_perm": [0,2,1]
      },
      {
        "op_name": "Add_594",
        "param_target": "inputs",
        "param_name": "pos",
        "pre_process_transpose_perm": [0,2,1]
      },

      {
        "op_name": "Add_800",
        "param_target": "inputs",
        "param_name": "pos",
        "pre_process_transpose_perm": [0,2,1]
      },
      {
        "op_name": "Add_1006",
        "param_target": "inputs",
        "param_name": "pos",
        "pre_process_transpose_perm": [0,2,1]
      },
      {
        "op_name": "Add_1212",
        "param_target": "inputs",
        "param_name": "pos",
        "pre_process_transpose_perm": [0,2,1]
      }
    ]      
  }

Description

When using the post_process_transpose_perm option in an OP that is placed in front of a Slice OP, the split values of Slice are also changed.

For example:

When I'm changing the output format of Reshape_191 to (0,2,3,1), the split values of Split_224 are also changed to [8352, 522, 135, 2088]

Is this a bug?

[PINTO0309]

No. The internal process is too complex to explain to you.
This tricky model can lead to errors because of the very complex decision process for transposing NCHW to NHWC.

If I were you, I would try to verify the following,

      {
        "op_name": "Reshape_191",
        "param_target": "outputs",
        "param_name": "value",
        "post_process_transpose_perm": [0,2,3,1]
      },    
      {
        "op_name": "Split_224",
        "param_target": "inputs",
        "param_name": "value",
        "pre_process_transpose_perm": [x,x,x,x]
      },

or

      {
        "op_name": "Reshape_191",
        "param_target": "outputs",
        "param_name": "value",
        "post_process_transpose_perm": [0,2,3,1]
      },    
      {
        "op_name": "Split_224",
        "param_target": "attributes",
        "param_name": "axis",
        "values": 3
      },
      {
        "op_name": "Split_224",
        "param_target": "inputs",
        "param_name": "value",
        "pre_process_transpose_perm": [x,x,x,x]
      },

etc...

Btw, Splits of Split that are not equal in size will be converted to FlexSplitV by TFLite's specification, so it is strongly discouraged.

[On-JungWoan]

Hi, @PINTO0309 . Thank you for your hard work. I was able to solve the problem by using replace.json, as you had suggested. Thank you for your answer.

And, I use a multi-scale feature map. The scales of the multi-scale feature map are 8352, 2088, 522, 135, respectively. I have been using this feature map by flattening it into a 11097-dimensional tensor. Would it be possible to prevent the generation of FlexSplitv by putting each feature map into a list instead of flattening them?

[PINTO0309]

A creative change of mindset is needed.

Simply replace Split with Slice.

a = [:, 0:8352, ...]
b = [:, 8352:10440, ...]
c = [:, 10440:10962, ...]
d = [:, 10962:11097, ...]

The best advice I can give you is to always specify the start and end positions numerically when slicing. If the end index is unspecified, the maximum value of INT64, 9223372036854775807, is embedded in the model. This is because it could be an obstacle if you want to scale down the model to INT32, INT8, etc. in the future.

Deprecated operations

a = [:, :8352, ...]
b = [:, 8352:10440, ...]
c = [:, 10440:10962, ...]
d = [:, 10962:, ...]

The above is my meddling advice for your future model design. onnx2tf automatically tries to replace the above Split with Slice as much as possible. However, it will only work when the conditions are met to replace it with Slice, so it is best to be aware of the ramifications of deploying to a device at the time of model design, if at all possible.

Tools cannot understand the intentions of the person who designed the model, so it is best not to rely on tools for everything.

• Enhanced FlexSplitV generation suppression #262

• default
  

• onnx2tf
  

[On-JungWoan]

As you advised, I made some changes to the script. I can't see any errors related to the split now. Thank you very much.

Before

value_list = value.split([int(H_ * W_) for (H_, W_) in value_spatial_shapes], dim=1)

After

    start_idx = []
    for idx, (H_, W_) in enumerate(value_spatial_shapes):
        tmp = int(H_ * W_)
        if len(start_idx) > 0:
            tmp += start_idx[idx-1]
        start_idx.append(tmp)

    value_list = [
        value[:,0:start_idx[0],:,:],
        value[:,start_idx[0]:start_idx[1],:,:],
        value[:,start_idx[1]:start_idx[2],:,:],
        value[:,start_idx[2]:start_idx[3],:,:],
    ]