onnx -> cvimodel failed and i can't see how to fix

[luoluoter]

onnx -> milr + run_calibration

model_transform.py \
  --model_name f32_xiao \
  --model_def f32_xiao.onnx \
  --input_shapes "[[1,224,224,3]]" \
  --mean "0.0,0.0,0.0" \
  --scale "0.0039216,0.0039216,0.0039216" \
  --keep_aspect_ratio \
  --pixel_format "rgb" \
  --output_names "sequential_4/sequential_1/model1/conv_pw_13_relu/Relu6;sequential_4/sequential_1/model1/conv_pw_13_bn/FusedBatchNormV3;sequential_4/sequential_1/model1/conv_pw_13/Conv2D" \
  --test_input test.jpg \
  --test_result f32_xiao_result.npz \
  --mlir f32_xiao.mlir \
  --channel_format "nhwc" \
  --input_types F32
# OK

run_calibration.py \
  f32_xiao.mlir \
  --dataset project \
  --input_num 100 \
  -o f32_xiao_calib_table
# OK

milr -> cvimodel

model_deploy.py \
--mlir f32_xiao.mlir \
--quantize F16 \
--quant_input \
--processor cv181x \
--calibration_table f32_xiao_calib_table \
--test_input test.jpg \
--test_reference f32_xiao_top_outputs.npz \
--customization_format RGB_PACKED \
--fuse_preprocess \
--aligned_input \
--model f32_xiao_int8_sym_new.cvimodel

ERROR LOG

2025/03/31 14:26:54 - INFO : TPU-MLIR v1.16.beta.0-41-g5eba2a0e2-20250327
2025/03/31 14:26:54 - INFO : 
  load_config Preprocess args : 
        resize_dims           : [224, 224]
        keep_aspect_ratio     : True
        keep_ratio_mode       : letterbox
        pad_value             : 0
        pad_type              : center
        input_dims            : [224, 224]
        --------------------------
        mean                  : [0.0, 0.0, 0.0]
        scale                 : [0.0039216, 0.0039216, 0.0039216]
        --------------------------
        pixel_format          : rgb
        channel_format        : nhwc

2025/03/31 14:26:54 - INFO : Add preprocess, set the following params:
2025/03/31 14:26:54 - INFO : 
         _____________________________________________________ 
        | preprocess:                                           |
        |   (x - mean) * scale                                  |
        '-------------------------------------------------------'
  config Preprocess args : 
        resize_dims           : [224, 224]
        keep_aspect_ratio     : True
        keep_ratio_mode       : letterbox
        pad_value             : 0
        pad_type              : center
        --------------------------
        mean                  : [0.0, 0.0, 0.0]
        scale                 : [1.0, 1.0, 1.0]
        --------------------------
        pixel_format          : rgb
        channel_format        : nhwc

[Running]: tpuc-opt f32_xiao.mlir --processor-assign="chip=cv181x mode=F16 num_device=1 num_core=1 addr_mode=auto high_precision=False" --import-calibration-table="file=f32_xiao_calib_table asymmetric=False" --processor-top-optimize --fuse-preprocess="mode=F16 customization_format=RGB_PACKED align=True" --convert-top-to-tpu=" asymmetric=False doWinograd=False q_group_size=0 matmul_perchannel=False gelu_mode=normal" --canonicalize --weight-fold -o f32_xiao_cv181x_f16_tpu.mlir
Entering FusePreprocessPass.
Inserting PermuteOp.
Inserting SliceOp.
Inserting DWConvOp.
Inserting SwapChannelOp.
PreprocessFuse : tpu.Preprocess succeed!
[Success]: tpuc-opt f32_xiao.mlir --processor-assign="chip=cv181x mode=F16 num_device=1 num_core=1 addr_mode=auto high_precision=False" --import-calibration-table="file=f32_xiao_calib_table asymmetric=False" --processor-top-optimize --fuse-preprocess="mode=F16 customization_format=RGB_PACKED align=True" --convert-top-to-tpu=" asymmetric=False doWinograd=False q_group_size=0 matmul_perchannel=False gelu_mode=normal" --canonicalize --weight-fold -o f32_xiao_cv181x_f16_tpu.mlir
[CMD]: model_runner.py --input f32_xiao_in_ori.npz --model f32_xiao_cv181x_f16_tpu.mlir --output f32_xiao_cv181x_f16_tpu_outputs.npz  
[                                                  ] 0%Floating point exception (core dumped)

and i try int8 too. also failed.

# int8 test
model_deploy.py \
--mlir f32_xiao.mlir \
--quantize INT8 \
--quant_input \
--processor cv181x \
--calibration_table f32_xiao_calib_table \
--test_input test.jpg \
--test_reference f32_xiao_top_outputs.npz \
--customization_format RGB_PACKED \
--fuse_preprocess \
--aligned_input \
--model f32_xiao_int8_sym_new.cvimodel

###### ERROR
2025/03/31 14:26:08 - INFO : TPU-MLIR v1.16.beta.0-41-g5eba2a0e2-20250327
2025/03/31 14:26:08 - INFO : 
  load_config Preprocess args : 
        resize_dims           : [224, 224]
        keep_aspect_ratio     : True
        keep_ratio_mode       : letterbox
        pad_value             : 0
        pad_type              : center
        input_dims            : [224, 224]
        --------------------------
        mean                  : [0.0, 0.0, 0.0]
        scale                 : [0.0039216, 0.0039216, 0.0039216]
        --------------------------
        pixel_format          : rgb
        channel_format        : nhwc

2025/03/31 14:26:08 - INFO : Add preprocess, set the following params:
2025/03/31 14:26:08 - INFO : 
         _____________________________________________________ 
        | preprocess:                                           |
        |   (x - mean) * scale                                  |
        '-------------------------------------------------------'
  config Preprocess args : 
        resize_dims           : [224, 224]
        keep_aspect_ratio     : True
        keep_ratio_mode       : letterbox
        pad_value             : 0
        pad_type              : center
        --------------------------
        mean                  : [0.0, 0.0, 0.0]
        scale                 : [1.0, 1.0, 1.0]
        --------------------------
        pixel_format          : rgb
        channel_format        : nhwc

[Running]: tpuc-opt f32_xiao.mlir --processor-assign="chip=cv181x mode=INT8 num_device=1 num_core=1 addr_mode=auto high_precision=False" --import-calibration-table="file=f32_xiao_calib_table asymmetric=False" --processor-top-optimize --fuse-preprocess="mode=INT8 customization_format=RGB_PACKED align=True" --convert-top-to-tpu=" asymmetric=False doWinograd=False q_group_size=0 matmul_perchannel=False gelu_mode=normal" --canonicalize --weight-fold -o f32_xiao_cv181x_int8_sym_tpu.mlir
Entering FusePreprocessPass.
Inserting PermuteOp.
Inserting SliceOp.
Inserting ScalelutOp.
Inserting SwapChannelOp.
PreprocessFuse : tpu.Preprocess succeed!
[Success]: tpuc-opt f32_xiao.mlir --processor-assign="chip=cv181x mode=INT8 num_device=1 num_core=1 addr_mode=auto high_precision=False" --import-calibration-table="file=f32_xiao_calib_table asymmetric=False" --processor-top-optimize --fuse-preprocess="mode=INT8 customization_format=RGB_PACKED align=True" --convert-top-to-tpu=" asymmetric=False doWinograd=False q_group_size=0 matmul_perchannel=False gelu_mode=normal" --canonicalize --weight-fold -o f32_xiao_cv181x_int8_sym_tpu.mlir
[CMD]: model_runner.py --input f32_xiao_in_ori.npz --model f32_xiao_cv181x_int8_sym_tpu.mlir --output f32_xiao_cv181x_int8_sym_tpu_outputs.npz  
loc("serving_default_sequential_1_input:0_preprocess_scale_lut"): error: 'tpu.ScaleLut' op attribute 'scale' failed to satisfy constraint: 64-bit float array attribute
python3: /workspace/bindings/pymlir/pymodule.cpp:34: void py_module::load(std::string): Assertion `module_' failed.
Aborted (core dumped)

[luoluoter]

my models in the classify_tflite_onnx.zip.

it include 4 files:

• int8.tflite
• int8.onnx (← in8.tflite + tf2onnx)
• float32.tflite
• float32.onnx (← float32.tflite + tf2onnx)

---

I assume my operation with tf2onnx was incorrect, so the “tflite -> cvimodel” way would also be acceptable. Thank you to everyone for your help.

[jdsalmonson]

For what it is worth, I've encountered this error as well when trying to implement --fuse_preprocess to create my quantized cvimodel using model_deploy for the cv181x processor. I get the same ScaleLut error because the scale attribute is not being created correctly. This works fine if I omit the --fuse_preprocess and use --symmetric or --asymmetric input instead.