testMNNFromOnnx.py 验证Qwen-1_8B-Chat ONNX 出错

[sunnyzhaohui]

平台(如果交叉编译请再附上交叉编译目标平台):

Linux

Github版本:

参考文档 https://mnn-docs.readthedocs.io/en/latest/tools/convert.html#id4
验证 testMNNFromOnnx.py

编译方式:

1）魔搭下载Qwen-1_8B-Chat 模型文件
2）使用如下脚本生成 onnx 和 mnn
python3 llm_export.py --embed_bin
--embed_bf16
--onnx_path onnx
--type Qwen-1_8B-Chat
--path ./Qwen-1_8B-Chat
--export_embed
--export_token
--mnn_path mnn
--export

3）使用命令检查 onnx 文件
python ./tools/script/testMNNFromOnnx.py ./transformers/llm/export/onnx/llm.onnx

4）输出结果日志：

(wangenv) cmdc2023@fxzs-ai-gpu:~/zhwang_workSpace/MNN/MNN$ python ./tools/script/testMNNFromOnnx.py ./transformers/llm/export/onnx/llm.onnx
Dir exist
onnx/test.onnx
tensor(float)
tensor(bool)
tensor(int64)
tensor(float)
['logits', 'presents']
inputs:
input_ids
onnx/
attention_mask
onnx/
position_ids
onnx/
past_key_values
onnx/
2024-06-26 09:06:12.759355532 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running Concat node. Name:'/h.0/attn/Concat_9' Status Message: /onnxruntime_src/onnxruntime/core/framework/op_kernel.cc:83 virtual OrtValue* onnxruntime::OpKernelContext::OutputMLValue(int, const onnxruntime::TensorShape&) status.IsOK() was false. Shape mismatch attempting to re-use buffer. {1,1,16,128} != {1,2,16,128}. Validate usage of dim_value (values should be > 0) and dim_param (all values with the same string should equate to the same size) in shapes in the model.

Traceback (most recent call last):
File "./tools/script/testMNNFromOnnx.py", line 287, in
t.Test()
File "./tools/script/testMNNFromOnnx.py", line 263, in Test
self.__run_onnx()
File "./tools/script/testMNNFromOnnx.py", line 172, in __run_onnx
outputs = ort_session.run(None, inputs)
File "/home/cmdc2023/miniconda3/envs/wangenv/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Concat node. Name:'/h.0/attn/Concat_9' Status Message: /onnxruntime_src/onnxruntime/core/framework/op_kernel.cc:83 virtual OrtValue* onnxruntime::OpKernelContext::OutputMLValue(int, const onnxruntime::TensorShape&) status.IsOK() was false. Shape mismatch attempting to re-use buffer. {1,1,16,128} != {1,2,16,128}. Validate usage of dim_value (values should be > 0) and dim_param (all values with the same string should equate to the same size) in shapes in the model.

[jxt1234]

这个是默认输入不支持，可以自行修改校验脚本。也可以跳过这一步直接转mnn 使用，这个模型一般没有问题

[github-actions]

Marking as stale. No activity in 60 days.