You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
benchmark_app -m ../Qwen1.5-1.8B-Chat-OpenVINO-int4/openvino_model.xml -d CPU -shape "input_ids[1,786],attention_mask[1,786],position_ids[1,786],beam_idx[1]"
Step-by-step reproduction
benchmark_app -m ../Qwen1.5-1.8B-Chat-OpenVINO-int4/openvino_model.xml -d CPU -shape "input_ids[1,786],attention_mask[1,786],position_ids[1,786],beam_idx[1]"
Relevant log output
(openvino24.1) yongshuai_wang@ubuntu-DeepLearning-4801031:/juicefs-algorithm/workspace/vision/yongshuai_wang/models/hf-models$ benchmark_app -m ../Qwen1.5-1.8B-Chat-OpenVINO-int4/openvino_model.xml -d CPU -shape "input_ids[1,786],attention_mask[1,786],position_ids[1,786],beam_idx[1]"
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.1.0-33-20d7ce923d1
[ INFO ]
[ INFO ] Device info:
[ INFO ] CPU
[ INFO ] Build ................................. 2024.1.0-33-20d7ce923d1
[ INFO ]
[ INFO ]
[Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified incommand line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files
[ INFO ] Loading model files
[ INFO ] Read model took 131.17 ms
[ INFO ] Original model I/O parameters:
[ INFO ] Model inputs:
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [?,?]
[ INFO ] attention_mask (node: attention_mask) : i64 / [...] / [?,?]
[ INFO ] position_ids (node: position_ids) : i64 / [...] / [?,?]
[ INFO ] beam_idx (node: beam_idx) : i32 / [...] / [?]
[ INFO ] Model outputs:
[ INFO ] logits (node: aten::to/Convert) : f32 / [...] / [?,?,151936]
[Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1
[ INFO ] Reshaping model: 'input_ids': [1,786], 'attention_mask': [1,786], 'position_ids': [1,786], 'beam_idx': [1]
[ INFO ] Reshape model took 318.09 ms
[Step 6/11] Configuring input of the model
[ INFO ] Model inputs:
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,786]
[ INFO ] attention_mask (node: attention_mask) : i64 / [...] / [1,786]
[ INFO ] position_ids (node: position_ids) : i64 / [...] / [1,786]
[ INFO ] beam_idx (node: beam_idx) : i32 / [...] / [1]
[ INFO ] Model outputs:
[ INFO ] logits (node: aten::to/Convert) : f32 / [...] / [1,786,151936]
[Step 7/11] Loading the model to the device
[ ERROR ] Exception from src/inference/src/cpp/core.cpp:106:
Exception from src/inference/src/dev/plugin.cpp:54:
Exception from src/plugins/intel_cpu/src/cpu_memory.cpp:475:
Can not create StaticMemory object. The memory desc is undefined
Traceback (most recent call last):
File "/data/yongshuai_wang/miniconda3/envs/openvino24.1/lib/python3.10/site-packages/openvino/tools/benchmark/main.py", line 408, in main
compiled_model = benchmark.core.compile_model(model, benchmark.device, device_config)
File "/data/yongshuai_wang/miniconda3/envs/openvino24.1/lib/python3.10/site-packages/openvino/runtime/ie_api.py", line 521, in compile_model
super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src/inference/src/cpp/core.cpp:106:
Exception from src/inference/src/dev/plugin.cpp:54:
Exception from src/plugins/intel_cpu/src/cpu_memory.cpp:475:
Can not create StaticMemory object. The memory desc is undefined
Issue submission checklist
I'm reporting an issue. It's not a question.
I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
There is reproducer code and related data files such as images, videos, models, etc.
The text was updated successfully, but these errors were encountered:
HPUedCSLearner
changed the title
[Bug]: [benmark_app] : Can not create StaticMemory object. The memory desc is undefined
[Bug]: [benchmark_app] : Can not create StaticMemory object. The memory desc is undefined
Jun 5, 2024
This problem has troubled me for a long time. I have tried many versions of OpenVINO, including building from source code, installing with pip, and installing with apt, and all of them report the same error.
I would like to know how to use OpenVINO's benchmark_app tool to perform performance tests on current mainstream large models such as qwen, chatglm, etc.
Is it that benchmark_app testing of large models is not yet supported?
Please let me know, thank you very much!
Hi @HPUedCSLearner!
We recommend to use dedicated llm_bench for these purposes. It is the most convinient way to benchmark LLM workloads with OpenVINO runtime.
Benchmark_app is mostly dedicated to profile "traditional" models and might lack some LLM related functionality.
OpenVINO Version
openvino24.1
Operating System
Ubuntu 20.04 (LTS)
Device used for inference
CPU
Framework
None
Model used
LLM : qwen
Issue description
benchmark_app -m ../Qwen1.5-1.8B-Chat-OpenVINO-int4/openvino_model.xml -d CPU -shape "input_ids[1,786],attention_mask[1,786],position_ids[1,786],beam_idx[1]"
Step-by-step reproduction
benchmark_app -m ../Qwen1.5-1.8B-Chat-OpenVINO-int4/openvino_model.xml -d CPU -shape "input_ids[1,786],attention_mask[1,786],position_ids[1,786],beam_idx[1]"
Relevant log output
Issue submission checklist
The text was updated successfully, but these errors were encountered: