[AUTO] Check if actual device cached blob exists before model compilation#29636
Conversation
github-actions
bot
added
category: inference
…ing the cache ability from AUTO plugin. 2. update the test cases of caching model for AUTO when cache is enabled by user.
…ed_blob_exists_before_model_compilation_in_AUTO
…ed_blob_exists_before_model_compilation_in_AUTO
src/plugins/auto/src/plugin.cpp
Outdated
| } | ||
|
|
||
| if (enable_startup_cpu && num_blob_files == devices_to_be_merged.size() - 1) { | ||
| LOG_DEBUG_TAG("Will disable CPU for acclerator when all blob files found"); |
There was a problem hiding this comment.
maybe more concrete debug message for why you are disabling CPU
There was a problem hiding this comment.
btw, if GPU.0 does not have the blob, GPU.1 does have, it will still have cpu start up enabled right?
There was a problem hiding this comment.
@wangleis , disable CPU on both startup fallback and runtime fallback if any cache blob found. Is this expected behavior? or we need to check if cache blobs exist for all of device in candidate list?
There was a problem hiding this comment.
When cache blob exist for candidate device, AUTO will pass model path to target device as pass through mode. Then AUTO need to disable startup fallback and runtime fallback.
There was a problem hiding this comment.
will disable CPU if any blob file found for a device in candidate list.
…ed_blob_exists_before_model_compilation_in_AUTO
4 participants
Details:
Add an new property for AUTO to indicate whether the core is querying the cache ability from virtual device.Tickets: