-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
VitisAI EP Context Model #20926
VitisAI EP Context Model #20926
Conversation
Tried addressing part of the issues with wrapped classes Fixed extra forward declaration
Fixed some piecemeal issues Fixed an incorrect header inclusion Got rid of NodeAttrHelper Fixed piecemeal issues
Commented out an explicit and direct header inclusion
Dealt with const-ness of class members Fixed mismatched types due to previous changes Added missing namespace prefixes Removed incorrect methods
@microsoft-github-policy-service agree |
onnxruntime/core/providers/vitisai/vitisai_execution_provider.cc
Outdated
Show resolved
Hide resolved
/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline |
Azure Pipelines successfully started running 9 pipeline(s). |
/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed |
Azure Pipelines successfully started running 9 pipeline(s). |
…. This one is tightly coupled with the closed-source backend of VitisAI EP. 2) Interim commit. Algorithm implementation selection for MD5 is TODO.
/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed |
Pull request contains merge conflicts. |
/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline |
Pull request contains merge conflicts. |
/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed |
/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline |
Azure Pipelines successfully started running 9 pipeline(s). |
1 similar comment
Azure Pipelines successfully started running 9 pipeline(s). |
/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed |
/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline |
Azure Pipelines successfully started running 9 pipeline(s). |
1 similar comment
Azure Pipelines successfully started running 9 pipeline(s). |
…vitisai-ep-context-model-2
/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed |
/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline |
Azure Pipelines successfully started running 9 pipeline(s). |
1 similar comment
Azure Pipelines successfully started running 9 pipeline(s). |
/azp run Linux OpenVINO CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline |
Azure Pipelines successfully started running 3 pipeline(s). |
Why so many commits
"Open" (?) questions
VitisAI EP made three variant implementations that have respective pros and conds (and of course we can combine them)
Serialize and cache the list of compute capabilites and the original ONNX model itself
In
ComputeCapability()
, serialize and cache the backend compilation cache and the related necessary cache info such as cache dir and cache keyIn
Compile()
, serialize and cache the backend compilation cache and the related necessary cache info such as cache dir and cache keyEP context model creation
Session option configuration
kOrtSessionOptionEpContextEnable
(aka "ep.context_enable") is enabled.IExecutionProvider::GetEpContextNodes()
method.CreateEpContextModel()
in the file "graph_partitioner.cc"get_ep_context_node()
,Node::Name()
is used to check whether a node is an EP context node. This limits that EP model creation can only happen inIExecutionProvider::Compile()
.IExecutionProvider::GetEpContextNodes()
and (2) dumping the EP context model by EP itself.QNNExecutionProvider
VitisAIExecutionProvider
IExecutionProvider::GetEpContextNodes()
at all.TensorrtExecutionProvider
IExecutionProvider::GetEpContextNodes()
OpenVINOExecutionProvider
(?)What to cache in EP context nodes
VitisAIExecutionProvider
IExecutionProvider::GetCapability()
.IExecutionProvider::GetCapability()
is only called once by ONNXRT.ComputeCapability
onnx::FunctionProto
QNNExecutionProvider
TensorrtExecutionProvider
MIGraphXExecutionProvider
OpenVINOExecutionProvider
Requirements