-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Make OrtDevice generic to support plugin EP usage. #24985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Remove unnecessary device id from OrtMemoryInfo. The OrtDevice in OrtMemoryInfo provides this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR makes OrtDevice generic to support plugin EP usage by removing vendor‐specific hardcoded memory types and introducing explicit vendor IDs. Key changes include updating device construction for various execution providers and allocators, revising memory type comparisons and hash functions, and exposing a new CreateMemoryInfo_V2 API.
Reviewed Changes
Copilot reviewed 60 out of 60 changed files in this pull request and generated no comments.
Show a summary per file
File | Description |
---|---|
onnxruntime/core/providers/dml/DmlExecutionProvider/src/ExecutionProvider.cpp | Updates OrtDevice construction for DML EP with an explicit vendor ID. |
onnxruntime/core/providers/dml/DmlExecutionProvider/src/DmlExternalBufferAllocator.h | Changes the OrtDevice instantiation to use GPU type with vendor MICROSOFT. |
onnxruntime/core/providers/cuda/* | Replaces memory type CUDA_PINNED with HOST_ACCESSIBLE and updates device id usage. |
onnxruntime/core/providers/cann/* | Updates allocator and data transfer code to use HOST_ACCESSIBLE for cann. |
onnxruntime/core/framework/* & include/* | Refactors OrtMemoryInfo and OrtDevice usages to remove the id field and incorporate vendor IDs. |
Comments suppressed due to low confidence (2)
onnxruntime/core/providers/dml/DmlExternalBufferAllocator.h:27
- The device type is changed from DML (as used in other parts of the DML provider) to GPU here; please verify that this conversion is intentional for correct device identification.
OrtDevice(OrtDevice::GPU, OrtDevice::MemType::DEFAULT, OrtDevice::VendorIds::MICROSOFT, 0)
onnxruntime/core/framework/allocator.cc:184
- For DML memory info, the device type is set to GPU with vendor MICROSOFT, which differs from the ExecutionProvider construction that uses OrtDevice::DML; please ensure this inconsistency is intentional.
OrtDevice(OrtDevice::GPU, OrtDevice::MemType::DEFAULT, OrtDevice::VendorIds::MICROSOFT, device_id)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Description
Make OrtDevice generic by removing vendor specific hardcoded memory types and replacing with generic DEFAULT/HOST_ACCESSIBLE values and an explicit vendor id to differentiate.
Remove unnecessary device id from OrtMemoryInfo. The OrtDevice in OrtMemoryInfo provides this.
Motivation and Context
Support plugin EPs