Skip to content

Make OrtDevice generic to support plugin EP usage. #24985

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jun 13, 2025

Conversation

skottmckay
Copy link
Contributor

Description

Make OrtDevice generic by removing vendor specific hardcoded memory types and replacing with generic DEFAULT/HOST_ACCESSIBLE values and an explicit vendor id to differentiate.

Remove unnecessary device id from OrtMemoryInfo. The OrtDevice in OrtMemoryInfo provides this.

Motivation and Context

Support plugin EPs

@jywu-msft jywu-msft requested a review from Copilot June 10, 2025 05:09
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR makes OrtDevice generic to support plugin EP usage by removing vendor‐specific hardcoded memory types and introducing explicit vendor IDs. Key changes include updating device construction for various execution providers and allocators, revising memory type comparisons and hash functions, and exposing a new CreateMemoryInfo_V2 API.

Reviewed Changes

Copilot reviewed 60 out of 60 changed files in this pull request and generated no comments.

Show a summary per file
File Description
onnxruntime/core/providers/dml/DmlExecutionProvider/src/ExecutionProvider.cpp Updates OrtDevice construction for DML EP with an explicit vendor ID.
onnxruntime/core/providers/dml/DmlExecutionProvider/src/DmlExternalBufferAllocator.h Changes the OrtDevice instantiation to use GPU type with vendor MICROSOFT.
onnxruntime/core/providers/cuda/* Replaces memory type CUDA_PINNED with HOST_ACCESSIBLE and updates device id usage.
onnxruntime/core/providers/cann/* Updates allocator and data transfer code to use HOST_ACCESSIBLE for cann.
onnxruntime/core/framework/* & include/* Refactors OrtMemoryInfo and OrtDevice usages to remove the id field and incorporate vendor IDs.
Comments suppressed due to low confidence (2)

onnxruntime/core/providers/dml/DmlExternalBufferAllocator.h:27

  • The device type is changed from DML (as used in other parts of the DML provider) to GPU here; please verify that this conversion is intentional for correct device identification.
OrtDevice(OrtDevice::GPU, OrtDevice::MemType::DEFAULT, OrtDevice::VendorIds::MICROSOFT, 0)

onnxruntime/core/framework/allocator.cc:184

  • For DML memory info, the device type is set to GPU with vendor MICROSOFT, which differs from the ExecutionProvider construction that uses OrtDevice::DML; please ensure this inconsistency is intentional.
OrtDevice(OrtDevice::GPU, OrtDevice::MemType::DEFAULT, OrtDevice::VendorIds::MICROSOFT, device_id)

Copy link
Member

@yuslepukhin yuslepukhin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@skottmckay skottmckay merged commit a5494ed into main Jun 13, 2025
89 checks passed
@skottmckay skottmckay deleted the skottmckay/UpdateOrtDeviceAndMemoryInfo branch June 13, 2025 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants