Skip to content

Expose Python OrtDeviceVendorId enum and use it for vendor-aware OrtDevice aliases#27594

Merged
tianleiwu merged 1 commit intomainfrom
tlwu/20260309/ort_device_vendor_id_python_api
Mar 12, 2026
Merged

Expose Python OrtDeviceVendorId enum and use it for vendor-aware OrtDevice aliases#27594
tianleiwu merged 1 commit intomainfrom
tlwu/20260309/ort_device_vendor_id_python_api

Conversation

@tianleiwu
Copy link
Contributor

Summary

This change replaces the Python-side PCI vendor ID constants in onnxruntime_inference_collection.py with a
public OrtDeviceVendorId enum, exports that enum from onnxruntime.__init__, and continues using vendor-aware
OrtDevice construction for well-known aliases like "cuda", "dml", and "cann".

The goal is to make vendor IDs reusable across Python APIs without duplicating raw integer constants while still
fixing the plugin EP allocator lookup issue for OrtValue.ortvalue_from_numpy(..., "cuda", ...).

Problem

The Python wrapper now needs vendor IDs in more than one place.

Keeping them as standalone integer constants is workable for a narrow fix, but it does not give Python callers a
clear public API for vendor identity. As more APIs accept or return vendor IDs, users would have to either:

  • remember the raw PCI ID values
  • depend on private implementation details
  • repeat ad hoc constants in their own code

That is not a good public surface for something that is now part of regular Python device construction flows.

Why We Need This Change

Dynamic EP registration is intended to let a Python package gain hardware capability without requiring that hardware
support to be built into the package itself.

That only works if the Python-side device description matches the device identity used by dynamically registered EP
allocators and data transfers.

Without the underlying vendor-aware alias behavior:

  • registering the CUDA plugin library succeeds
  • sessions can use CudaPluginExecutionProvider
  • but Python cannot create CUDA OrtValues with OrtValue.ortvalue_from_numpy(..., "cuda", 0)

At the same time, without a public enum:

  • vendor IDs remain scattered as raw integers
  • Python callers do not have a clean symbolic way to specify vendor-specific "gpu" / "npu" devices
  • future vendor-aware APIs would keep expanding the same constant-style pattern

Example Use Case

Our immediate use case is the CUDA plugin EP flow from Python.

We register libonnxruntime_providers_cuda_plugin.so from Python and create sessions with
CudaPluginExecutionProvider. That part works.

Stage 4 of the plugin flow needs to create GPU-resident OrtValues:

onnxruntime.OrtValue.ortvalue_from_numpy(array, "cuda", 0)

Before the vendor-aware alias fix, that failed in a CPU-only Python package even after the CUDA plugin was
registered, because the Python wrapper constructed a generic GPU OrtDevice without the NVIDIA vendor ID.

With this change, Python also has a public enum for vendor IDs, so callers can write explicit vendor-aware code
when using generic device names:

onnxruntime.OrtValue.ortvalue_from_numpy(
    array,
    "gpu",
    0,
    onnxruntime.OrtDeviceVendorId.NVIDIA,
)

Fix

The change does three things:

  1. Replace the Python-side PCI vendor ID constants with an IntEnum named OrtDeviceVendorId.

  2. Export OrtDeviceVendorId from onnxruntime.__init__ so it is part of the public Python API.

  3. Keep the vendor-aware alias behavior in OrtDevice.make(...) so that the historical shorthand aliases:

    • "cuda" -> OrtDeviceVendorId.NVIDIA
    • "dml" -> OrtDeviceVendorId.MICROSOFT
    • "cann" -> OrtDeviceVendorId.HUAWEI

    use the 4-argument C.OrtDevice(...) constructor with an explicit vendor ID.

Generic device names like "gpu" and "npu" continue to behave as before unless the caller explicitly provides a
vendor ID, and callers can now use either an integer or OrtDeviceVendorId.

Why This Approach

An enum is the better public API here because it:

  • keeps Python aligned with the core runtime vendor ID definitions in ortdevice.h
  • preserves integer compatibility because IntEnum still works naturally with the pybind layer
  • gives users readable, discoverable names instead of undocumented raw PCI IDs
  • scales better as vendor-aware device APIs become more common

This keeps the original plugin fix intact while improving the Python API shape instead of just adding more module
constants.

Validation

Validated in the Python layer by:

  • confirming the new enum-based implementation preserves vendor-aware alias handling for "cuda", "dml", and "cann"
  • exporting OrtDeviceVendorId from the top-level onnxruntime package
  • adding Python test coverage that checks OrtDevice.make("cuda", 0) resolves to the NVIDIA vendor ID via the enum
  • running python -m compileall on the updated Python files

Targeted pytest execution could not be completed in this workspace because the local source tree does not provide an
importable onnxruntime.capi module without a built package.

Notes

This PR keeps backward compatibility for existing Python call sites:

  • shorthand aliases like "cuda" continue to work
  • explicit vendor_id arguments can still be passed as integers
  • callers now also have the option to use onnxruntime.OrtDeviceVendorId

@tianleiwu tianleiwu merged commit dd58b54 into main Mar 12, 2026
91 checks passed
@tianleiwu tianleiwu deleted the tlwu/20260309/ort_device_vendor_id_python_api branch March 12, 2026 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants