Skip NVML tests if the hardware doesn't support NVML#1585
Skip NVML tests if the hardware doesn't support NVML#1585mdboom merged 11 commits intoNVIDIA:mainfrom
Conversation
|
Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
/ok to test |
There was a problem hiding this comment.
Pull request overview
This PR updates the test suites to skip NVML-related tests on platforms where NVML is present in the Python bindings but not actually supported by the underlying hardware/driver (e.g., Jetson Orin), as an alternative approach to parts of #1583.
Changes:
- Added an
hardware_supports_nvml()probe helper (cached) that attempts a minimal NVML call to detect true NVML support. - Extended NVML-related skip conditions in
cuda_coretests to include the new hardware probe. - Added a module-level skip for the
cuda_bindings/tests/nvmltest package when NVML isn’t supported by the platform.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| cuda_core/tests/test_device.py | Extends to_system_device test to skip when NVML hardware support is missing. |
| cuda_core/tests/system/conftest.py | Updates shared NVML skip marker to include a hardware support probe. |
| cuda_bindings/tests/nvml/init.py | Skips the entire NVML test package at import time when NVML isn’t supported. |
| cuda_bindings/cuda/bindings/_test_helpers/arch_check.py | Introduces hardware_supports_nvml() probe used by multiple test suites. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
cuda_core/tests/test_device.py
Outdated
| if not _system.CUDA_BINDINGS_NVML_IS_COMPATIBLE or not hardware_supports_nvml(): | ||
| with pytest.raises(RuntimeError): | ||
| device.to_system_device() | ||
| pytest.skip("NVML support requires cuda.bindings version 12.9.6+ or 13.1.2+") |
There was a problem hiding this comment.
The new hardware_supports_nvml() branch still asserts device.to_system_device() raises RuntimeError. On NVML-incompatible hardware (but bindings version compatible), to_system_device() can succeed up to constructing cuda.core.system.Device, which may then raise an nvml.NotSupportedError (not a RuntimeError), or potentially a different NVML error. This makes the test unreliable on the very platforms you’re trying to support. Consider splitting the cases: keep the RuntimeError assertion only for the version-incompatible path, and for not hardware_supports_nvml() just pytest.skip(...) (or assert the appropriate NVML exception type if you want coverage).
This comment has been minimized.
This comment has been minimized.
|
/ok to test |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
/ok to test |
|
/ok to test |
|
/ok to test |
|
/ok to test |
|
/ok to test |
|
/ok to test |
|
/ok to test |
This comment has been minimized.
This comment has been minimized.
3 similar comments
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
leofang
added
the
to-be-backported
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 12.9.x
git worktree add -d .worktree/backport-1585-to-12.9.x origin/12.9.x
cd .worktree/backport-1585-to-12.9.x
git switch --create backport-1585-to-12.9.x
git cherry-pick -x 1d7c89eded65451dfffcf0fb52622c10efb056b2 |
* Skip NVML tests if the hardware doesn't support NVML * Also skip converting between Device types * Fix test_to_system_device test * Update cuda_bindings/cuda/bindings/_test_helpers/arch_check.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix test * Simplify check * Fix import when cuda_bindings is incompatible * Fix logic * Move over a single test from NVIDIA#1583 * Fix check --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Skip NVML tests if the hardware doesn't support NVML * Also skip converting between Device types * Fix test_to_system_device test * Update cuda_bindings/cuda/bindings/_test_helpers/arch_check.py * Fix test * Simplify check * Fix import when cuda_bindings is incompatible * Fix logic * Move over a single test from #1583 * Fix check --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Some hardware, such as Jetson Orin, doesn't support NVML at all. We should skip the NVML-related tests on those platforms.
This is an alternative to most of #1583 (though it doesn't address the non-NVML change there).