NvTensorRtRtx dependency on CUDA device name removed #1485
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
NvTensorRtRtx dependency on using ORT CUDA EP removed.
BLSharda
changed the title
- Fixing formatting for CICD failure - Adding final inheritance for CUDA and NvTensorRtRtx EP interfaces impl structs. - Removing Graph Capture for NvTensorRtRtx EP for now.
Fixing compilation error for linux builds
…ub.com/BLSharda/onnxruntime-genai into bhaviksharda/nvep_removes_cuda_ep_deps
BLSharda
changed the title
|
@kunal-vaishnavi and @baijumeswani , can you help review and merge this change? |
- Removing cuda graph knob for now until we test it for functional and perf aspect.
|
@baijumeswani, can you please help with review and merge this change? |
Could you elaborate or share the relevant onnxruntime pull-request so we can track the reason for this chance? |
In PR 24656 CUDA dependency is removed (USE_CUDA) for NvTensorRtRtx EP so we need to switch device name from "cuda" to "NvTensorRtRtx". |
baijumeswani
approved these changes
May 29, 2025
baijumeswani
pushed a commit
that referenced
this pull request
May 29, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
GenAI EP NvTensorRtRtx dependency on using CUDA device removed. This is to align with latest changes in ORT and NV EP.
Also, still requires onnxruntime-genai-cuda.dll but device name is NvTensorRtRtx instead of CUDA. And the memory type is still "Cuda" for ORT API.