cuda: add support for thor - #11999
Closed
dhiltgen wants to merge 1 commit into
Closed
Conversation
Collaborator
Author
|
AWS g5g T4G arm64 system uses CC 7.5, so I need to add that into the CC list to avoid regressions. |
AGX Thor is CC 11.0, which requires CUDA v13. Apparently arm64 SBSA CUDA support started at CC 7.5, so older arm CC targets we had been shipping were never used.
Collaborator
Author
|
It looks like v13 only works on driver 580 and up, so this change will wind up breaking systems out there running slightly older drivers. I don't think it's safe to merge, and we likely need some variation on #12000 to avoid breakage. On the g5g instance with a 535 driver this fails to use the GPU with: |
dhiltgen
marked this pull request as draft
August 21, 2025 21:07
Closed
|
You can use our wheels at this moment: https://pypi.jetson-ai-lab.io/sbsa/cu130 |
|
@dhiltgen how is thig going? do you need help? |
|
I do had the same error when using ollama. I was able to get it running using the instructions form the link. |
Collaborator
Author
|
v13 support has been added through #12000 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I'm exploring a few different options on how we can add Jetson AGX Thor support. This seems like it is probably the lowest risk change.
AGX Thor is CC 11.0, which requires CUDA v13.
Apparently arm64 SBSA CUDA support started at CC 8.0, so the 7.x and older arm CC targets we had been shipping were never used.
Eventually we'll need to support v13 for other new GPUs as they come out, so I'm also exploring adding back the dual CUDA stack support and trying v11 + v13 with "virtual" targets and leveraging JIT to keep the footprint reasonable, but I don't have that working yet on Windows.