Repository navigation
AMD GPU support in open-source Run:ai: please reconsider, or let the community carry it #5023
manmohanbrahma
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Thank you to @sarad-pm, @oferbaratz, @nsober and @camielRunai for the quick and clear answers on the AMD threads. I understand the position: AMD GPUs aren't in the support matrix, aren't tested, and aren't on the roadmap, so #3939, #3940, #3946, #4274 and PR #3950 were closed as not planned. I'd like to ask you to reconsider, and I'll say plainly why.
Why it matters for an open-source Run:ai
The value of opening Run:ai is that it can run on the infrastructure people actually have. Many GPU clusters today are mixed or moving that way: AMD Instinct next to NVIDIA, or AMD only. An open orchestrator that only works with one vendor's GPUs leaves those users with a choice between forking it and not using it. Neither is good for the project.
The ask is small. Run:ai already reads AMD labels in places: the AMD family table in cluster-sync, and AMD handling in the status-updater. What's missing is narrow:
amd.com/gpuin the node inventory and the Overview GPU totalNone of these changes the NVIDIA path, and each one is already running in a lab. We run Run:ai 3.0.20 with an NVIDIA RTX A6000 and an AMD Radeon PRO W7900 in one cluster, through Open Inference Fabric. GPU fractions on AMD are going through KAI Scheduler upstream (kai-scheduler/KAI-Scheduler#2556, kai-scheduler/gpu-fractioning#175).
A proposal that doesn't add to NVIDIA's support burden
amdlabel and a named owner on our side, so nothing lands on your team unreviewed or unowned.If that isn't possible
We'll keep AMD support going for the community in a fork, as a stop-gap: Run:ai's code with the AMD changes, kept in step with your releases, under its own name and following TRADEMARKS.md. We'd much rather not split the project, so we'd prefer to do this work here, and we'd offer every change back.
Could the maintainers look at this under the project's decision process (GOVERNANCE.md), and could the AMD threads be reopened while it's discussed?
Manmohan Brahma, Open Inference Fabric
All reactions