We are excited to announce a new open-source release of our GIM SR-IOV Driver. Along with regular bug fixes and improvements, this release is aligned with ROCm 10.1.
New in this release:
- Ubuntu 26.04 support - added as a host and guest OS on MI210X, MI300X, MI325X, MI350X, and MI355X
- Ubuntu 26.04 and RHEL 10.2 support - added Ubuntu 26.04 as a host OS and Ubuntu 26.04 and RHEL 10.2 as a guest OS on Radeon V710
- VF power-cap control - Enables power-cap management for virtual functions on MI210X
Hardware / Features Supported for this Release
| AMD GPU | Host OS | Guest OS | VF per GPU - Compute Mode - Memory Mode | Infinity Fabric (XGMI) Configuration | ROCm / Graphics Version |
|---|---|---|---|---|---|
| AMD Instinct MI355X / AMD Instinct MI350X | Ubuntu 26.04 Kernel 7.0 | Ubuntu 26.04 | 1 - SPX - NPS1 | Between 8 GPUs | ROCm 10.1 |
| Ubuntu 24.04 Kernel 6.14 | Ubuntu 24.04 | 1 - SPX - NPS1 2* - DPX - NPS2 8* - CPX - NPS2 | |||
| RHEL 10.0 | 1 - SPX - NPS1 | ||||
| RHEL 9.6 | 1 - SPX - NPS1 | ||||
| AMD Instinct MI325X | Ubuntu 26.04 Kernel 7.0 | Ubuntu 26.04 | 1 - SPX - NPS1 | Between 8 GPUs | ROCm 10.1 |
| Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | 1 - SPX - NPS1 | |||
| RHEL 10.2 Kernel 6.12.0-211 | RHEL 10.2 | 1 - SPX - NPS1 | |||
| RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4 | 1 - SPX - NPS1 | |||
| Ubuntu 26.04 | 1 - SPX - NPS1 | ||||
| AMD Instinct MI300X | Ubuntu 26.04 Kernel 7.0 | Ubuntu 26.04 | 1 - SPX - NPS1 | Between 8 GPUs | ROCm 10.1 |
| Ubuntu 24.04 Kernel 6.14 | Ubuntu 24.04 | 1 - SPX - NPS1 8* - CPX - NPS4 | |||
| Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | 1 - SPX - NPS1 | |||
| RHEL 10.2 Kernel 6.12.0-211 | RHEL 10.2 | 1 - SPX - NPS1 | |||
| RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4 | 1 - SPX - NPS1 8* - CPX - NPS4 | |||
| Ubuntu 24.04 | 1 - SPX - NPS1 8* - CPX - NPS4 | ||||
| AMD Instinct MI210X | Ubuntu 26.04 Kernel 7.0 | Ubuntu 26.04 | 1 | Between 4 / 8 GPUs | ROCm 10.1 |
| RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4 | 1 | |||
| Ubuntu 22.04 | 1 | ||||
| AMD Radeon PRO V710 | Ubuntu 26.04 Kernel 7.0 | ROCm and Graphics: Ubuntu 26.04 Ubuntu 24.04 RHEL 10.2 RHEL 9.6 |
1 to 12 | N/A | ROCm 10.1 amdgpu 6.4.2.2 |
| Ubuntu 24.04 Kernel 6.14 | ROCm and Graphics: Ubuntu 24.04 RHEL 10.2 RHEL 9.6 |
1 to 12 |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX | ||
| 2* | DPX | MI35XX | ||
| 8* | CPX | MI35XX | MI300X |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF Support requires a compatible firmware. Please refer to the ROCm Firmware Compatibility Table
Host AMD SMI Versions
- Library: 64.6.1
- Tool: 38.3.2
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- On VMs with multiple VF assigned, detaching the VF (e.g. with virsh detach-device) while a workload is running will cause the guest kernel to refuse further GPU work. The VM needs to be restarted before GPU work can be issued again. This issue is limited to SPX/NPS1 configuration only.
- MI350X: 1 VF mode, there is a small chance VF FLRs will fail due to an SMU timeout if a reset is triggered concurrently with an active GFX workload. The VF FLR will be promoted to a whole GPU reset in these cases. This reset may interrupt workloads running on other VFs in a multi-tenancy setup.
- Latest Ubuntu Kernel version 6.8.0-136-generic has a regression causing boot times to increase. This regression is being reverted, until then please use Ubuntu Kernel version 6.8.0-90-generic.