vmm: add NVIDIA GPUDirect P2P support.

[thomasbarrett]

On platforms where PCIe P2P is supported, inject a PCI capability into NVIDIA GPU to indicate support.

A previous PR added support for PCIe P2P between VFIO devices. The NVIDIA driver does not utilize PCIe P2P unless it detects hardware support. The PCIe specification doesn't require platforms to support PCIe P2P traffic between root ports (although many systems do). Within a virtual machine, information about the host PCIe bridge is lost, so NVIDIA recommends the use of an injected PCI capability to signal support for PCIe P2P between GPUs.

This PR adds a new argument x_nv_gpudirect_clique to VFIO devices --device path=<PCI_DEVICE_PATH>,x_nv_gpudirect_clique=<CLIQUE_ID> that injects a PCI capability indicating the "P2P clique" that the GPU belongs to. This option is only valid for modern datacenter NVIDIA GPUs (NVIDIA Turing, Ampere, Hopper, and Lovelace). The NVIDIA provided specification can be found here.

After enabling this feature, we benchmarked a decrease in GPU P2P latency from 12 us to 1.4 us compared to the fallback shared memory communication mechanism.

The nvidia-smi utility can be used to confirm that the P2P support. For example, the following cloud-hypervisor guest has 4 NVIDIA L40S GPUS.

nvidia-smi topo -p2p r
 	GPU0	GPU1	GPU2	GPU3	
 GPU0	X	OK	OK	OK	
 GPU1	OK	X	OK	OK	
 GPU2	OK	OK	X	OK	
 GPU3	OK	OK	OK	X	

Legend:

  X    = Self
  OK   = Status Ok
  CNS  = Chipset not supported
  GNS  = GPU not supported
  TNS  = Topology not supported
  NS   = Not supported
  U    = Unknown

[rbradford]

I slightly prefer the naming QEMU has - x-nv-gpudirect-clique - will also make it easier for folks to google, wdyt?

[thomasbarrett]

Yeah that makes sense to me @rbradford. Do you like x-nv-gpudirect-clique or nv-gpudirect-clique? We don’t have a “x-” naming convention in cloud-hypervisor?

[rbradford]

Yeah that makes sense to me @rbradford. Do you like x-nv-gpudirect-clique or nv-gpudirect-clique? We don’t have a “x-” naming convention in cloud-hypervisor?

Yes, I think the x- prefix is a good idea.

[rbradford]

@thomasbarrett the CI configuration has changed so please rebase your work on the latest "main" branch

[rbradford]

Thanks @thomasbarrett - without access to this specific hardware am trusting your testing :-)