feat: pv-iommu + tpu fixes - #1
Draft
azenla wants to merge 11 commits into
Draft
Conversation
Port of Teddy Astie's RFC v3 driver for the Xen PV-IOMMU interface. Under
Xen the IOMMU belongs to the hypervisor, so a guest cannot use anything
that needs one -- VFIO, or DMA protection of its own. The PV-IOMMU
hypercall lets it manage IOMMU contexts through Xen instead, and this
driver presents that as a normal iommu_ops.
https://patchwork.kernel.org/project/xen-devel/list/?series=906092
Changes needed against 6.18:
- pgsize_bitmap moved from struct iommu_ops to struct iommu_domain, so
the mask the hypervisor reports is now stashed and applied to each
domain as it is allocated, including the identity domain.
- iommu_alloc_page()/iommu_free_page() are gone; the page table code uses
iommu_alloc_pages_sz()/iommu_free_pages().
CONFIG_XEN_IOMMU still depends on XEN_DOM0, matching the interface as
posted: the hypercall names devices by machine BDF, which a guest does not
see.
The driver itself was never Dom0-specific -- it only checks xen_domain() and sends the SBDF it sees, which is what the hypervisor now translates. The restriction lived entirely in Kconfig, so drop it to depends on XEN. Guests do need one adjustment. Dom0 sees the real topology, bridges included, and Xen tracks all of it. A guest only gets the endpoints vPCI assigned to it, sitting under an emulated bridge that has no device on the Xen side, so don't claim bridges there; attaching a context to one can only fail. A guest also has to call IOMMU_init before it has any contexts to allocate, which the driver already does unconditionally.
xen_iommu_identity_domain is statically allocated, and the core only fills in type and ops for domains it allocated itself. Left at zero the type is IOMMU_DOMAIN_BLOCKED, not identity, and the ops pointer is NULL, so anything the core routes to this domain either does the wrong thing or dereferences NULL. Name both explicitly. attach_dev is the same as for a paging domain -- the identity domain is just context 0.
A PCI host bridge's memory windows are reserved in IOVA space so that an
IOVA cannot collide with an address the bridge would route to MMIO rather
than to memory. Under a paravirtual IOMMU that reasoning does not apply:
the IOVA space belongs to the hypervisor, is programmed by hypercall, and
is not the bridge's address space at all.
Worse, the windows a Xen guest sees are invented. pcifront hands out
iomem_resource itself as the root bus window, and a PVH guest has no host
bridge _CRS so Linux falls back to its catch-all default, clipped only by
the guest's physical address width:
pci_bus 0000:00: root bus resource [mem 0x00000000-0x3fffffffffff]
Reserving that swallows the whole aperture the hypervisor reported, and
every IOVA allocation fails -- even for a single page:
nvidia 0000:00:01.0: IOVA alloc failed: len=1 limit=7fffffffff shift=12
granule=4096 start_pfn=1 aperture=0-7fffffffff
NVRM: GPU0 osIovaMap: failed to map allocation (status = 0x59)
The device then never gets a DMA mapping and the driver cannot attach. This
is inert without an IOMMU, since there is no IOVA space to reserve, so it
only appears once a guest drives one.
Let the driver owning the IOVA space opt out. Kept to a private header
rather than a flag in struct iommu_ops: adding a field there changes the
CRC of every exported symbol whose type graph reaches it, which breaks
every already-built out-of-tree module.
The BDF a PV guest sees for a passed-through device is invented by pciback in the hardware domain and never reaches the hypervisor, so it cannot be used to name the device in a hypercall. PV-IOMMU needs to. pciback already publishes both halves of the mapping in its xenstore directory -- dev-N is the machine BDF, vdev-N the one this domain sees -- and pcifront already reads vdev-N when detaching, so the mapping is right there. Walk it and hand the machine BDF back. Use it in the PV-IOMMU driver for both reattach and the reserved-region query. With vPCI the SBDF a guest sees is one Xen assigned and can translate, so this only changes what a PV guest sends. Both end up naming a device the hypervisor can find.
A PV guest reaches its devices through xen-swiotlb, which hands them machine addresses. Attaching one to a translated IOMMU context as well leaves two translations disagreeing about every address, and the DMA layer rejects the combination outright: WARNING: CPU: 0 PID: 11 at kernel/dma/mapping.c:881 dma_supported+0x5e/0x80 dma_set_mask+0x24/0xe0 nv_set_dma_address_size+0x52/0x70 [nvidia] nv_pci_probe+0x28f/0x1090 [nvidia] dma_supported() warns and returns false when a device both uses iommu-dma and has dma_map_ops, which in a PV guest it always does. The driver's dma_set_mask() then fails and every mapping after it is wrong; the GPU here got as far as GspStatusQueueInit before returning NV_ERR_RESET_REQUIRED. Report IOMMU_DOMAIN_IDENTITY as the default domain type there, which is context 0 and what the hypervisor already advertises as identity. Normal DMA stays on xen-swiotlb and translated contexts can still be asked for explicitly, which is what a guest wanting one for VFIO would do. Guests whose devices come through vPCI are unaffected; they have no dma_map_ops of their own and use iommu-dma as before.
A PV guest does not own the MSI-X table of a passed-through device. Xen programs it when mapping the pirq and maps the table read-only, so the mask, message and unmask writes in pci_write_msg_msix() fault. vfio-pci refreshes the cached message unconditionally before enabling a vector, to undo a backdoor reset, which crashes a PV guest: BUG: unable to handle page fault for address: ffffc9004001d000 #PF: supervisor write access in kernel mode RIP: __pci_write_msi_msg+0x74/0x1f0 vfio_msi_set_vector_signal+0x2c9/0x320 Return early as the is_virtual case already does; nothing in a PV guest programs the table itself, the Xen MSI domain routes allocation through PHYSDEVOP_map_pirq and Xen does the write.
A PV domain took the single-page path unconditionally, one hypercall per 4K page, because pfn_to_gfn(x + 1) != pfn_to_gfn(x) + 1 there in general and the batched path assumed the whole request was contiguous. Every subop also carries an IOTLB flush on the hypervisor side, so a large mapping costs a flush per page: a 4G premapping is a million hypercalls and a million flushes, which no caller waits out. Walk the request and issue one subop per maximal run contiguous in both dfn and gfn, as the TODO describing this asked for. Behaviour is unchanged where pfn_to_gfn is the identity, the run then covering the whole request. map_single_pages remains as a module parameter.
An IOMMU does not sit behind itself, so the hypervisor has no device for it and reattach fails with -ENODEV. The IOMMU core does not allow a driver to fail the first domain attach, so claiming it takes down registration of the whole driver: WARNING at drivers/iommu/iommu.c:3037 iommu_setup_default_domain iommu_device_register xen_iommu_init xen-iommu: Unable to register Xen IOMMU device -19 Only reachable in the initial domain, which sees the real topology; a guest is never given the IOMMU. Skip PCI class 0806.
pci_disable_msix() clears msix_enabled in pci_msix_shutdown() before pci_free_msi_irqs() reaches the domain teardown, so by the time xen_pv_teardown_msi_irqs() runs the flag can no longer say which of the two the device was using. It always took the MSI branch. The backend therefore never saw XEN_PCI_OP_disable_msix and left MSI-X enabled on the real device, and a later guest asking to enable it got -EALREADY from xen_pcibk_enable_msix(). Seen as a device usable exactly once per assignment. Take the type from a descriptor instead. The descriptor lock is already held by the caller.
A map subop covers a run of frames contiguous in both dfn and gfn, and in a PV domain gfns are machine frames, so a scattered buffer breaks into runs of a few pages. Each subop then flushed, and against an emulated IOMMU a flush is a command and a completion wait that costs far more than the mapping does. Mapping several gigabytes took long enough for callers to give up. The core already has somewhere to put this: ask the hypervisor to skip the per-subop flush and issue one over the whole range from iotlb_sync_map(). Gated on IOMMUCAP_deferred_flush so an older hypervisor keeps flushing per subop rather than never flushing at all.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TPU fixes and PV-IOMMU for Linux.