Skip to content

Commit c930ac4

Browse files
committed
Merge: VFIO update to v6.3
MR: https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/2383 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2188088 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2187660 Upstream Status: mainline Testing: Device assignment to Windows and Linux VMs, GPU, USB, NVMe, PF & VF NIC with NIC hotplug. Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Approved-by: Mika Penttilä <mpenttil@redhat.com> Approved-by: Jerry Snitselaar <jsnitsel@redhat.com> Approved-by: Thomas Huth <thuth@redhat.com> Approved-by: Cédric Le Goater <clg@redhat.com> Signed-off-by: Jan Stancek <jstancek@redhat.com>
2 parents 04554d1 + 0dda569 commit c930ac4

38 files changed

+641
-563
lines changed

Documentation/driver-api/vfio-mediated-device.rst

Lines changed: 1 addition & 107 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ devices as examples, as these devices are the first devices to use this module::
6262
| mdev.ko |
6363
| +-----------+ | mdev_register_parent() +--------------+
6464
| | | +<------------------------+ |
65-
| | | | | nvidia.ko |<-> physical
65+
| | | | | ccw_device.ko|<-> physical
6666
| | | +------------------------>+ | device
6767
| | | | callbacks +--------------+
6868
| | Physical | |
@@ -71,12 +71,6 @@ devices as examples, as these devices are the first devices to use this module::
7171
| | | | | i915.ko |<-> physical
7272
| | | +------------------------>+ | device
7373
| | | | callbacks +--------------+
74-
| | | |
75-
| | | | mdev_register_parent() +--------------+
76-
| | | +<------------------------+ |
77-
| | | | | ccw_device.ko|<-> physical
78-
| | | +------------------------>+ | device
79-
| | | | callbacks +--------------+
8074
| +-----------+ |
8175
+---------------+
8276

@@ -272,106 +266,6 @@ these callbacks are supported in the TYPE1 IOMMU module. To enable them for
272266
other IOMMU backend modules, such as PPC64 sPAPR module, they need to provide
273267
these two callback functions.
274268

275-
Using the Sample Code
276-
=====================
277-
278-
mtty.c in samples/vfio-mdev/ directory is a sample driver program to
279-
demonstrate how to use the mediated device framework.
280-
281-
The sample driver creates an mdev device that simulates a serial port over a PCI
282-
card.
283-
284-
1. Build and load the mtty.ko module.
285-
286-
This step creates a dummy device, /sys/devices/virtual/mtty/mtty/
287-
288-
Files in this device directory in sysfs are similar to the following::
289-
290-
# tree /sys/devices/virtual/mtty/mtty/
291-
/sys/devices/virtual/mtty/mtty/
292-
|-- mdev_supported_types
293-
| |-- mtty-1
294-
| | |-- available_instances
295-
| | |-- create
296-
| | |-- device_api
297-
| | |-- devices
298-
| | `-- name
299-
| `-- mtty-2
300-
| |-- available_instances
301-
| |-- create
302-
| |-- device_api
303-
| |-- devices
304-
| `-- name
305-
|-- mtty_dev
306-
| `-- sample_mtty_dev
307-
|-- power
308-
| |-- autosuspend_delay_ms
309-
| |-- control
310-
| |-- runtime_active_time
311-
| |-- runtime_status
312-
| `-- runtime_suspended_time
313-
|-- subsystem -> ../../../../class/mtty
314-
`-- uevent
315-
316-
2. Create a mediated device by using the dummy device that you created in the
317-
previous step::
318-
319-
# echo "83b8f4f2-509f-382f-3c1e-e6bfe0fa1001" > \
320-
/sys/devices/virtual/mtty/mtty/mdev_supported_types/mtty-2/create
321-
322-
3. Add parameters to qemu-kvm::
323-
324-
-device vfio-pci,\
325-
sysfsdev=/sys/bus/mdev/devices/83b8f4f2-509f-382f-3c1e-e6bfe0fa1001
326-
327-
4. Boot the VM.
328-
329-
In the Linux guest VM, with no hardware on the host, the device appears
330-
as follows::
331-
332-
# lspci -s 00:05.0 -xxvv
333-
00:05.0 Serial controller: Device 4348:3253 (rev 10) (prog-if 02 [16550])
334-
Subsystem: Device 4348:3253
335-
Physical Slot: 5
336-
Control: I/O+ Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
337-
Stepping- SERR- FastB2B- DisINTx-
338-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
339-
<TAbort- <MAbort- >SERR- <PERR- INTx-
340-
Interrupt: pin A routed to IRQ 10
341-
Region 0: I/O ports at c150 [size=8]
342-
Region 1: I/O ports at c158 [size=8]
343-
Kernel driver in use: serial
344-
00: 48 43 53 32 01 00 00 02 10 02 00 07 00 00 00 00
345-
10: 51 c1 00 00 59 c1 00 00 00 00 00 00 00 00 00 00
346-
20: 00 00 00 00 00 00 00 00 00 00 00 00 48 43 53 32
347-
30: 00 00 00 00 00 00 00 00 00 00 00 00 0a 01 00 00
348-
349-
In the Linux guest VM, dmesg output for the device is as follows:
350-
351-
serial 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
352-
0000:00:05.0: ttyS1 at I/O 0xc150 (irq = 10) is a 16550A
353-
0000:00:05.0: ttyS2 at I/O 0xc158 (irq = 10) is a 16550A
354-
355-
356-
5. In the Linux guest VM, check the serial ports::
357-
358-
# setserial -g /dev/ttyS*
359-
/dev/ttyS0, UART: 16550A, Port: 0x03f8, IRQ: 4
360-
/dev/ttyS1, UART: 16550A, Port: 0xc150, IRQ: 10
361-
/dev/ttyS2, UART: 16550A, Port: 0xc158, IRQ: 10
362-
363-
6. Using minicom or any terminal emulation program, open port /dev/ttyS1 or
364-
/dev/ttyS2 with hardware flow control disabled.
365-
366-
7. Type data on the minicom terminal or send data to the terminal emulation
367-
program and read the data.
368-
369-
Data is loop backed from hosts mtty driver.
370-
371-
8. Destroy the mediated device that you created::
372-
373-
# echo 1 > /sys/bus/mdev/devices/83b8f4f2-509f-382f-3c1e-e6bfe0fa1001/remove
374-
375269
References
376270
==========
377271

Documentation/driver-api/vfio.rst

Lines changed: 61 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -242,26 +242,28 @@ group and can access them as follows::
242242
VFIO User API
243243
-------------------------------------------------------------------------------
244244

245-
Please see include/linux/vfio.h for complete API documentation.
245+
Please see include/uapi/linux/vfio.h for complete API documentation.
246246

247247
VFIO bus driver API
248248
-------------------------------------------------------------------------------
249249

250250
VFIO bus drivers, such as vfio-pci make use of only a few interfaces
251251
into VFIO core. When devices are bound and unbound to the driver,
252-
the driver should call vfio_register_group_dev() and
253-
vfio_unregister_group_dev() respectively::
252+
Following interfaces are called when devices are bound to and
253+
unbound from the driver::
254254

255-
void vfio_init_group_dev(struct vfio_device *device,
256-
struct device *dev,
257-
const struct vfio_device_ops *ops);
258-
void vfio_uninit_group_dev(struct vfio_device *device);
259255
int vfio_register_group_dev(struct vfio_device *device);
256+
int vfio_register_emulated_iommu_dev(struct vfio_device *device);
260257
void vfio_unregister_group_dev(struct vfio_device *device);
261258

262-
The driver should embed the vfio_device in its own structure and call
263-
vfio_init_group_dev() to pre-configure it before going to registration
264-
and call vfio_uninit_group_dev() after completing the un-registration.
259+
The driver should embed the vfio_device in its own structure and use
260+
vfio_alloc_device() to allocate the structure, and can register
261+
@init/@release callbacks to manage any private state wrapping the
262+
vfio_device::
263+
264+
vfio_alloc_device(dev_struct, member, dev, ops);
265+
void vfio_put_device(struct vfio_device *device);
266+
265267
vfio_register_group_dev() indicates to the core to begin tracking the
266268
iommu_group of the specified dev and register the dev as owned by a VFIO bus
267269
driver. Once vfio_register_group_dev() returns it is possible for userspace to
@@ -270,28 +272,64 @@ ready before calling it. The driver provides an ops structure for callbacks
270272
similar to a file operations structure::
271273

272274
struct vfio_device_ops {
273-
int (*open)(struct vfio_device *vdev);
275+
char *name;
276+
int (*init)(struct vfio_device *vdev);
274277
void (*release)(struct vfio_device *vdev);
278+
int (*bind_iommufd)(struct vfio_device *vdev,
279+
struct iommufd_ctx *ictx, u32 *out_device_id);
280+
void (*unbind_iommufd)(struct vfio_device *vdev);
281+
int (*attach_ioas)(struct vfio_device *vdev, u32 *pt_id);
282+
int (*open_device)(struct vfio_device *vdev);
283+
void (*close_device)(struct vfio_device *vdev);
275284
ssize_t (*read)(struct vfio_device *vdev, char __user *buf,
276285
size_t count, loff_t *ppos);
277-
ssize_t (*write)(struct vfio_device *vdev,
278-
const char __user *buf,
279-
size_t size, loff_t *ppos);
286+
ssize_t (*write)(struct vfio_device *vdev, const char __user *buf,
287+
size_t count, loff_t *size);
280288
long (*ioctl)(struct vfio_device *vdev, unsigned int cmd,
281289
unsigned long arg);
282-
int (*mmap)(struct vfio_device *vdev,
283-
struct vm_area_struct *vma);
290+
int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
291+
void (*request)(struct vfio_device *vdev, unsigned int count);
292+
int (*match)(struct vfio_device *vdev, char *buf);
293+
void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
294+
int (*device_feature)(struct vfio_device *device, u32 flags,
295+
void __user *arg, size_t argsz);
284296
};
285297

286298
Each function is passed the vdev that was originally registered
287-
in the vfio_register_group_dev() call above. This allows the bus driver
288-
to obtain its private data using container_of(). The open/release
289-
callbacks are issued when a new file descriptor is created for a
290-
device (via VFIO_GROUP_GET_DEVICE_FD). The ioctl interface provides
291-
a direct pass through for VFIO_DEVICE_* ioctls. The read/write/mmap
292-
interfaces implement the device region access defined by the device's
293-
own VFIO_DEVICE_GET_REGION_INFO ioctl.
299+
in the vfio_register_group_dev() or vfio_register_emulated_iommu_dev()
300+
call above. This allows the bus driver to obtain its private data using
301+
container_of().
302+
303+
::
304+
305+
- The init/release callbacks are issued when vfio_device is initialized
306+
and released.
307+
308+
- The open/close device callbacks are issued when the first
309+
instance of a file descriptor for the device is created (eg.
310+
via VFIO_GROUP_GET_DEVICE_FD) for a user session.
311+
312+
- The ioctl callback provides a direct pass through for some VFIO_DEVICE_*
313+
ioctls.
314+
315+
- The [un]bind_iommufd callbacks are issued when the device is bound to
316+
and unbound from iommufd.
317+
318+
- The attach_ioas callback is issued when the device is attached to an
319+
IOAS managed by the bound iommufd. The attached IOAS is automatically
320+
detached when the device is unbound from iommufd.
321+
322+
- The read/write/mmap callbacks implement the device region access defined
323+
by the device's own VFIO_DEVICE_GET_REGION_INFO ioctl.
324+
325+
- The request callback is issued when device is going to be unregistered,
326+
such as when trying to unbind the device from the vfio bus driver.
294327

328+
- The dma_unmap callback is issued when a range of iovas are unmapped
329+
in the container or IOAS attached by the device. Drivers which make
330+
use of the vfio page pinning interface must implement this callback in
331+
order to unpin pages within the dma_unmap range. Drivers must tolerate
332+
this callback even before calls to open_device().
295333

296334
PPC64 sPAPR implementation note
297335
-------------------------------

Documentation/s390/pci.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Entries specific to zPCI functions and entries that hold zPCI information.
5151

5252
The slot entries are set up using the function identifier (FID) of the
5353
PCI function. The format depicted as XXXXXXXX above is 8 hexadecimal digits
54-
with 0 padding and lower case hexadecimal digitis.
54+
with 0 padding and lower case hexadecimal digits.
5555

5656
- /sys/bus/pci/slots/XXXXXXXX/power
5757

@@ -66,7 +66,7 @@ Entries specific to zPCI functions and entries that hold zPCI information.
6666

6767
- function_handle
6868
Low-level identifier used for a configured PCI function.
69-
It might be useful for debuging.
69+
It might be useful for debugging.
7070

7171
- pchid
7272
Model-dependent location of the I/O adapter.

Documentation/s390/vfio-ap.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -553,7 +553,6 @@ These are the steps:
553553
* ZCRYPT
554554
* S390_AP_IOMMU
555555
* VFIO
556-
* VFIO_MDEV
557556
* KVM
558557

559558
If using make menuconfig select the following to build the vfio_ap module::

Documentation/s390/vfio-ccw.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -176,7 +176,7 @@ The process of how these work together.
176176
Use the 'mdev_create' sysfs file, we need to manually create one (and
177177
only one for our case) mediated device.
178178
3. vfio_mdev.ko drives the mediated ccw device.
179-
vfio_mdev is also the vfio device drvier. It will probe the mdev and
179+
vfio_mdev is also the vfio device driver. It will probe the mdev and
180180
add it to an iommu_group and a vfio_group. Then we could pass through
181181
the mdev to a guest.
182182

@@ -219,8 +219,8 @@ values may occur:
219219
The operation was successful.
220220

221221
``-EOPNOTSUPP``
222-
The orb specified transport mode or an unidentified IDAW format, or the
223-
scsw specified a function other than the start function.
222+
The ORB specified transport mode or the
223+
SCSW specified a function other than the start function.
224224

225225
``-EIO``
226226
A request was issued while the device was not in a state ready to accept

MAINTAINERS

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19860,7 +19860,6 @@ F: fs/fat/
1986019860

1986119861
VFIO DRIVER
1986219862
M: Alex Williamson <alex.williamson@redhat.com>
19863-
R: Cornelia Huck <cohuck@redhat.com>
1986419863
L: kvm@vger.kernel.org
1986519864
S: Maintained
1986619865
T: git https://github.com/awilliam/linux-vfio.git

arch/s390/Kconfig

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -780,7 +780,9 @@ config EADM_SCH
780780
config VFIO_CCW
781781
def_tristate n
782782
prompt "Support for VFIO-CCW subchannels"
783-
depends on S390_CCW_IOMMU && VFIO_MDEV
783+
depends on S390_CCW_IOMMU
784+
depends on VFIO
785+
select VFIO_MDEV
784786
help
785787
This driver allows usage of I/O subchannels via VFIO-CCW.
786788

@@ -790,14 +792,16 @@ config VFIO_CCW
790792
config VFIO_AP
791793
def_tristate n
792794
prompt "VFIO support for AP devices"
793-
depends on S390_AP_IOMMU && VFIO_MDEV && KVM
795+
depends on S390_AP_IOMMU && KVM
796+
depends on VFIO
794797
depends on ZCRYPT
798+
select VFIO_MDEV
795799
help
796-
This driver grants access to Adjunct Processor (AP) devices
797-
via the VFIO mediated device interface.
800+
This driver grants access to Adjunct Processor (AP) devices
801+
via the VFIO mediated device interface.
798802

799-
To compile this driver as a module, choose M here: the module
800-
will be called vfio_ap.
803+
To compile this driver as a module, choose M here: the module
804+
will be called vfio_ap.
801805

802806
endmenu
803807

arch/s390/configs/debug_defconfig

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -580,7 +580,6 @@ CONFIG_MLX5_INFINIBAND=m
580580
CONFIG_SYNC_FILE=y
581581
CONFIG_VFIO=m
582582
CONFIG_VFIO_PCI=m
583-
CONFIG_VFIO_MDEV=m
584583
CONFIG_VIRTIO_PCI=m
585584
CONFIG_VIRTIO_BALLOON=m
586585
CONFIG_VIRTIO_INPUT=y

arch/s390/configs/defconfig

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -572,7 +572,6 @@ CONFIG_MLX5_INFINIBAND=m
572572
CONFIG_SYNC_FILE=y
573573
CONFIG_VFIO=m
574574
CONFIG_VFIO_PCI=m
575-
CONFIG_VFIO_MDEV=m
576575
CONFIG_VIRTIO_PCI=m
577576
CONFIG_VIRTIO_BALLOON=m
578577
CONFIG_VIRTIO_INPUT=y

arch/s390/include/asm/idals.h

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,9 @@
2323
#define IDA_SIZE_LOG 12 /* 11 for 2k , 12 for 4k */
2424
#define IDA_BLOCK_SIZE (1L<<IDA_SIZE_LOG)
2525

26+
#define IDA_2K_SIZE_LOG 11
27+
#define IDA_2K_BLOCK_SIZE (1L << IDA_2K_SIZE_LOG)
28+
2629
/*
2730
* Test if an address/length pair needs an idal list.
2831
*/
@@ -42,6 +45,15 @@ static inline unsigned int idal_nr_words(void *vaddr, unsigned int length)
4245
(IDA_BLOCK_SIZE-1)) >> IDA_SIZE_LOG;
4346
}
4447

48+
/*
49+
* Return the number of 2K IDA words needed for an address/length pair.
50+
*/
51+
static inline unsigned int idal_2k_nr_words(void *vaddr, unsigned int length)
52+
{
53+
return ((__pa(vaddr) & (IDA_2K_BLOCK_SIZE - 1)) + length +
54+
(IDA_2K_BLOCK_SIZE - 1)) >> IDA_2K_SIZE_LOG;
55+
}
56+
4557
/*
4658
* Create the list of idal words for an address/length pair.
4759
*/

0 commit comments

Comments
 (0)