Skip to content

Conversation

@PvsNarasimha
Copy link

IOMMU - PCI Multisegment Support

  • iommu/amd: Update amd_iommu_fault structure to include PCI seg ID
  • iommu/amd: Update device_state structure to include PCI seg ID
  • iommu/amd: Print PCI segment ID in error log messages
  • iommu/amd: Specify PCI segment ID when getting pci device
  • iommu/amd: Include PCI segment ID when initialize IOMMU
  • iommu/amd: Introduce get_device_sbdf_id() helper function
  • iommu/amd: Flush upto last_bdf only
  • iommu/amd: Remove global amd_iommu_[dev_table/alias_table/last_bdf]
  • iommu/amd: Update set_dev_entry_bit() and get_dev_entry_bit()
  • iommu/amd: Update (un)init_device_table_dma()
  • iommu/amd: Update set_dte_irq_entry
  • iommu/amd: Update dump_dte_entry
  • iommu/amd: Update iommu_ignore_device
  • iommu/amd: Update set_dte_entry and clear_dte_entry
  • iommu/amd: Convert to use per PCI segment rlookup_table
  • iommu/amd: Update alloc_irq_table and alloc_irq_index
  • iommu/amd: Update amd_irte_ops functions
  • iommu/amd: Introduce struct amd_ir_data.iommu
  • iommu/amd: Update irq_remapping_alloc to use IOMMU lookup helper function
  • iommu/amd: Convert to use rlookup_amd_iommu helper function
  • iommu/amd: Convert to use per PCI segment irq_lookup_table
  • iommu/amd: Introduce per PCI segment rlookup table size
  • iommu/amd: Introduce per PCI segment alias table size
  • iommu/amd: Introduce per PCI segment device table size
  • iommu/amd: Introduce per PCI segment last_bdf
  • iommu/amd: Introduce per PCI segment unity map list
  • iommu/amd: Introduce per PCI segment alias_table
  • iommu/amd: Introduce per PCI segment old_dev_tbl_cpy
  • iommu/amd: Introduce per PCI segment dev_data_list
  • iommu/amd: Introduce per PCI segment irq_lookup_table
  • iommu/amd: Introduce per PCI segment rlookup table
  • iommu/amd: Introduce per PCI segment device table
  • iommu/amd: Introduce pci segment structure
  • iommu/amd: Update struct iommu_dev_data definition

Run Test cases

As per the following AMD/RHEL document, the PCI multi segment is supported by both AMD's Genoa and Turin processors:
https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/tuning-guides/58469_amd-epyc-9005-tg-redhat-enterprise-linux.pdf
We checked our Turin system (where we done the testing) & found 4 PCI segments [ PCI multiple segment support ].
Please see the follow the steps

  1. BIOS: Advanced --> AMD CBS -> DF common options --> Number of PCI segments [change to 4]

Kernel Log messages
amd@volcano9dee-host:~$ sudo dmesg | grep -i domain

[sudo] password for amd:
[    0.718557] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x3ffb00000000-0x3ffbffffffff] (base 0x3ffb00000000)
[    0.718558] PCI: MMCONFIG for domain 0001 [bus 00-ff] at [mem 0x3ffb10000000-0x3ffb1fffffff] (base 0x3ffb10000000)
[    0.718558] PCI: MMCONFIG for domain 0002 [bus 00-ff] at [mem 0x3ffb20000000-0x3ffb2fffffff] (base 0x3ffb20000000)
[    0.718559] PCI: MMCONFIG for domain 0003 [bus 00-ff] at [mem 0x3ffb30000000-0x3ffb3fffffff] (base 0x3ffb30000000)
[    0.845414] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x3ffb00000000-0x3ffbffffffff] (base 0x3ffb00000000)
[    0.845414] PCI: MMCONFIG for domain 0001 [bus 00-ff] at [mem 0x3ffb10000000-0x3ffb1fffffff] (base 0x3ffb10000000)
[    0.845415] PCI: MMCONFIG for domain 0002 [bus 00-ff] at [mem 0x3ffb20000000-0x3ffb2fffffff] (base 0x3ffb20000000)
[    0.845415] PCI: MMCONFIG for domain 0003 [bus 00-ff] at [mem 0x3ffb30000000-0x3ffb3fffffff] (base 0x3ffb30000000)
root@volcano9dee-host:/home/amd/Narasimha/kernel# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-5.15.152withpatches+ root=/dev/mapper/volcano9dee--host--vg-root ro quiet splash iommu=on la57 amd_iommu_dump=1 amd_iommu_debug=1
root@volcano9dee-host:/home/amd/Narasimha/kernel# sudo dmesg | grep "AMD-Vi"
[    0.262613] AMD-Vi: AMD-Vi: Using IVHD type 0x40
[    0.262615] AMD-Vi: AMD-Vi: device: 0001:c0:00.2 cap: 0040 flags: 30 info 0000
[    0.262616] AMD-Vi: AMD-Vi:        mmio-addr: 00000000deb00000
[    0.262618] AMD-Vi: AMD-Vi: PCI segment : 0x1, last bdf : 0xfffe
[    0.262749] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0001:c0:00.3 flags: 00
[    0.262750] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0001:ff:1f.6
[    0.262872] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0001:00:00.3 flags: 00
[    0.262872] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0001:3f:1f.6
[    0.262999] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[240])               devid: 0001:c0:00.1
[    0.263001] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[244])               devid: 0001:00:00.1
[    0.263002] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0096[ID01])           devid: 0001:c0:00.4
[    0.263003] AMD-Vi: ivrs, add hid:AMDI0096, uid:ID01, rdevid:114688
[    0.263006] AMD-Vi: AMD-Vi: device: 0001:40:00.2 cap: 0040 flags: 30 info 0000
[    0.263007] AMD-Vi: AMD-Vi:        mmio-addr: 00000000d8900000
[    0.263014] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0001:40:00.3 flags: 00
[    0.263015] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0001:7f:1f.6
[    0.263141] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0001:80:00.3 flags: 00
[    0.263142] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0001:bf:1f.6
[    0.263268] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[241])               devid: 0001:40:00.1
[    0.263269] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[245])               devid: 0001:80:00.1
[    0.263271] AMD-Vi: AMD-Vi: device: 0000:00:00.2 cap: 0040 flags: 30 info 0000
[    0.263272] AMD-Vi: AMD-Vi:        mmio-addr: 00000000dfb00000
[    0.263273] AMD-Vi: AMD-Vi: PCI segment : 0x0, last bdf : 0xffff
[    0.263390] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0000:00:00.3 flags: 00
[    0.263391] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0000:3f:1f.6
[    0.263517] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0000:c0:00.3 flags: 00
[    0.263518] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0000:ff:1f.6
[    0.263654] AMD-Vi: AMD-Vi:   DEV_ALIAS_RANGE                 devid: 0000:ff:00.0 flags: 00 devid_to: 0000:00:14.5
[    0.263656] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0000:ff:1f.7
[    0.263660] AMD-Vi: AMD-Vi:   DEV_SPECIAL(HPET[0])           devid: 0000:00:14.0
[    0.263661] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[128])               devid: 0000:00:14.0
[    0.263662] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[242])               devid: 0000:00:00.1
[    0.263662] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[246])               devid: 0000:c0:00.1
[    0.263663] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0020[ID00])           devid: 0000:00:14.5
[    0.263664] AMD-Vi: ivrs, add hid:AMDI0020, uid:ID00, rdevid:160
[    0.263664] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0020[ID01])           devid: 0000:00:14.5
[    0.263665] AMD-Vi: ivrs, add hid:AMDI0020, uid:ID01, rdevid:160
[    0.263666] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0020[ID02])           devid: 0000:00:14.5
[    0.263666] AMD-Vi: ivrs, add hid:AMDI0020, uid:ID02, rdevid:160
[    0.263667] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0020[ID03])           devid: 0000:00:14.5
[    0.263667] AMD-Vi: ivrs, add hid:AMDI0020, uid:ID03, rdevid:160
[    0.263668] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0095[ID00])           devid: 0000:00:00.4
[    0.263669] AMD-Vi: ivrs, add hid:AMDI0095, uid:ID00, rdevid:0
[    0.263670] AMD-Vi: AMD-Vi: device: 0000:80:00.2 cap: 0040 flags: 30 info 0000
[    0.263671] AMD-Vi: AMD-Vi:        mmio-addr: 00000000cb900000
[    0.263677] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0000:80:00.3 flags: 00
[    0.263678] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0000:bf:1f.6
[    0.263804] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0000:40:00.3 flags: 00
[    0.263805] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0000:7f:1f.6
[    0.263931] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[243])               devid: 0000:80:00.1
[    0.263932] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[247])               devid: 0000:40:00.1
[    0.263934] AMD-Vi: AMD-Vi: device: 0003:c0:00.2 cap: 0040 flags: 30 info 0000
[    0.263934] AMD-Vi: AMD-Vi:        mmio-addr: 00000000c5900000
[    0.263935] AMD-Vi: AMD-Vi: PCI segment : 0x3, last bdf : 0xfffe
[    0.264055] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0003:c0:00.3 flags: 00
[    0.264056] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0003:ff:1f.6
[    0.264182] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0003:00:00.3 flags: 00
[    0.264182] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0003:3f:1f.6
[    0.264309] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[248])               devid: 0003:c0:00.1
[    0.264310] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[252])               devid: 0003:00:00.1
[    0.264311] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0096[ID03])           devid: 0003:c0:00.4
[    0.264311] AMD-Vi: ivrs, add hid:AMDI0096, uid:ID03, rdevid:245760
[    0.264313] AMD-Vi: AMD-Vi: device: 0003:40:00.2 cap: 0040 flags: 30 info 0000
[    0.264313] AMD-Vi: AMD-Vi:        mmio-addr: 00000000bf000000
[    0.264321] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0003:40:00.3 flags: 00
[    0.264321] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0003:7f:1f.6
[    0.264447] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0003:80:00.3 flags: 00
[    0.264448] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0003:bf:1f.6
[    0.264575] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[249])               devid: 0003:40:00.1
[    0.264575] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[253])               devid: 0003:80:00.1
[    0.264577] AMD-Vi: AMD-Vi: device: 0002:00:00.2 cap: 0040 flags: 30 info 0000
[    0.264578] AMD-Vi: AMD-Vi:        mmio-addr: 00000000b9200000
[    0.264578] AMD-Vi: AMD-Vi: PCI segment : 0x2, last bdf : 0xfffe
[    0.264698] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0002:00:00.3 flags: 00
[    0.264699] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0002:3f:1f.6
[    0.264825] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0002:c0:00.3 flags: 00
[    0.264825] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0002:ff:1f.6
[    0.264952] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[250])               devid: 0002:00:00.1
[    0.264953] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[254])               devid: 0002:c0:00.1
[    0.264953] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0020[ID00])           devid: 0002:00:14.5
[    0.264954] AMD-Vi: ivrs, add hid:AMDI0020, uid:ID00, rdevid:131232
[    0.264954] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0020[ID01])           devid: 0002:00:14.5
[    0.264955] AMD-Vi: ivrs, add hid:AMDI0020, uid:ID01, rdevid:131232
[    0.264955] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0020[ID02])           devid: 0002:00:14.5
[    0.264956] AMD-Vi: ivrs, add hid:AMDI0020, uid:ID02, rdevid:131232
[    0.264957] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0020[ID03])           devid: 0002:00:14.5
[    0.264957] AMD-Vi: ivrs, add hid:AMDI0020, uid:ID03, rdevid:131232
[    0.264958] AMD-Vi: AMD-Vi:   DEV_ACPI_HID(AMDI0096[ID02])           devid: 0002:00:00.4
[    0.264958] AMD-Vi: ivrs, add hid:AMDI0096, uid:ID02, rdevid:131072
[    0.264959] AMD-Vi: AMD-Vi: device: 0002:80:00.2 cap: 0040 flags: 30 info 0000
[    0.264960] AMD-Vi: AMD-Vi:        mmio-addr: 00000000b3000000
[    0.264967] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0002:80:00.3 flags: 00
[    0.264967] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0002:bf:1f.6
[    0.265094] AMD-Vi: AMD-Vi:   DEV_SELECT_RANGE_START  devid: 0002:40:00.3 flags: 00
[    0.265094] AMD-Vi: AMD-Vi:   DEV_RANGE_END           devid: 0002:7f:1f.6
[    0.265221] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[251])               devid: 0002:80:00.1
[    0.265222] AMD-Vi: AMD-Vi:   DEV_SPECIAL(IOAPIC[255])               devid: 0002:40:00.1
[    0.992850] pci 0001:c0:00.2: AMD-Vi: IOMMU performance counters supported
[    0.992865] pci 0001:40:00.2: AMD-Vi: IOMMU performance counters supported
[    0.992892] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.992912] pci 0000:80:00.2: AMD-Vi: IOMMU performance counters supported
[    0.992934] pci 0003:c0:00.2: AMD-Vi: IOMMU performance counters supported
[    0.992956] pci 0003:40:00.2: AMD-Vi: IOMMU performance counters supported
[    0.992974] pci 0002:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.992997] pci 0002:80:00.2: AMD-Vi: IOMMU performance counters supported
[    1.037825] pci 0001:c0:00.2: AMD-Vi: Found IOMMU cap 0x40
[    1.037827] AMD-Vi: Extended features (0xa5bf732fa2295afe): PPR X2APIC NX GT [5] IA GA PC GA_vAPIC
[    1.037833] pci 0001:40:00.2: AMD-Vi: Found IOMMU cap 0x40
[    1.037834] AMD-Vi: Extended features (0xa5bf732fa2295afe): PPR X2APIC NX GT [5] IA GA PC GA_vAPIC
[    1.037836] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    1.037837] AMD-Vi: Extended features (0xa5bf732fa2295afe): PPR X2APIC NX GT [5] IA GA PC GA_vAPIC
[    1.037839] pci 0000:80:00.2: AMD-Vi: Found IOMMU cap 0x40
[    1.037840] AMD-Vi: Extended features (0xa5bf732fa2295afe): PPR X2APIC NX GT [5] IA GA PC GA_vAPIC
[    1.037842] pci 0003:c0:00.2: AMD-Vi: Found IOMMU cap 0x40
[    1.037843] AMD-Vi: Extended features (0xa5bf732fa2295afe): PPR X2APIC NX GT [5] IA GA PC GA_vAPIC
[    1.037846] pci 0003:40:00.2: AMD-Vi: Found IOMMU cap 0x40
[    1.037846] AMD-Vi: Extended features (0xa5bf732fa2295afe): PPR X2APIC NX GT [5] IA GA PC GA_vAPIC
[    1.037849] pci 0002:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    1.037849] AMD-Vi: Extended features (0xa5bf732fa2295afe): PPR X2APIC NX GT [5] IA GA PC GA_vAPIC
[    1.037852] pci 0002:80:00.2: AMD-Vi: Found IOMMU cap 0x40
[    1.037852] AMD-Vi: Extended features (0xa5bf732fa2295afe): PPR X2APIC NX GT [5] IA GA PC GA_vAPIC
[    1.037855] AMD-Vi: Interrupt remapping enabled
[    1.037855] AMD-Vi: Virtual APIC enabled
[    1.037855] AMD-Vi: X2APIC enabled
[    1.273843] AMD-Vi: AMD IOMMUv2 loaded and initialized
  • This message is generated by the following patch:
amd@volcano9dee-host:~/iommu_fin$ grep -nr "DUMP_printk"
0032-iommu-amd-Print-PCI-segment-ID-in-error-log-messages.patch:28:-                    DUMP_printk("device: %02x:%02x.%01x cap: %04x "
0032-iommu-amd-Print-PCI-segment-ID-in-error-log-messages.patch:33:+                    DUMP_printk("device: %04x:%02x:%02x.%01x cap: %04x "
0032-iommu-amd-Print-PCI-segment-ID-in-error-log-messages.patch:38:                     DUMP_printk("       mmio-addr: %016llx\n",
0013-iommu-amd-Introduce-per-PCI-segment-rlookup-table-si.patch:78:     DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf);
0009-iommu-amd-Introduce-per-PCI-segment-unity-map-list.patch:127:-     DUMP_printk("%s devid_start: %02x:%02x.%x devid_end: %02x:%02x.%x"
0009-iommu-amd-Introduce-per-PCI-segment-unity-map-list.patch:129:+     DUMP_printk("%s devid_start: %04x:%02x:%02x.%x devid_end: "
0027-iommu-amd-Remove-global-amd_iommu_-dev_table-alias_t.patch:187:                    DUMP_printk("  DEV_ALL\t\t\tflags: %02x\n", e->flags);
0027-iommu-amd-Remove-global-amd_iommu_-dev_table-alias_t.patch:197:    DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf);
0027-iommu-amd-Remove-global-amd_iommu_-dev_table-alias_t.patch:260:    DUMP_printk("Using IVHD type %#x\n", amd_iommu_target_ivhd_type);
0012-iommu-amd-Introduce-per-PCI-segment-alias-table-size.patch:58:     DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf);
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:105:-                   DUMP_printk("  DEV_SELECT\t\t\t devid: %02x:%02x.%x "
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:106:+                   DUMP_printk("  DEV_SELECT\t\t\t devid: %04x:%02x:%02x.%x "
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:116:                    DUMP_printk("  DEV_SELECT_RANGE_START\t "
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:128:-                   DUMP_printk("  DEV_ALIAS\t\t\t devid: %02x:%02x.%x "
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:129:+                   DUMP_printk("  DEV_ALIAS\t\t\t devid: %04x:%02x:%02x.%x "
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:139:                    DUMP_printk("  DEV_ALIAS_RANGE\t\t "
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:158:-                   DUMP_printk("  DEV_EXT_SELECT\t\t devid: %02x:%02x.%x "
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:159:+                   DUMP_printk("  DEV_EXT_SELECT\t\t devid: %04x:%02x:%02x.%x "
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:169:                    DUMP_printk("  DEV_EXT_SELECT_RANGE\t devid: "
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:181:-                   DUMP_printk("  DEV_RANGE_END\t\t devid: %02x:%02x.%x\n",
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:183:+                   DUMP_printk("  DEV_RANGE_END\t\t devid: %04x:%02x:%02x.%x\n",
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:206:-                   DUMP_printk("  DEV_SPECIAL(%s[%d])\t\tdevid: %02x:%02x.%x\n",
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:207:+                   DUMP_printk("  DEV_SPECIAL(%s[%d])\t\tdevid: %04x:%02x:%02x.%x\n",
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:228:-                   DUMP_printk("  DEV_ACPI_HID(%s[%s])\t\tdevid: %02x:%02x.%x\n",
0030-iommu-amd-Include-PCI-segment-ID-when-initialize-IOM.patch:231:+                   DUMP_printk("  DEV_ACPI_HID(%s[%s])\t\tdevid: %04x:%02x:%02x.%x\n",
0011-iommu-amd-Introduce-per-PCI-segment-device-table-siz.patch:98:     DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf);
0010-iommu-amd-Introduce-per-PCI-segment-last_bdf.patch:143:+   DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf);
0010-iommu-amd-Introduce-per-PCI-segment-last_bdf.patch:220:    DUMP_printk("Using IVHD type %#x\n", amd_iommu_target_ivhd_type);

I have also tested using the command:

lspci -tv // This gives a tree view of the PCI devices.

Additionally, I used the following command to inspect the devices associated with IOMMU:

readlink -f /sys/class/iommu/*/device
The results of this command show the paths to the devices correctly assigned to each IOMMU group:
/sys/devices/pci0001:c0/0001:c0:00.2
/sys/devices/pci0001:40/0001:40:00.2
/sys/devices/pci0000:00/0000:00:00.2
/sys/devices/pci0000:80/0000:80:00.2
/sys/devices/pci0003:c0/0003:c0:00.2
/sys/devices/pci0003:40/0003:40:00.2
/sys/devices/pci0002:00/0002:00:00.2
/sys/devices/pci0002:80/0002:80:00.2
  • Created File system on different disks & done basic dd command runs

  • Disk PCI passthrough to a VM. Created file system & tested with basic dd command.

  • Note 1:
    BIOS → Advanced → AMD CBS → DF Common Options → Number of PCI Segments = 4 or 2
    Without backported patches, the system won't boot.

  • Note 2:
    BIOS → Advanced → AMD CBS → DF Common Options → Number of PCI Segments = Auto or 1
    System boots fine with or without backported patches.

hegdevasant and others added 30 commits June 11, 2025 15:00
commit d02674d upstream.

struct iommu_dev_data contains member "pdev" to point to pci_dev. This is
valid for only PCI devices and for other devices this will be NULL. This
causes unnecessary "pdev != NULL" check at various places.

Replace "struct pci_dev" member with "struct device" and use to_pci_dev()
to get pci device reference as needed. Also adjust setup_aliases() and
clone_aliases() function.

No functional change intended.

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-2-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Malathi <amalathi@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 404ec4e upstream.

Newer AMD systems can support multiple PCI segments, where each segment
contains one or more IOMMU instances. However, an IOMMU instance can only
support a single PCI segment.

Current code assumes that system contains only one pci segment (segment 0)
and creates global data structures such as device table, rlookup table,
etc.

Introducing per PCI segment data structure, which contains segment
specific data structures. This will eventually replace the global
data structures.

Also update `amd_iommu->pci_seg` variable to point to PCI segment
structure instead of PCI segment ID.

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-3-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Malathi <amalathi@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 04230c1 upstream.

Introduce per PCI segment device table. All IOMMUs within the segment
will share this device table. This will replace global device
table i.e. amd_iommu_dev_table.

Also introduce helper function to get the device table for the given IOMMU.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-4-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Malathi <amalathi@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit eda797a upstream.

This will replace global rlookup table (amd_iommu_rlookup_table).
Add helper functions to set/get rlookup table for the given device.
Also add macros to get seg/devid from sbdf.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-5-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Malathi <amalathi@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 333e581 upstream.

This will replace global irq lookup table (irq_lookup_table).

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-6-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Malathi <amalathi@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 39a303b upstream.

This will replace global dev_data_list.

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-7-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit eb21ef0 upstream.

It will remove global old_dev_tbl_cpy. Also update copy_device_table()
copy device table for all PCI segments.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-8-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 99fc4ac upstream.

This will replace global alias table (amd_iommu_alias_table).

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-9-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit b618ae6 upstream.

Newer AMD systems can support multiple PCI segments. In order to support
multiple PCI segments IVMD table in IVRS structure is enhanced to
include pci segment id. Update ivmd_header structure to include "pci_seg".

Also introduce per PCI segment unity map list. It will replace global
amd_iommu_unity_map list.

Note that we have used "reserved" field in IVMD table to include "pci_seg
id" which was set to zero. It will take care of backward compatibility
(new kernel will work fine on older systems).

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-10-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 3079590 upstream.

Current code uses global "amd_iommu_last_bdf" to track the last bdf
supported by the system. This value is used for various memory
allocation, device data flushing, etc.

Introduce per PCI segment last_bdf which will be used to track last bdf
supported by the given PCI segment and use this value for all per
segment memory allocations. Eventually it will replace global
"amd_iommu_last_bdf".

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-11-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit b5c8529 upstream

With multiple pci segment support, number of BDF supported by each
segment may differ. Hence introduce per segment device table size
which depends on last_bdf. This will replace global
"device_table_size" variable.

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-12-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: chaithanyaLagisetty <LagisettyVenkata.NagaChaitanya@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 74ce42a upstream.

It will replace global "alias_table_size" variable.

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-13-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: chaithanyaLagisetty <LagisettyVenkata.NagaChaitanya@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit ec12dd1 upstream.

It will replace global "rlookup_table_size" variable.

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-14-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: chaithanyaLagisetty <LagisettyVenkata.NagaChaitanya@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 0217ed5 upstream.

Then, remove the global irq_lookup_table.

[Backport change]
In file `drivers/iommu/amd/iommu.c`, in function `amd_iommu_update_ga()`,
the call get_irq_table(iommu, devid) was already removed by commit 0a27c66,
so this change is not required.

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-15-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: chaithanyaLagisetty <LagisettyVenkata.NagaChaitanya@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 8b71c9b upstream.

Use rlookup_amd_iommu() helper function which will give per PCI
segment rlookup_table.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-16-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: chaithanyaLagisetty <LagisettyVenkata.NagaChaitanya@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
…tion

commit 9873ae6 upstream.

To allow IOMMU rlookup using both PCI segment and device ID.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-17-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Arukonda Rahul <arukonda.rahul@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 9457d75 upstream.

Add a pointer to struct amd_iommu to amd_ir_data structure, which
can be used to correlate interrupt remapping data to a per-PCI-segment
interrupt remapping table.

[Backport change]

This patch triggers a compilation warning due to the unused variable
int devid = ir_data->irq_2_irte.devid;. Although commit 0a27c66
(already backported to the kernel) was supposed to remove it, the line
wasn't removed. To avoid compilation warning, it has now been removed.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-18-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Arukonda Rahul <arukonda.rahul@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit c4649a4 upstream.

Pass amd_iommu structure as one of the parameter to amd_irte_ops functions
since its needed to activate/deactivate the iommu.

[Backport change]

The commits c0d05db and 13809bd, which introduced the
name change for the __modify_irte_ga and modify_irte_ga APIs in the
current kernel, are more recent than this upstream patch. Therefore,
we are retaining the names __modify_irte_ga and modify_irte_ga as
they are, and only updating the argument types as per the upstream patch.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-19-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Arukonda Rahul <arukonda.rahul@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit e6457d7 upstream.

Pass amd_iommu structure as one of the parameter to these functions
as its needed to retrieve variable tables inside these functions.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-20-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Arukonda Rahul <arukonda.rahul@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit ccacd94 upstream.

Then, remove the global amd_iommu_rlookup_table and rlookup_table_size.

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-21-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Arukonda Rahul <arukonda.rahul@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 54625ef upstream.

Start using per PCI segment data structures instead of global data
structures.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-22-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Malathi <amalathi@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit ccbb091 upstream.

Start using per PCI segment device table instead of global
device table.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-23-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Malathi <amalathi@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 4cc053d upstream.

Start using per PCI segment device table instead of global
device table.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-24-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Malathi <amalathi@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit c7d3112 upstream.

Start using per PCI segment device table instead of global
device table.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-25-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Malathi <amalathi@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 1ab5a15 upstream.

Include struct amd_iommu_pci_seg as a function parameter since
we need to access per PCI segment device table.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-26-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Malathi <amalathi@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 56fb795 upstream.

To include a pointer to per PCI segment device table.

Also include struct amd_iommu as one of the function parameter to
amd_iommu_apply_erratum_63() since it is needed when setting up DTE.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-27-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: chaithanyaLagisetty <LagisettyVenkata.NagaChaitanya@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 401360e upstream.

Replace them with per PCI segment device table.
Also remove dev_table_size, alias_table_size, amd_iommu_last_bdf
variables.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-28-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: chaithanyaLagisetty <LagisettyVenkata.NagaChaitanya@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit a3cf6ab upstream.

Fix amd_iommu_flush_dte_all() and amd_iommu_flush_tlb_all() to flush
upto last_bdf only.

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-29-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: chaithanyaLagisetty <LagisettyVenkata.NagaChaitanya@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit bf87972 upstream.

Current get_device_id() only provide 16-bit PCI device ID (i.e. BDF).
With multiple PCI segment support, we need to extend the helper function
to include PCI segment ID.

So, introduce a new helper function get_device_sbdf_id() to replace
the current get_pci_device_id().

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-30-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: chaithanyaLagisetty <LagisettyVenkata.NagaChaitanya@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit a45627b upstream.

Extend current device ID variables to 32-bit to include the 16-bit
segment ID when parsing device information from IVRS table to initialize
each IOMMU.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-31-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: chaithanyaLagisetty <LagisettyVenkata.NagaChaitanya@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
ssuthiku-amd and others added 4 commits June 11, 2025 15:00
commit e5670e1 upstream.

Upcoming AMD systems can have multiple PCI segments. Hence pass PCI
segment ID to pci_get_domain_bus_and_slot() instead of '0'.

Co-developed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-32-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Arukonda Rahul <arukonda.rahul@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit b36a5b0 upstream.

Print pci segment ID along with bdf. Useful for debugging.

Co-developed-by: Suravee Suthikulpaint <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpaint <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-34-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Arukonda Rahul <arukonda.rahul@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 196dff7 upstream.

Rename struct device_state.devid variable to struct device_state.sbdf
and extend it to 32-bit to include the 16-bit PCI segment ID via
the helper function get_pci_sbdf_id().

Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-35-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Arukonda Rahul <arukonda.rahul@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
commit 214a05c upstream.

Rename 'device_id' as 'sbdf' and extend it to 32bit so that we can
pass PCI segment ID to ppr_notifier(). Also pass PCI segment ID to
pci_get_domain_bus_and_slot() instead of default value.

Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Link: https://lore.kernel.org/r/20220706113825.25582-36-vasant.hegde@amd.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Arukonda Rahul <arukonda.rahul@amd.com>
Signed-off-by: PvsNarasimha <PVS.NarasimhaRao@amd.com>
@guojinhui-liam
Copy link
Collaborator

LGTM.

quanxianwang pushed a commit to quanxianwang/velinux-kernel that referenced this pull request Sep 28, 2025
commit ab1de7e upstream.

The commit 4f7e723 ("cgroup: Fix threadgroup_rwsem <-> cpus_read_lock()
deadlock") fixed the deadlock between cgroup_threadgroup_rwsem and
cpus_read_lock() by introducing cgroup_attach_{lock,unlock}() and removing
cpus_read_{lock,unlock}() from cpuset_attach(). But cgroup_transfer_tasks()
was missed and not handled, which will cause th following warning:

 WARNING: CPU: 0 PID: 589 at kernel/cpu.c:526 lockdep_assert_cpus_held+0x32/0x40
 CPU: 0 PID: 589 Comm: kworker/1:4 Not tainted 6.4.0-rc2-next-20230517 openvelinux#50
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
 Workqueue: events cpuset_hotplug_workfn
 RIP: 0010:lockdep_assert_cpus_held+0x32/0x40
 <...>
 Call Trace:
  <TASK>
  cpuset_attach+0x40/0x240
  cgroup_migrate_execute+0x452/0x5e0
  ? _raw_spin_unlock_irq+0x28/0x40
  cgroup_transfer_tasks+0x1f3/0x360
  ? find_held_lock+0x32/0x90
  ? cpuset_hotplug_workfn+0xc81/0xed0
  cpuset_hotplug_workfn+0xcb1/0xed0
  ? process_one_work+0x248/0x5b0
  process_one_work+0x2b9/0x5b0
  worker_thread+0x56/0x3b0
  ? process_one_work+0x5b0/0x5b0
  kthread+0xf1/0x120
  ? kthread_complete_and_exit+0x20/0x20
  ret_from_fork+0x1f/0x30
  </TASK>

So just use the cgroup_attach_{lock,unlock}() helper to fix it.

Reported-by: Zhao Gongyi <zhaogongyi@bytedance.com>
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
Acked-by: Muchun Song <songmuchun@bytedance.com>
Fixes: 05c7b7a ("cgroup/cpuset: Fix a race between cpuset_attach() and cpu hotplug")
Cc: stable@vger.kernel.org # v5.17+
Signed-off-by: Tejun Heo <tj@kernel.org>
@guojinhui-liam
Copy link
Collaborator

The pull request is meged in 5.15-velinux locally, so here close it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants