Test 841 requires memory under 2^32?

[hrw]

On SBSA-ref there is no memory below 2^40 address.

Test 841 wants memory in 32-bit address space:

 841 : NP type-1 PCIe supp 32-bit only                START

       BDF - 0x300
       BDF - 0x400
       BDF - 0x500
       BDF - 0x600
       BDF - 0x700
       BDF - 0x800
       BDF - 0x10000
       BDF - 0x20000
       BDF - 0x40000
       Skipping Non Type-1 headers
       BDF - 0x50000
       Skipping Non Type-1 headers
       BDF - 0x60000
       Skipping Non Type-1 headers
       BDF - 0x80000
       NP type-1 pcie is not 32-bit mem type
       Failed on PE -    0
       PCI_MM_04
       Checkpoint --  1                           : Result:  FAIL
         END

Doc says:

Systems compliant to this specification must support 32-bit programming of NP BARs on such
endpoints. This can be achieved in two ways:

Method 1: PE physical address space can be reserved below 4GB, while maintaining a one-to-one mapping between PE physical address space and NP memory address space.

Method 2: It is also possible to use a fixed offset translation scheme that creates a fixed offset in direction between PE physical address space, and PCI memory. This allows a window in PE physical address space that is above 4G to be mirrored in PCI memory space below 4G. This requires support in the PHB. Furthermore, firmware must program the PHB with the fixed offset, and to supply this information to the OS.

Looks like we would need to go with method 2 then?

[chetan-rathore]

Hi @hrw,

We are checking the query and provide update soon.

Thanks,
ACS team

[chetan-rathore]

Hello @hrw,

Can you help us with below data to debug the issue.

1. Qemu model and Qemu command
2. dump of "pci 08 00 00 -i" command on uefi shell

Thanks,
ACS team

[hrw]

Log with:

• exact QEMU (HEAD) command line
• UEFI "pci" commmand
• UEFI "pci 00 08 00 -i" command

log-pci.txt

I use QEMU master HEAD with UEFI built also with HEAD of edk2 and edk2-platforms repos. Can attach firmware image if needed.

[chetan-rathore]

Hello @hrw,

The failure was seen for 0x80000 device ( Bus 8 Dev 0 Func 0)

BDF - 0x80000
NP type-1 pcie is not 32-bit mem type
Failed on PE - 0

But the pci dump command list no such device, the only device at bus 8 is
.00 08 09 00 ==> .Multimedia Device - Audio device
Vendor 1274 Device 5000 Prog Interface 0

In case if same setup of QEMU was used for running BSA test and sharing pci cmd output, can you please attach bsa log with below command, to check PCIe devices found by BSA suite.

Bsa.efi -t 841 -v 1

Thanks,
ACS team

[hrw]

Here is output of ./boot-sbsa-ref.sh --cmd="fs0:bsa.efi -t 841 -v 1" command:

log-841.txt

(boot-sbsa-ref.sh is in https://github.com/hrw/sbsa-ref-status repo)

[hrw]

Thanks for looking into it.

[chetan-rathore]

Hello @hrw,

The support of ITS and SMMU + booting qemu with number of different PCIe devices is great. :clap

On the failure, the PCIe device on which test is failing is a "PCI Express to PCI/PCI-X Bridge"

Pci Express device capability structure:
CapID( 0): 10 NextCap Ptr( 1): 40
Cap Register( 2): 0072
Capability Version(3:0): 0x0002
Device/PortType(7:4): PCI Express to PCI/PCI-X Bridge

It is a Type1 header device, but is requesting 64-bit NP memory.

Shell> pci 07 00 00 -i
PCI Segment 00 Bus 07 Device 00 Func 00 [EFI 0007000000]
00000000: 36 1B 0E 00 07 00 B0 00-00 00 04 06 00 00 01 00 6...............
00000010: 04 00 00 81 00 00 00 00-07 08 08 00 00 00 A0 00 ................

Bits 0 -> 0 // memory address requested
Bits 21 -> 10 // 64-bit address
Bits 3 -> 0 // NP memory

As per BSA and PCIe spec, Type1 headers supports 32-bit addresses

I For non-prefetchable (NP) memory, Type 1 headers only support 32-bit addresses. This implies that endpoints
on the other end of a PCI-to-PCI bridge only support 32-bit NP BARs.
RPCI_MM_04 Systems compliant to this specification must support 32-bit programming of NP BARs on such endpoints.
This can be achieved in two ways:

Thanks,
ACS team

[hrw]

I mailed QEMU devel ML:

https://lore.kernel.org/qemu-devel/20230911062751-mutt-send-email-mst@kernel.org/T/

Please check the answer.

[chetan-rathore]

Hi @hrw,

We are having an internal discussion around the PCIe prefetchable bits and type (32/64-bit), will keep you updated.

Thanks,
ACS team

[hrw]

I have Arm support case 03445990 opened for it as well.

[chetan-rathore]

Hi @hrw,

The requirement of a PCIe device requesting NP memory should use 32-bit address only (32-bit BAR) is applicable only for EP.
The test will be updated to skip the check for Type1 headers.

Thanks,
ACS team

[chetan-rathore]

Hi @hrw,

Further updates on top of the last one.

As RP can only support 32-bit address for NP memory living downstream of a Type-1 header, EP which are downstream to the RP for NP memory should only have 32-bit address programmed in the BAR's.

But PCIe specification doesn't mandate BAR Type Bit to be 32-bit for NP memory. It can be 64-bit but the address programmed should be 32-bit address (below 4GB). In case system is mapping PCI memory in above 4GB range, it can use address translation offset.

RPCI_MM_04 Systems compliant to this specification must support 32-bit programming of NP BARs on such endpoints.
This can be achieved in two ways:
Method 1: PE physical address space can be reserved below 4GB, whilst maintaining a one to one mapping
between PE physical address space and NP memory address space.
Method 2: It is also possible to use a fixed offset translation scheme that creates a fixed offset indirection
between PE physical address space, and PCI memory. This allows a window in PE physical address
space that is above 4G to be mirrored in PCI memory space below 4G. This requires support in the PHB.
Furthermore, firmware must program the PHB with the fixed offset, and to supply this information to the
OS [4].

Based on above inputs, there is no FAIL case from BSA specification perspective for current test. In case NP BAR is programmed with 64-bit address, it will be PCIe enumeration software issue but not hardware issue.

We will deprecate the test case.

Thanks,
ACS team.

[hrw]

Thanks