pci screamer & pcileech, tracing completion TLP

[raghu-elastics]

Procured pci screamer, I am trying to understand the working of the screamer card to trace PCIe TLP packets using pcileech tool, but not able to see any dump of completion TLP after invoking memory read using following command.

"pcileech.exe tlp -vvv -in 000000203f0080ff001ad000"

As I understand the pcieleech pushes the Read TLP onto the screamer and screamer responds back with TLP and that TLP is captured on usb serial port.

Is my assumption wrong?

[ufrisk]

The probe command is super aggressive and it's likely that DMA will stop working until you restart your computer (or boot it from power-off state) again. Don't use it unless you absolutely have to (especially if on an AMD system). https://github.com/ufrisk/LeechCore/wiki/Device_FPGA_AMD_Thunderbolt In the above you see that all reads in your probe failed.

After a fresh boot try running pcileech.exe display -min 0x1000 -device fpga -vvv to see TLPs.

If you have a success then try the TLP command. pcileech.exe tlp -device fpga -vvv -in ...

Another issue may be that you may have the wrong PCIe ReqID (3f00) in your TLP. But I'm unable to judge this, you'll see it very clearly in the display example above though.

Please let me know if you have a better success with this.

[raghu-elastics]

Thanks @ufrisk for response.

Yes the ReqID was wrong earlier(attached the screenshot of BDF), I changed the requester TLP to have correct one and tried, but its the same issue.
Attached the screenshot

VT-d is disabled in BIOS

And another question I wanted to ask is this tool compatible with Linux env, do you want to check any piece of code for more info in pcileech & leechcore?

[ufrisk]

it should work on linux as well, it's found in the releases section.

this might actually be a good idea since you may get some good info about whats up if you do a dmesg (if DMA is blocked and the device tries to read).

In your example PCILeech have identified ReqID 0200 (and it fail to get an answer for some unknown reason) and you specify ReqID 0100 (which is wrong? and you would never get an answer back that way).

[raghu-elastics]

Screamer card bdf is 01:00.0 and completion TLP for single TLP read is received

$ sudo ./pcileech tlp -vvv -in 00000020010080ff001ae000

[+] using FTDI device: 0403:601f (bus 1, device 3)
[+] FTDIFTDI SuperSpeed-FIFO Bridge000000000001
TLP: Transmitting PCIe TLP. (use -vvv option for detailed info).

TX: MRd32: Len: 020 ReqID: 0100 BE_FL: ff Tag: 80 Addr: 001ae000
0000 00 00 00 20 01 00 80 ff 00 1a e0 00 ... ........

RX: CplD: Len: 020 ReqID: 0100 CplID: 0000 Status: 0 BC: 080 Tag: 80 LowAddr: 00
0000 4a 00 00 20 00 00 00 80 01 00 80 00 b1 b7 b7 18 J.. ............
0010 e1 be 2e 28 5b c7 5d 68 0b ce c4 58 73 5d 75 f2 ...([.]h...Xs]u.
0020 23 54 ec c2 e0 3d e6 92 b0 34 7f a2 ed 19 eb b6 #T...=...4 .....
0030 bd 10 72 86 0b ed 0d 42 5b e4 94 72 79 5a 7f f5 ..r....B[..ryZ .
0040 29 53 e6 c5 e8 b0 ee 1f b8 b9 77 2f 44 11 42 be )S........w/D.B.
0050 14 18 db 8e 2a 85 2c 2a 7a 8c b5 1a a4 89 a2 26 ....*.,*z......&
0060 f4 80 3b 16 2b 47 2d e8 7b 4e b4 d8 02 b7 04 18 ..;.+G-.{N......
0070 52 be 9d 28 35 16 33 b9 65 1f aa 89 4a bb 4c 14 R..(5.3.e...J.L.
0080 1a b2 d5 24 fb fa fd 55 ab f3 64 65 ...$...U..de
[root@util-04 files]#

I have requirement for continuously capturing completion TLP generated by screamer, kind of daemon mode
I checked the options, tried

$ sudo ./pcileech tlp -vvv -tlpwait 60

[+] using FTDI device: 0403:601f (bus 1, device 3)
[+] FTDIFTDI SuperSpeed-FIFO Bridge000000000001
Action_TlpTx: Invalid TLP (too short).

2. Tried some random setpci command to generate config TLP towards screamer, but no luck, the above tlpwait option doesnt capture the pcileech,
   I see the FTrace for setpci generating pci_config_* traces, so there is some read/write activity when I do setpci, but no TLP's generation on screamer
   is there option to daemonize and continuously capture TLPs on screamer?

[ufrisk]

There are two things at play here,

1. The Xilinx PCIe core filters out TLPs which reach the core parts of the config space related to the PCIe core itself. It's usually 0x0-0x80 something in the config space. I haven't figured out a way to receive them, but it may be possible if you try to edit the auto-generated Xilinx PCIe IP core files somehow.

2. The PCILeech firmware filters out configuration packets since these may fill up internal buffers in rare cases. You may either compile a new bitstream after you changed this constant: https://github.com/ufrisk/pcileech-fpga/blob/9a21b4d31e2afd7fea2aaba1d8306c662eab9ebf/PCIeSquirrel/src/pcileech_fifo.sv#L269 or by using the LeechCore API to set this bit to zero before every use of the device when you wish to receive config space TLPs (in space not controlled by the Xilinx PCIe IP core in (1)).

---

As far as a daemon goes, I haven't implemented anything like that in the pcileech binary, but it should be possible to do by making use of the LeechCore API - which is the backend of PCILeech. I also have a Python PIP package for LeechCore which will allow you to receive/send TLPs.

BARs are not implemented though, but if you receive TLPs I guess you should be able to implement them in software - just that the latency will be terrible coz of the USB.

[raghu-elastics]

thanks @ufrisk , I will try and go through the code and get back to you!

[ufrisk]

I'm closing the issue since I never heard anything back. I'm assuming things resolved itself 👍