-
Notifications
You must be signed in to change notification settings - Fork 129
PCIE Programming
In the projects of NetFPGA-10G, Reference Flash shows configuration of hardware and implementation procedure for programming FPGA with a bitstream stored in Flash memories. Here, this project is to implement PCIE programming enabled hardware for NetFPGA-10G projects such as Reference NIC and Reference Switch. The current projects with PCIE programming allows users programing FPGA with a bitstream data stored in Flash memories loaded through PCIE endpoint incorporated in the FPGA. The implemented hardware can be used not only to save binary image into Flash memories, but also to load the binary image to be programmed into FPGA.
For PCIE programming, Dip SW3 of the NetFPGA-10g platform needs to be changed from default settings ( M0:On, M1:On, M2:On) to PCIE programming settings (M0:Off, M1:On, M2:On). This FPGA configuration is described in Xilinx Virtex 5 Configuration Guide. Figure on below shows place of the Dip SW3 and setting values for necessary configuration. More detailed information about the SW setting can be found in Production manual. The PCIE programming setting enables both JTAG and PCIE interface to program FPGA. ![Alt text](pcie_programming_1.png "Test image" "size=100x100pixels")
As an example, figure on below shows PCIE programming hardware configuration of the Reference NIC incorporating external memory controller (EMC) for bitstream data transaction and GPIO for the transaction control. In the hardware, while the EMC transfers the data to Current projects such as reference nic, reference switch, etc need additional resources for PCIE programming. As described in a project of reference flash, external memory controller, gpio, cpld, and flash memories are required by PCIE programming HW implementation. An external memory controller is used to transfer binary data to a flash memory from a host PC. A signal of gpio is connected to CPLD from FPGA, which is used to select which image stored in flash memories is loaded for FPGA programming through CPLD. CPLD controls data transfer from and to FPGA and FLASH memories.
![Alt text](pcie_programming_2.png "Test image" "size=100x100pixels")
As described, this PCIE programming project is to enable current projects to program FPGA with bitstream in Flash memories. Thus, current projects (Refere Nic, Reference Switch, Reference Router, Reference Switch Lite) have to be implemented with EMC and GPIO as illustrated the figure on above. CPLD is also required to transact and control bitstream data. On below, PCIE programming enabled hardware implementation procedure is described step by step. As an example procedure, the reference nic and reference switch projects are used as an example of PCIE programming enabled hardware implementation. It is assumed that the Dip SW3 is set for PCIE programming from default setting.
- Bit image preparation of Reference Nic project.
Go to ./NetFPGA-10G-live/projects/reference_nic/
make
N.B. ‘make’ without any argument implements hardware as before.
- After completing bit file generation, convert a bit image to a binary bitstream.
Go to ./ NetFPGA-10G-live/projects/reference_nic/bitfiles/
./bit2bin.sh <bit_file_path_and_name>
- Bit image preparation of Reference Switch project.
Go to ./NetFPGA-10G-live/projects/reference_switch/
make
N.B. ‘make’ without any arguments implements hardware without PCIE programming function.
- After completing bit file generation, convert a bit image to a binary bitstream.
Go to ./ NetFPGA-10G-live/projects/reference_nic/bitfiles/
./bit2bin.sh <bit_file_path_and_name>
- Create Cpld image.
Go to ./ NetFPGA-10G-live/projects/reference_nic/cpld/
make
Find out cpld.jed file. This cpld image file can be used other project for PCIE programming.
- FPGA and CPLD programming via JTAG for initial configuration.
Go to ./NetFPGA-10G-live//tools/scripts/
./impact_run.sh <bit_file_path_and_name> <cpld_file_path_name>
e.g.) ./impact_run.sh ../../projects/reference_nic/bitfiles/reference_nic.bit ../../projects/reference_nic/cpld/cpld.jed```
-
Reboot a host PC. Once the PCIE programming hardware implementation is succeeded, rebooting PC or power-cycle process can be avoided.
-
Find out PCIE configuration
lspci -d *:4244 -vxx
You should see NetFPGA-10G Xilinx PCIE configuration information from this command.
-
Remove nf10.ko kernel driver (if it is loaded) and load nf10.ko kernel driver.
rmmod nf10.ko
insmod ./NetFPGA-10G-live/projects/reference_nic/sw/host/driver/nf10.ko
A user may need to compile before loading the kernel driver.
- Transfer binary data to flash memories.
Go to ./NetFPGA-10G-live/ projects/reference_nic/sw/host/pcieprog/
make
Transfer bitstream to flash a
./nf10_configure -b <bin_file_path_name> -f a
./nf10_configure -b /root/TestBitfiles/reference_nic.bin -f a
This command shows message on below.
Flash image: /root/TestBitfiles/reference_nic.bin
Manufacturer ID: 49
Programming flash 'a'.
[........................................................
Transfer bitstream to flash b
./nf10_configure -b <bin_file_path_name> -f b
./nf10_configure -b /root/TestBitfiles/reference_nic.bin -f b
This message shows on below.
Flash image: /root/TestBitfiles/reference_nic.bin
Manufacturer ID: 49
Programming flash 'b'.
[..........................................................................
-
Reboot to find out an image loaded from Flash a. Find out PCIE configuration
lspci –d *:4244 –vxx
You should see PCIE configuration information from this command.
- User can select Flash A or Flash B.
Current image will be an image loaded from Flash A. Now, configure FPGA with Flash B
Go to ./NetFPGA-10G-live/ projects/reference_nic/sw/host/pcieprog/
./nf10_flash_b
Reconfigure PCIE configuration
./NetFPGA-10G-live/tools/scripts/pci_rescan_run.sh
Remove and load nf10.ko kernel driver.
rmmod nf10.ko
insmod nf10.ko
Then run dmesg and verify the image details
Configure back to Flash B
Go to ./NetFPGA-10G-live/ projects/reference_nic/sw/host/pcieprog/
./nf10_flash_b
Reconfigure PCIE configuration
./NetFPGA-10G-live/tools/scripts/pci_rescan_run.sh
Remove and load nf10.ko kernel driver.
rmmod nf10.ko
insmod nf10.ko
Then run dmesg and verify the image details