Example design for the Quad Gigabit Ethernet FMC using an FPGA based hardware packet generator/checker to demonstrate maximum throughput.
This project is designed for Vivado 2018.2. If you are using an older version of Vivado, then you MUST use an older version of this repository. Refer to the list of commits to find links to the older versions of this repository.
In order to test the Ethernet FMC using this design, you need to use an Ethernet cable to loopback ports 0 and 2, and ports 1 and 3. You will also need the following:
- Vivado 2018.2
- Vivado HLS 2018.2
- Ethernet FMC
- Supported FMC carrier board (see list of supported carriers above)
- Two Ethernet cables
- Xilinx Soft TEMAC license
Supported carrier boards
- Zynq-7000 ZedBoard
- LPC connector (use zedboard.xdc)
- Zynq-7000 MicroZed 7Z010 and 7Z020 with MicroZed FMC Carrier
- LPC connector (use mzfmc-7z010-7z020.xdc)
- Zynq-7000 PicoZed 7Z015, 7Z020 and 7Z030 with PicoZed FMC Carrier Card V2
- LPC connector (use pzfmc-7z015.xdc, or pzfmc-7z020.xdc, or pzfmc-7z030.xdc)
- Kintex-7 KC705 Evaluation board
- LPC connector (use kc705-lpc.xdc)
- HPC connector (use kc705-hpc.xdc)
- Zynq UltraScale+ ZCU102 Evaluation board Rev 1.0
- HPC0 connector (use zcu102-hpc0.xdc)
Note about 7Z010 devices:
The designs for the 7Z010 device differ slightly from the main design, using less resources to allow them to fit within the resource constraints of the smaller device. These designs use only 2 traffic generators, connected to ports 0 and 1, while ports 2 and 3 are connected to AXI Ethernet Subsystem blocks that are looped back on themselves. In this configuration, we can still test all ports at maximum throughput and detect dropped frames on all 4 ports, however we lose the ability to detect bit errors on two of the ports.
When testing these designs, note that we cannot force bit errors on ports 2 and 3, because they are connected in loopback; hence you can expect the dropped frame counters of ports 0 and 1 to always read 0 unless frames are being corrupted elsewhere in the system.
This project is used for testing the Quad Gigabit Ethernet FMC at maximum throughput. The design contains 4 AXI Ethernet blocks and 4 hardware traffic generators/checkers. The traffic generators will both generate Ethernet frames and check received frames for bit errors. The transmitted frames contain fixed destination and source MAC addresses, the Ethertype, a payload of random data and the FCS checksum.
To use the sources in this repository, please follow these steps:
- Download the repo as a zip file and extract the files to a directory on your hard drive --OR-- Git users: clone the repo to your hard drive
- Open Windows Explorer, browse to the repo files on your hard drive.
- In the HLS directory, double-click on the
- In the Vivado directory, you will find multiple batch files (*.bat).
Double click on the batch file that is appropriate to your hardware,
for example, double-click
build-zedboard.batif you are using the ZedBoard. This will generate a Vivado project for your hardware platform.
- Run Vivado and open the project that was just created.
- Click Generate bitstream.
- When the bitstream is successfully generated, select
File->Export->Export Hardware. In the window that opens, tick "Include bitstream" and "Local to project".
- Return to Windows Explorer and browse to the SDK directory in the repo.
- Double click the
build-sdk.batbatch file. The batch file will run the
build-sdk.tclscript and build the SDK workspace containing the hardware design and the software application.
- Run Xilinx SDK (DO NOT use the Launch SDK option from Vivado) and select the workspace to be the SDK directory of the repo.
- Connect and power up the hardware.
- Open a Putty terminal to view the UART output.
- In the SDK, select
Xilinx Tools->Program FPGA.
- Right-click on the application and select
Run As->Launch on Hardware (System Debugger)
Installation of MicroZed and PicoZed board definition files
To use the projects for the MicroZed and PicoZed, you must first install the board definition files for those boards into your Vivado and Xilinx SDK installation.
The following folders contain the board definition files and can be found in this project repository at this location:
Copy those folders and their contents into the
C:\Xilinx\Vivado\2018.2\data\boards\board_files folder (this may
be different on your machine, depending on your Vivado installation directory). You also need to make a copy into the
Xilinx SDK installation at this location:
In order to test an Ethernet device at maximum throughput (back-to-back packets at 1Gbps), one could setup the MACs to loopback to each other and then send packets to each port from an external source such as a PC which could compare the returned packets to the sent ones. However, it is generally difficult to use a PC Ethernet port at full throughput, because a PC typically has too many overheads which create a delay between consecutive packets. For this reason, this design uses four hardware packet generator/checkers that are implemented in the FPGA. These generator/checkers drive the AXI Ethernet cores (the MACs) with a continuous stream of packets. By using the FPGA to generate the Ethernet packets, we are able to exploit almost 100% of the potential bandwidth.
The software application sets up the MACs in promiscuous mode which allows them to pass through all packets, regardless of their destination MAC address. It also sets them up to receive the FCS (checksum) from the user design, rather than calculating and inserting it itself.
Detecting Bit Errors
Counting Dropped Frames
Due to the FCS (checksum) which is present in every Ethernet packet, most bit errors that are injected into the system will result in dropped packets at the receiving MAC (ie. the receiving MAC will reject packets where the FCS does not match the frame data). Therefore, our primary method for detection of bit errors involves polling the MACs for rejected frames. The number of rejected frames is tracked by the software application.
To ensure that the MACs are truly rejecting frames with bit errors, we inject one bit error into one packet per second, on all 4 ports. Our design supplies the FCS to the transmit interface of the MACs, rather than having the MACs calculate and append the FCS. This allows us to inject a bit error that should render the FCS incorrect for the frame.
Checking Received Frames
The checksum should prevent most bit errors from getting through the receiving MAC however it is not an infallible detection system. To detect the errors that do not result in rejected frames, the Ethernet Traffic Generator IP reads the received packets and compares them with the known transmitted frame. Any bit errors are counted and accessible to the processor through a software register.
Ethernet Traffic Generator IP
The traffic generator IP was designed in Vivado HLS (High-level Synthesis) and is coded in C++. Vivado HLS allows hardware algorithms to be programmed in the C/C++ language which offers tremendous advantages over VHDL and Verilog, especially when developing packet processing systems. This example design serves as a good platform for developing Ethernet packet processing algorithms with the Ethernet FMC.
The Ethernet Traffic Generator IP can be simulated in Vivado by using the RTL testbench that
is included with the project. The Vivado project contains two block designs,
used for implementation and simulation respectively. The
design_2 block design contains one
instantiation of the Ethernet Traffic Generator IP (the DUT) and one IP core that is designed to initialize the
software registers of the DUT. The RTL testbench connects the output of the DUT (TX frames) to the input
of the DUT (RX frames). To run the simulation, simply open the Vivado project and select
Run Simulation->Run Behavioral Simulation.
This design is actually used as a production test for the Quad Gigabit Ethernet FMC because it places maximum stress on the PHYs, which forces the maximum current consumption, heat dissipation and possibility for cross-talk between lanes. It can however be a very useful design for people who need to communicate over Ethernet with another FPGA or an Ethernet device that can support the high throughput.
Check the following if the project fails to build or generate a bitstream:
1. Are you using the correct version of Vivado for this version of the repository?
Check the version specified in the Requirements section of this readme file. Note that this project is regularly maintained to the latest version of Vivado and you may have to refer to an earlier commit of this repo if you are using an older version of Vivado.
2. Did you follow the Build instructions in this readme file?
All the projects in the repo are built, synthesised and implemented to a bitstream before being committed, so if you follow the instructions, there should not be any build issues.
3. Did you copy/clone the repo into a short directory structure?
Vivado doesn't cope well with long directory structures, so copy/clone the repo into a short directory structure such as
C:\projects\. When working in long directory structures, you can get errors relating to missing files, particularly files
that are normally generated by Vivado (FIFOs, etc).
Feel free to modify the code for your specific application.
Fork and share
If you port this project to another hardware platform, please send me the code or push it onto GitHub and send me the link so I can post it on my website. The more people that benefit, the better.
This project was developed by Opsero Inc., a tight-knit team of FPGA experts delivering FPGA products and design services to start-ups and tech companies. Follow our blog, FPGA Developer, for news, tutorials and updates on the awesome projects we work on.