Repository navigation
Zero lab 2
Welcome to this lab in which you will learn how to create a simple DMA that acquire a image stream signal and writes it to a shared memory.
In most video processing designs, image data are stored in external memories outside the FPGA chip. This is because images are like very large arrays off data for example, let's take an RGB full HD image uncompressed, the size of the "array" would be :
S = 1920x1080x24 b ~= 6.2 MB
Storing this array insied the zybo board would not be possible since the FPGA implemented on it is a XC7Z020 which have only 630 KB. In fact, we would need to use 10 Zybo boards just for storage..Not the best idea, this is the situation where external memories becomes very handy. This ICs are made for storage larger amount of data with high speed access.
There are a couple of things to take into account when working with external memories
- storage capabilities (how many images can we store in it ?)
- storage access latency (what's the response delay of memory before i can get/send my first pixel ?)
- bandwhidth (how many images can we read/write at the same time ?)
In most video processing applications memory access bandwidth is offtenly the limitant factor. Designing an efficient DMA IP is critical to ensure the highgest stability and frame rate processing of a video processing design.
For the memory interface we will have to instanciate a memory transfer bus, there are a few of those in the industry such as Wishbone or Avalon, in our cas, we will use AXI4 standard. For the video stream interface a common choice is the AXI4-stream standard.
a description of these standard is given here :
- AXI4 : https://circuitcellar.com/research-design-hub/design-solutions/understanding-the-amba-axi4-spec/
- AXI4-stream : https://docs.xilinx.com/r/en-US/ug1399-vitis-hls/How-AXI4-Stream-Works
At this point, we may be tempt to create an ultra-light design that connects our AXI4 interface to the AXI4-Stream bus. However, this will eventually result in a very poor exploitation of memory interface which then results in a poor real time processing performance. To be more specific, a memory access start always with a "handshake" between the host (our IP) and device (a memory controller) during this process the IP sends to the memory controller informations about the transfer to be done (the adress, amount of data to read/write) and perform primary checks (Is the controller ready or not ?) handshake procedure itself does not take a lot amount of time howeve, imagine performing this small task times the number of pixels in a image and this will increase by fair amount the time needed to accomplish the full transfer.
To prevent this behavior we need to perform what's called a "bursting access" meaning performing the leaste possible amount of handshakes during the image transfer. Since we are working with images, let's pretend we want to transfer the image line after line, that way we use, ideally, one handshake to acquire a line.
In order to do so, we need a "local memory buffer" (also called line buffer in this case)
In our example, we will work with 8bit grayscale image
A synoptic diagram of the design is given below :
Dataflow development consist of establishing the basic description of our main components for both the actual IP design and the testbench we are going to use later on.
In our case, we have to find a way to describe
- external main memory
- AXI4 interface
- AXI4 stream interface
- line buffer
- Generic AXI4 stream provider
As best practice, we tend to consider first interfaces and testbench componens before getting into the internal components. We offtently refer to this step as "establishing layout"
You can take a look at "VDMAUnit_s2mm" function to see the interfaces description
void VDMAUnit_s2mm (video_stream_in &STR_video_in, param_type_0 image_w, param_type_0 image_h, volatile ddr_type *image_out){
#pragma HLS INTERFACE axis port = STR_video_in
#pragma HLS INTERFACE ap_none port = image_w
#pragma HLS INTERFACE ap_none port = image_h
#pragma HLS INTERFACE m_axi port=image_out depth=1310720 offset=direct
VDMA_s2mm(STR_video_in, image_w, image_h, image_out);
}
video_stream_in, param_type_0 and ddr_type are defined in dma_s2mm_utils.h, these are the equivalent of std_logic_vector and records types
The core functional part is given below
void VDMA_s2mm (video_stream_in &STR_video_in, param_type_0 image_w, param_type_0 image_h, volatile ddr_type *image_out){
static video_stream_element_in video_elt_in;
static ddr_type input_register;
height_counter : for (int j=0; j<image_h; j++){
width_counter : for (int i=0; i<(image_w/4); i++){
//acquire four pixels i.e. four bytes
stackup_pixels : for (int j = 0; j<4; j++){
//#pragma HLS PIPELINE II = 1
STR_video_in >> video_elt_in;
input_register.range((j+1)*PIXEL_RES_g-1,j*PIXEL_RES_g) = video_elt_in.data;
}
//send register content to AXI4 controller
image_out[i+j*(IMAGE_MAX_W_g/4)] = input_register;
}
}
}
The AXI4 streaming consumming engine is called with "STR_video_in >> video_elt_in;" insctruction ; each call correspond to acquiring a pixel data (an 8 bit stream packet)
Because external memories have, in general, a 32 bits length, we must operation a format adaptation to store our pixels correctly. In our case, we can store 4 pixels in 1 external memory adress space. Hence the use of "stackup_pixels" for loop :
stackup_pixels : for (int j = 0; j<4; j++){
//#pragma HLS PIPELINE II = 1
STR_video_in >> video_elt_in; // read a pixel
input_register.range((j+1)*PIXEL_RES_g-1,j*PIXEL_RES_g) = video_elt_in.data; //store pixel in register
}
The description "input_register.range(a,b) = pixel" is the equivalent in VHDL of "input_register(a downto b) <= pixel"
After completing this loop, data is ready for transfer. AXI4 write transfer is describe using image_out[i+j*(IMAGE_MAX_W_g/4)] = input_register;
Were, our memory model, "image_out" is a pointer to the testbench array
/!\ Note that the local linebuffer hasn't been described, has stated in "System design" section this should have a strong impact on performances, we should see that later in "gateware validation" section
Once with described our first version of the dataflow we are able to validate the correct description of our function. We will not evaluate, at this stage, the real-time processing factor ; In other terms, we want to validate that the image is acquired from our AXI4 stream provider model, transfered to the line buffer and finally stored to our memory model.
In order to perform dataflow validation on Vitis HLS start "C simulation"
You will see that C simulation is nothing more than compiling the C program and execute it.
Open ImageJ select import>text image navigate to <your_hls_project>/solution1/csim/build and open Pattern_out.txt, this image should look exactly like your input image.
To perform a proper verification, import the input text image located in the "src" folder. use Process > Image calculator, select Pattern_in.txt and Pattern_out.txt and perform substraction. The output of this process should be a black image.
At this step we've created our layout and validated it, now, it is time to add implementation context in our layout. To do so we will use directives which are commands meant to explain HLS what architecture we want to generate.
The tool will autonomously take decisions on implementation if the designer does not constraints it with directives. Remember that the primary objective of HLS is "Generate VHDL in order to move data from point A to point B" without the awarness of your project contraints in terms of real time processing or ressource utilization. In short "as long as data is effectively moved from A to B it doesn't matter if 90% of FPGA is used or if the frame rate is 0.2Hz" Hence the importance of directives which we will demonstrate
For example,
#pragma HLS INTERFACE
is a directive use to explain to HLS what interface we need, the description :
#pragma HLS INTERFACE ap_none port = my_port
will instanciate a simple std_logic_input input wheras
#pragma HLS INTERFACE axis port = my_port
will instanciate an AXI4 stream interface with all the necessary components required to driver ready and valid signals.
Perform HLS synthesis and observe the provided report
You can a "II violation" has been detected. by default, HLS try to perform generate a hardware description that is able to execute all the instruction with the highet trhoughtput possible.
In this case, HLS have a main directive of "generate hardware that has a pipeline flow of one output every clock cycle"
Since we need four pixels to generate one output and we transfer pixels one clock cyle at a time, it'is effectively impossible to create hardware that performs a 32b output each clock cyle (even for a human designer) II violation in this context it no surprising.
Violations and warnings should be avoided as much as possible, add a constraint directive to HLS by uncommenting pragmas directives within the code.
void VDMA_s2mm (video_stream_in &STR_video_in, param_type_0 image_w, param_type_0 image_h, volatile ddr_type *image_out){
static video_stream_element_in video_elt_in;
static ddr_type input_register;
height_counter : for (int j=0; j<image_h; j++){
width_counter : for (int i=0; i<(image_w/4); i++){
//acquire four pixels i.e. four bytes
#pragma HLS PIPELINE II = 4
stakup_pixels : for (int j = 0; j<4; j++){
#pragma HLS PIPELINE II = 1
STR_video_in >> video_elt_in;
input_register.range((j+1)*PIXEL_RES_g-1,j*PIXEL_RES_g) = video_elt_in.data;
}
//send register content to AXI4 controller
image_out[i+j*(IMAGE_W_o/4)] = input_register;
}
}
}
Perform HLS synthesis and observe the provided report
Now to get an overview of real-time capabilities of this design let's start a co-simulation. This will generate HDL testbench and execute vivado simulator,we will then be able to check our chronogram.
Before starting co-simultion, edit "generics_n_options.h" and reduce IMAGE_H_o downto to 2. This will allow fast cosimulation
let's take a look at waveform by calling vivado simulator GUI
We can see that AXI4 stream interface operates a maximum bandwidth (tready and tvalid signal are active all the time)
Handshakes on AXI4 interface are consitent, no interruption on the bus.
Gragutalition, this look like an efficient DMA core.
You mlay have noticed that line buffer didn't appeared in the C description. This is because this instanciation of AXI4 interface is very common thus, HLS has been designed to automatically add this feature when implementing an AXI4 interface.