use xfopencv in vivado HLS 2018.2

[tangjie77wd]

I have rebuilt your example--Standalone_HLS_Example according to "HLS_UseModel_Usage_Doc.pdf " successfully in Vivado HLS 2018.2 GUI . I noticed that there is no direct and easy function to convert BRG model to GRAY in xfOpenCV but there is hls::CvtColor() in HLS video library . Therefore, I want to use hls::Cvtcolor() to get gray image and use hls::Mat2AXIvideo() to get stream and then use xf::AXIvideo2xfMat() to do other works that can be done with xfOpenCV API library. But, error occurs ---"common/xf_axi_sdata.h:90:10: error: redefinition of ‘struct ap_axis<D, U, TI, TD>" !!! There is definition of ap_axis in "xf_axi_sdata.h" and there is the same definition of ap_axis "in hls_video.h" which i have to include it because i want to use functions like hls::Cvtcolor() and hls::Scale() which they are very complex in xfOpenCV but easy in hls video library . Can you provide me with some advices ?
`#include "hls_video.h"
#include <ap_int.h>

#include "opencv2/opencv.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "/home/jumper/FPGA_projects/HLS2018.2/xiangAn_wd/xfOpenCV/include/common/xf_sw_utils.h"
#include "/home/jumper/FPGA_projects/HLS2018.2/xiangAn_wd/xfOpenCV/include/common/xf_axi.h"
#include "/home/jumper/FPGA_projects/HLS2018.2/xiangAn_wd/xfOpenCV/include/common/xf_infra.h"

typedef ap_uint<1> uint1;
typedef ap_uint<11> uint11;

// maximum image size
#define MAX_WIDTH 1936
#define MAX_HEIGHT 1456

// typedef video library core structures
typedef hls::stream<ap_axiu<24,1,1,1> > AXI_STREAM;
typedef hls::stream<ap_axiu<8,1,1,1> > AXI_STREAM_GRAY;

typedef xf::Scalar<3, unsigned char> RGB_PIXEL;
typedef xf::Scalar<1, unsigned char> GRAY_PIXEL;

#if NO
#define NPC1 XF_NPPC1
#else
#define NPC1 XF_NPPC8
#endif

typedef xf::Mat<XF_8UC3, MAX_HEIGHT, MAX_WIDTH, NPC1> RGB_IMAGE;
typedef xf::Mat<XF_8UC1, MAX_HEIGHT, MAX_WIDTH, NPC1> GRAY_IMAGE;
typedef hls::Mat<MAX_HEIGHT, MAX_WIDTH, HLS_8UC3> HLS_RGB_IMAGE;
typedef hls::Mat<MAX_HEIGHT, MAX_WIDTH, HLS_8UC1> HLS_GRAY_IMAGE;

void FindTarget(HLS_RGB_IMAGE& srcImage,GRAY_IMAGE& dstImage)
{
HLS_RGB_IMAGE img(MAX_HEIGHT, MAX_WIDTH);
HLS_GRAY_IMAGE img1(MAX_HEIGHT, MAX_WIDTH);
GRAY_IMAGE img1_1(MAX_HEIGHT, MAX_WIDTH);
GRAY_IMAGE img2(MAX_HEIGHT, MAX_WIDTH);
GRAY_IMAGE img3(MAX_HEIGHT, MAX_WIDTH);
GRAY_IMAGE img4(MAX_HEIGHT, MAX_WIDTH);
GRAY_IMAGE img5(MAX_HEIGHT, MAX_WIDTH);

hls::Scale(srcImage,img,1.5);
hls::CvtColor<HLS_BGR2GRAY>(img,img1);
AXI_STREAM_GRAY hlsgray;
hls::Mat2AXIvideo(img1, hlsgray);
xf::AXIvideo2xfMat(hlsgray, img1_1);


xf::Threshold<XF_THRESHOLD_TYPE_BINARY,XF_8UC1,MAX_HEIGHT, MAX_WIDTH,NPC1>(img1_1,img2,38,0,255);

xf::erode<XF_BORDER_CONSTANT,XF_8UC1,MAX_HEIGHT, MAX_WIDTH,NPC1>(img2,img3);
xf::erode<XF_BORDER_CONSTANT,XF_8UC1,MAX_HEIGHT, MAX_WIDTH,NPC1>(img3,img4);
xf::erode<XF_BORDER_CONSTANT,XF_8UC1,MAX_HEIGHT, MAX_WIDTH,NPC1>(img4,img5);

xf::Mat<XF_8UC1, MAX_HEIGHT, MAX_WIDTH, NPC1> dsty(MAX_HEIGHT,MAX_WIDTH);
xf::Mat<XF_8UC1, MAX_HEIGHT, MAX_WIDTH, NPC1> dstx(MAX_HEIGHT,MAX_WIDTH);
xf::Sobel<XF_BORDER_CONSTANT,3,XF_8UC1,XF_8UC1,MAX_HEIGHT,MAX_WIDTH,NPC1>(img5, dstx,dsty);
xf::magnitude<XF_L1NORM,XF_8UC1,XF_8UC1, MAX_HEIGHT,MAX_WIDTH,NPC1>(dstx, dsty,dstImage);

}
`

[bgouthamb]

Hello @tangjie77wd ,

Will moving the operations Scaling and Color-Conversion, out of the FindTarget function, work for you?

If yes, you can perform these operations using OpenCV equivalent functions (Since you have OpenCV includes) , copy the output of the resultant cv::Mat into xf::Mat using copyTO() and send that xf::Mat to the FindTarget function with xf::Mat arguments.

See below psuedo code:

/**Scaling and Color-Conversion using openCV functions/
void opencv_func(cv::Mat input, xf::Mat output)
{
    cv::Mat output_1, output_2;
    input.convertTo(output_1, CV_8UC3, 1.5, 0);
    cv::CvtColor(output_1, output_2, CV_BGR2GRAY);
    output.copyTo(output_2.data);
}

/*Function with only xfOpenCV functions*/
void FindTarget(xf::Mat<..> inimg, xf::Mat<...> dstimg){
    xf::Threshold<...>();
    xf::erode<...>();
    .
   .
   .
}

[tangjie77wd]

Hello @tangjie77wd ,

Will moving the operations Scaling and Color-Conversion, out of the FindTarget function, work for you?

If yes, you can perform these operations using OpenCV equivalent functions (Since you have OpenCV includes) , copy the output of the resultant cv::Mat into xf::Mat using copyTO() and send that xf::Mat to the FindTarget function with xf::Mat arguments.

See below psuedo code:

/**Scaling and Color-Conversion using openCV functions/
void opencv_func(cv::Mat input, xf::Mat output)
{
    cv::Mat output_1, output_2;
    input.convertTo(output_1, CV_8UC3, 1.5, 0);
    cv::CvtColor(output_1, output_2, CV_BGR2GRAY);
    output.copyTo(output_2.data);
}

/*Function with only xfOpenCV functions*/
void FindTarget(xf::Mat<..> inimg, xf::Mat<...> dstimg){
    xf::Threshold<...>();
    xf::erode<...>();
    .
   .
   .
}

@bgouthamb Thanks for your reply ! I have replaced hls video functions with opencv functions and a error information occurs----
xf_structs.h: In instantiation of ‘class xf::Mat<9, 1456, 1936, 8>’: ../../../src/test.cpp:21:65: required from here ../../../xfOpenCV/include/common/xf_structs.h:378:18: error: no type named ‘name’ in ‘struct DataType<9, 8>’
1, I think that it is because i put XF_8UC3 with XF_NPPC8 together like this 'xf::Mat<XF_8UC3, MAX_HEIGHT, MAX_WIDTH, XF_NPPC8>' . Does it not support in xfOpenCV ? This error information disappears when i change XF_NPPC8 to XF_NPPC1 !!
2, But a new error information cames---'error: no match for ‘operator>>’ (operand types are ‘RGB_IMAGE {aka xf::Mat<9, 1456, 1936, 1>}’ and ‘RGB_PIXEL {aka xf::Scalar<3, unsigned char>}’)' ! I think it cames from the following code:
`xf::Mat<XF_8UC3, MAX_HEIGHT, MAX_WIDTH, NPC1> img(MAX_HEIGHT, MAX_WIDTH);

loop_height: for (int i = 0; i < MAX_HEIGHT; i++) {
    loop_width: for (int j = 0; j < MAX_WIDTH; j++) {

    	xf::Scalar<3, unsigned char> src_data;
    	img>>src_data;

    	unsigned char B = 1.1*src_data.val[0];
    	unsigned char G = 1.1*src_data.val[1];
    	unsigned char R = 1.1*src_data.val[2];`

How can i get xf::Scalar from xf::Mat ?

[bgouthamb]

@tangjie77wd

1. XF_8UC3 is not yet supported, although it has been defined in the code. Please see UG1233 page 28 for the list of supported types. Zynq Ultrascale devices only support a maximum interface width of 128-bit. You need to take care of the size of the datatype being resolved for the declared xf::Mat.
2. operator '>>' has not been defined for xf::Scalar/xf::Mat. You can define one yourselves or access it as below:

xf::Mat<XF_8UC4, MAX_HEIGHT, MAX_WIDTH, NPC1> img(MAX_HEIGHT, MAX_WIDTH);
xf::Scalar<4, unsigned char> src_data;
for(int i=0,i<4;i++){
    src_data.val[i] = img.data[0].range(8*i+7,i*8);
}

Please post any coding related questions on the SDSoC forum.

[tangjie77wd]

@bgouthamb Thank you very much ! Do you mind if i ask the first question the first time i asked ----Can hls video function be used in the same source file with xfOpenCV function ? The reason i want to get the reply is that i have to replace hls video function with opencv function to avoid the error and you have not used hls video function in /xfOpenCV/example/dilation/xf_dilation_tb.cpp according to /xfopencv/HLS_Use_Model/HLS_UseModel_Usage_Doc.pdf ---section "Usage Example with interface functions"

[tangjie77wd]

Hello @tangjie77wd ,

Will moving the operations Scaling and Color-Conversion, out of the FindTarget function, work for you?

If yes, you can perform these operations using OpenCV equivalent functions (Since you have OpenCV includes) , copy the output of the resultant cv::Mat into xf::Mat using copyTO() and send that xf::Mat to the FindTarget function with xf::Mat arguments.

See below psuedo code:

/**Scaling and Color-Conversion using openCV functions/
void opencv_func(cv::Mat input, xf::Mat output)
{
    cv::Mat output_1, output_2;
    input.convertTo(output_1, CV_8UC3, 1.5, 0);
    cv::CvtColor(output_1, output_2, CV_BGR2GRAY);
    output.copyTo(output_2.data);
}

/*Function with only xfOpenCV functions*/
void FindTarget(xf::Mat<..> inimg, xf::Mat<...> dstimg){
    xf::Threshold<...>();
    xf::erode<...>();
    .
   .
   .
}

I am so sorry to bother you. The following error will come to me when synthesis
ERROR: [SYNCHK 200-11] /usr/local/include/opencv2/core/mat.hpp:364: Argument 'Mat.data' of function 'release' (/usr/local/include/opencv2/core/mat.hpp:364) has an unsynthesizable type (possible cause(s): pointer to pointer or global pointer). ERROR: [SYNCHK 200-42] /usr/local/include/opencv2/core/mat.hpp:350: pointer comparison is not supported. ERROR: [SYNCHK 200-71] /usr/local/include/opencv2/core/mat.hpp:353: function 'cv::Mat::create(int, int const*, int)' has no function body. ERROR: [SYNCHK 200-72] /home/jumper/FPGA_projects/HLS2018.2/xiangAn_wd/xfOpenCV/include/common/xf_structs.h:636: unsupported c/c++ library function 'malloc'. ERROR: [SYNCHK 200-61] /home/jumper/FPGA_projects/HLS2018.2/xiangAn_wd/xfOpenCV/include/common/xf_structs.h:645: unsupported memory access on variable which is (or contains) an array with unknown size at compile time. ERROR: [SYNCHK 200-71] simpleXFopencv/src/XiangAnWO3.cpp:27: function 'cv::_InputArray::_InputArray(cv::Mat const&)' has no function body. ERROR: [SYNCHK 200-71] simpleXFopencv/src/XiangAnWO3.cpp:27: function 'cv::_OutputArray::_OutputArray(cv::Mat&)' has no function body. ERROR: [SYNCHK 200-71] simpleXFopencv/src/XiangAnWO3.cpp:27: function 'cv::convertScaleAbs(cv::_InputArray const&, cv::_OutputArray const&, double, double)' has no function body. ERROR: [SYNCHK 200-71] simpleXFopencv/src/XiangAnWO3.cpp:28: function 'cv::cvtColor(cv::_InputArray const&, cv::_OutputArray const&, int, int)' has no function body. ERROR: [SYNCHK 200-22] /home/jumper/FPGA_projects/HLS2018.2/xiangAn_wd/xfOpenCV/include/common/xf_structs.h:591: memory copy is not supported unless used on bus interface possible cause(s): non-static/non-constant local array with initialization). ERROR: [SYNCHK 200-41] /usr/local/include/opencv2/core/mat.hpp:278: unsupported pointer reinterpretation from type 'Mat' to type 'Mat' on variable 'img1_0.step.p'. ERROR: [SYNCHK 200-71] /usr/local/include/opencv2/core/mat.hpp:278: function 'cv::fastFree(void*)' has no function body. INFO: [SYNCHK 200-10] 12 error(s), 1 warning(s). ERROR: [HLS 200-70] Synthesizability check failed. command 'ap_source' returned error code
Does opencv functions not support synthesis ? You have never used opencv functions in the source files in xfOpenCV/example/... but in testBeach file , is the reason of which due to opencv functions does not support synthesis ? Or the two functions (cv::cvtColor and cv::convertScaleAbs) are unsynthesizable functions unfortunatly because they have unsynthesizable types or use malloc() ? I have to use hls video functions (hls::Scale and hls::Cvtcolor) instead ,which means i have to meet the first error---"error: the redifination of ap_axiu and ap_axis in hls_video.h and xfopencv/common/..." . Oh, i am in a mess now!

[tangjie77wd]

All of these has been solved by changing xfopencv/include/common/xf_axi_sdata.h .

[Rooholla-KhorramBakht]

What change to the xfopencv/include/common/xf_axi_sdata.h fixed the problem?