undefined reference errors on native XRT API calls

[Heng-Zhou]

Environment:

• Xilinx Alveo U200 Data Center FPGA Acceleration card
• Ubuntu 22.04.4 LTS
• Vitis Unified IDE 2023.2
• gcc version 11.4.0, installed from "sudo apt install build-essential"
• Project: Vitis-Tutorials/Getting_Started/Vitis, Part 4: Build and Run with the Vitis unified IDE. link.

Problem: At step 6 of "Creating the Application Component" section, when I click "Build" command to build the host application, I got the following error messages:

-- Build files have been written to: /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build
/tools/Xilinx/Vitis/2023.2/tps/lnx64/cmake-3.24.2/bin/cmake -S/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host -B/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build --check-build-system CMakeFiles/Makefile.cmake 0
/tools/Xilinx/Vitis/2023.2/tps/lnx64/cmake-3.24.2/bin/cmake -E cmake_progress_start /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build/CMakeFiles /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build//CMakeFiles/progress.marks
/usr/bin/gmake  -f CMakeFiles/Makefile2 all
gmake[1]: Entering directory '/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build'
/usr/bin/gmake  -f CMakeFiles/host.dir/build.make CMakeFiles/host.dir/depend
/usr/bin/gmake  -f CMakeFiles/GenerateEmulationConfig.dir/build.make CMakeFiles/GenerateEmulationConfig.dir/depend
gmake[2]: Entering directory '/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build'
cd /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build && /tools/Xilinx/Vitis/2023.2/tps/lnx64/cmake-3.24.2/bin/cmake -E cmake_depends "Unix Makefiles" /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build/CMakeFiles/GenerateEmulationConfig.dir/DependInfo.cmake --color=
gmake[2]: Entering directory '/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build'
cd /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build && /tools/Xilinx/Vitis/2023.2/tps/lnx64/cmake-3.24.2/bin/cmake -E cmake_depends "Unix Makefiles" /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build/CMakeFiles/host.dir/DependInfo.cmake --color=
gmake[2]: Leaving directory '/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build'
/usr/bin/gmake  -f CMakeFiles/GenerateEmulationConfig.dir/build.make CMakeFiles/GenerateEmulationConfig.dir/build
gmake[2]: Leaving directory '/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build'
/usr/bin/gmake  -f CMakeFiles/host.dir/build.make CMakeFiles/host.dir/build
gmake[2]: Entering directory '/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build'
gmake[2]: Entering directory '/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build'
[ 33%] Generating emconfig.json
/tools/Xilinx/Vitis/2023.2/bin/emconfigutil --od . --nd 1 --platform /opt/xilinx/platforms/xilinx_u200_gen3x16_xdma_2_202110_1/xilinx_u200_gen3x16_xdma_2_202110_1.xpfm
[ 66%] Building CXX object CMakeFiles/host.dir/host.cpp.o
/usr/bin/g++ -DVITIS_MAJOR_VERSION=2023 -DVITIS_MINOR_VERSION=2 -DVITIS_VERSION=2023.2 -D_GLIBCXX_USE_CXX11_ABI=0 -D__USE_XOPEN2K8 -I/tools/Xilinx/Vitis_HLS/2023.2/include -I/opt/xilinx/xrt/include -g -std=c++14 -MD -MT CMakeFiles/host.dir/host.cpp.o -MF CMakeFiles/host.dir/host.cpp.o.d -o CMakeFiles/host.dir/host.cpp.o -c /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/host.cpp
[100%] Linking CXX executable host
/tools/Xilinx/Vitis/2023.2/tps/lnx64/cmake-3.24.2/bin/cmake -E cmake_link_script CMakeFiles/host.dir/link.txt --verbose=true
/usr/bin/g++ CMakeFiles/host.dir/host.cpp.o -o host   -L/opt/xilinx/xrt/lib  -Wl,-rpath,:/opt/xilinx/xrt/lib:/tools/Xilinx/Vitis_HLS/2023.2/lnx64/tools/fpo_v7_1 /opt/xilinx/xrt/lib/libxilinxopencl.so -lpthread -lrt -lstdc++ 
/usr/bin/ld: CMakeFiles/host.dir/host.cpp.o: in function `main':
/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/host.cpp:32: undefined reference to `xrt::device::load_xclbin(std::string const&)'
/usr/bin/ld: /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/host.cpp:37: undefined reference to `xrt::kernel::kernel(xrt::device const&, xrt::uuid const&, std::string const&, xrt::kernel::cu_access_mode)'
/usr/bin/ld: /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/host.cpp:40: undefined reference to `xrt::kernel::group_id(int) const'
... ...
[ERROR] collect2: error: ld returned 1 exit status
 gmake[2]: *** [CMakeFiles/host.dir/build.make:98: host] Error 1
 gmake[2]: Leaving directory '/home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build'
 gmake[1]: *** [CMakeFiles/Makefile2:85: CMakeFiles/host.dir/all] Error 2
 gmake[1]: *** Waiting for unfinished jobs....

I followed this solution in which it says _GLIBCXX_USE_CXX11_ABI=0 compiler flag could solve the issue. But it does not work.

I checked by executing the commands above one by one, and the "undefined reference errors on native XRT API calls" issue occurs when executing /usr/bin/gmake -f CMakeFiles/host.dir/build.make CMakeFiles/host.dir/build. Then I checked flags file /home/zhou/Vitis-Tutorials/Getting_Started/Vitis/example/workVADD/host/build/CMakeFiles/host.dir/flags.make, and found that the -D_GLIBCXX_USE_CXX11_ABI=0 flag is indeed included in the CXX_DEFINES variable. But it just doesn't work. The gcc is installed from official Ubuntu repository so it should be a standard version, so I think the solution does not apply; there should be some other causes for the "undefined reference" issue.

So, how to solve this issue?

[AnusheelXilinx]

Hi @Heng-Zhou, Thanks for the detailed information. Did you install the supported version of XRT and source it using "source /opt/xilinx/xrt/setup.sh"?

[Heng-Zhou]

Hi @Heng-Zhou, Thanks for the detailed information. Did you install the supported version of XRT and source it using "source /opt/xilinx/xrt/setup.sh"?

Yes. I installed XRT from U200 product webpage according to the OS version, and source /opt/xilinx/xrt/setup.sh command is in ~/.bashrc which is executed every time I log in.

[AnusheelXilinx]

Hi @Heng-Zhou, I tried on Ubuntu 22.04.01 and can see similar(not same) errors related to XRT as the Unified IDE Data Centre Application Component build steps miss to capture "-lxrt_coreutil"

Can you add the above library as per the above screenshot and let me know if that helps?

[Heng-Zhou]

Hi, @AnusheelXilinx, many thanks for pointing out the missing -lxrt_coreutil flag. I think we have figured out the root cause.

I added the -lxrt_coreutil flag in Libraries/Libraries (-l), Linker Settings/Miscellaneous/Linker flags, and Compiler Settings/Miscellaneous/Other flags. These flags should be shown in the generated flag files CMakeFiles/host.dir/link.txt or CMakeFiles/host.dir/flags.make. But no command in these files has -lxrt_coreutil flag set, which causes the issue.

Following is the UserConfig.cmake file that relates to settings of -lxrt_coreutil:

Following is the generated CMakeFiles/host.dir/link.txt file which, however, does not contain the -lxrt_coreutil flag in the command:

Having known that settings in UserConfig.cmake do not work, I then tried to add the -lxrt_coreutil flag in CMakeLists.txt of the host application using set (CMAKE_CXX_FLAGS "-lxrt_coreutil") or set (CMAKE_EXE_LINKER_FLAGS "-lxrt_coreutil"). However, this is the generated command:
/usr/bin/g++ -lxrt_coreutil CMakeFiles/host.dir/host.cpp.o -o host -L/opt/xilinx/xrt/lib -Wl,-rpath,:/opt/xilinx/xrt/lib:/tools/Xilinx/Vitis_HLS/2023.2/lnx64/tools/fpo_v7_1 /opt/xilinx/xrt/lib/libxilinxopencl.so -lpthread -lrt -lstdc++

The problem is the position of the -lxrt_coreutil flag -- it's before CMakeFiles/host.dir/host.cpp.o. If it is after the -o host, the errors would disappear (I tested it by running manually on command line).

So, the root issue is the bugs in Vitis Unified IDE 2023.2 I am using. Specifically, there are two bugs:

1. The configurations typed in fields of UserConfig.cmake does not apply (i.e., are not saved in generated files like link.txt or flags.make.)
2. The -lxrt_coreutil flag specified in CMakeLists.txt is misplaced.

Manually changing the generated files like link.txt or flags.make does not work because Vitis generates a new file every time I click the Build button in the Flow view.

So, can you confirm these 2 bugs for me in Vitis Unified IDE 2023.2? If they are confirmed, can you please submit a bug ticket to Vitis development team so that they can fix this bug in 2024.1 release? If you are busy, I can do that, but I'm afraid my bug report can not attract attention of Xilinx. Please let me know. Thanks.

[Heng-Zhou]

A follow up, if I edit CMakeLists.txt, I should have added -lxrt_coreutil flag to target_link_libraries at line 100:

After the above modification, the building of host application finished successfully.