How to build qnn_executor_runner for linux-gcc9.3?

[huangzhiyuan]

My requirements are that I want to compile the model on x86 host and run the inference on linux device using Qualcomm AI Engine, e.g. SA8295. So how to build qnn_executor_runner for linux-gcc9.3 not android? thanks~
the libQnnHtp.so is different in qnn.

$ find . -name libQnnHtp.so
./lib/aarch64-oe-linux-gcc9.3/libQnnHtp.so
./lib/aarch64-android/libQnnHtp.so

[kirklandsign]

Hi @haowhsu-quic do you have idea?

[chiwwang]

I think we can use CMake Toolchain File, just as we use it for building Android.
If you have a CMake toolchain file for linux-oe-gcc9.3, we can use it.
https://cmake.org/cmake/help/latest/variable/CMAKE_TOOLCHAIN_FILE.html

For example, change this line to linux-oe-gcc9.3 toolchain file:

executorch/backends/qualcomm/scripts/build.sh

Line 66 in 1993d92

-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \

[huangzhiyuan]

I think we can use CMake Toolchain File, just as we use it for building Android. If you have a CMake toolchain file for linux-oe-gcc9.3, we can use it. https://cmake.org/cmake/help/latest/variable/CMAKE_TOOLCHAIN_FILE.html

For example, change this line to linux-oe-gcc9.3 toolchain file:

executorch/backends/qualcomm/scripts/build.sh

Line 66 in 1993d92

-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \

@chiwwang thanks for your reply! Could you provide a more detailed steps since I did not found a CMake toolchain file in aarch64-oe-linux-9.3-glibc-2.31 (qcom cross build toolchain).

[chiwwang]

Hi @huangzhiyuan ,

I'm surprised they don't provide a CMake toolchain file. The best is if you can get support from them.

Nevertheless, I might try below content for a CMake toolchain file. I can find a aarch64-oe-linux-gcc-9.3 toolchain internally, but not sure if it's the same as yours.

In short, I'm trying to setup compilers and sysroot for CMake. Put below content in a XYZ.cmake, and pass XYZ.cmake to CMAKE_TOOLCHAIN_FILE or so.

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)

set(TOOLCHAIN_ROOT /path/to/aarch64-oe-linux-gcc-9.3/)
set(TOOLCHAIN_PREFIX ${TOOLCHAIN_ROOT}/sysroots/x86_64-oesdk-linux/usr/bin/aarch64-oe-linux/aarch64-oe-linux-)
set(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}gcc)
set(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}g++)
set(CMAKE_SYSROOT ${TOOLCHAIN_ROOT}/sysroots/aarch64-oe-linux)
set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE)

[huangzhiyuan]

@chiwwang big thanks and the cmake toolchain file works well!
But I got a runtime error as below:

[IVI]# ./qnn_executor_runner --model_path ./mv2.pte
I 00:00:00.004585 executorch:qnn_executor_runner.cpp:82] Model file ./mv2.pte is loaded.
I 00:00:00.004637 executorch:qnn_executor_runner.cpp:91] Using method forward
I 00:00:00.004661 executorch:qnn_executor_runner.cpp:139] Setting up planned buffer 0, size 606112.
[INFO][Qnn ExecuTorch] create QNN Logger with log_level 2
[WARNING]QnnDsp <W> Initializing HtpProvider

[INFO][Qnn ExecuTorch] Initialize Qnn backend parameters for Qnn executorch backend type 2
[INFO][Qnn ExecuTorch] Caching: Caching is in RESTORE MODE.
[WARNING]QnnDsp <W> HTP arch will be deprecated, please set SoC id instead.

[WARNING]QnnDsp <W> sg_stubPtr is not null, skip loadRemoteSymbols


/prj/qct/webtech_hyd18/mlg_user_admin/qaisw_source_repo/qaisw_repo_release/qnn_tools/QNNTools/Toolchain/Hexagon/hexagon-sdk-4.2.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc
[ERROR]QnnDsp <E> DspTransport.openSession qnn_open failed, 0x80000406

[ERROR]QnnDsp <E> IDspTransport: Unable to load lib 0x80000406

[ERROR]QnnDsp <E> DspTransport failed,cannot open session, error 0x00000009

[ERROR]QnnDsp <E> Unable to load Skel Library. transportStatus: 9

[ERROR]QnnDsp <E> Failed to retrieve skel build id: err: 1008

[ERROR]QnnDsp <E> Failed to create transport for device, error: 1008

[ERROR]QnnDsp <E> Failed to load skel, error: 1008

[ERROR]QnnDsp <E> Transport layer setup failed: 1008

[ERROR]QnnDsp <E> Failed to parse default platform info: 1008

[ERROR]QnnDsp <E> Failed to load default platform info: 1008

[ERROR]QnnDsp <E> Failed to parse platform config: 1008

[ERROR][Qnn ExecuTorch] Failed to create device_handle for Backend ID 6, error=1008
E 00:00:00.067402 executorch:QnnManager.cpp:85] Fail to configure Qnn device
E 00:00:00.067417 executorch:QnnExecuTorchBackend.cpp:140] Fail to initialize Qnn Manager
E 00:00:00.067421 executorch:method.cpp:102] Init failed for backend QnnBackend: 0x1
F 00:00:00.067435 executorch:qnn_executor_runner.cpp:157] In function main(), assert failed (method.ok()): Loading of method forward failed with status 0x1
Aborted (core dumped)

the steps I used:

• qnn_executor_runner：cross build from aarch64-oe-linux-gcc-9.3 toolchain as discussed earlier
• mv2.pte：python -m examples.qualcomm.scripts.export_example --model_name mv2
• so: libQnnHtp.so/libQnnHtpV68Skel.so/libQnnHtpV68Stub.so/libQnnSystem.so, adb push to device

Any suggestions for this error?

[chiwwang]

Hi @huangzhiyuan
It fails to load skel. So maybe we can check if we set ADSP_LIBRARY_PATH correctly.

Here is an example of pushing libraries. We need to use gcc9.3 variant of QNN though.

executorch/examples/qualcomm/scripts/utils.py

Line 81 in 5318baa

# necessary artifacts

I think you have done this.

Then here is the command used by our examples on devices:

executorch/examples/qualcomm/scripts/utils.py

Line 117 in 5318baa

qnn_executor_runner_cmds = " ".join(

You can see we need to set both LD_LIBRARY_PATH CPU side libraries, and ADSP_LIBRARY_PATH for Skel.

[shuyuan-wang]

Hi, did you solve this problem by setting ADSP_LIBRARY_PATH and LD_LIBRARY_PATH separately?