diff --git a/docs/source/backends-qualcomm.md b/docs/source/backends-qualcomm.md index 59634b9b39b..8e1b0ebfcb3 100644 --- a/docs/source/backends-qualcomm.md +++ b/docs/source/backends-qualcomm.md @@ -74,10 +74,9 @@ This example is verified with SM8550 and SM8450. - A compiler to compile AOT parts, e.g., the GCC compiler comes with Ubuntu LTS. - [Android NDK](https://developer.android.com/ndk). This example is verified with NDK 26c. - [Qualcomm AI Engine Direct SDK](https://developer.qualcomm.com/software/qualcomm-ai-engine-direct-sdk) - - Click the "Get Software" button to download a version of QNN SDK. - - However, at the moment of updating this tutorial, the above website doesn't provide QNN SDK newer than 2.22.6. - - The below is public links to download various QNN versions. Hope they can be publicly discoverable soon. - - [QNN 2.37.0](https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.37.0.250724/v2.37.0.250724.zip) + - Click the "Get Software" button to download the latest version of the QNN SDK. + - Although newer versions are available, we have verified and recommend using QNN 2.37.0 for stability. + - You can download it directly from the following link: [QNN 2.37.0](https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.37.0.250724/v2.37.0.250724.zip) The directory with installed Qualcomm AI Engine Direct SDK looks like: ``` @@ -136,86 +135,6 @@ cd $EXECUTORCH_ROOT ./backends/qualcomm/scripts/build.sh --release ``` -### AOT (Ahead-of-time) components: - -Python APIs on x64 are required to compile models to Qualcomm AI Engine Direct binary. - -```bash -cd $EXECUTORCH_ROOT -mkdir build-x86 -cd build-x86 -# Note that the below command might change. -# Please refer to the above build.sh for latest workable commands. -cmake .. \ - -DCMAKE_INSTALL_PREFIX=$PWD \ - -DEXECUTORCH_BUILD_QNN=ON \ - -DQNN_SDK_ROOT=${QNN_SDK_ROOT} \ - -DEXECUTORCH_BUILD_DEVTOOLS=ON \ - -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \ - -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \ - -DEXECUTORCH_ENABLE_EVENT_TRACER=ON \ - -DPYTHON_EXECUTABLE=python3 - -# nproc is used to detect the number of available CPU. -# If it is not applicable, please feel free to use the number you want. -cmake --build $PWD --target "PyQnnManagerAdaptor" "PyQnnWrapperAdaptor" -j$(nproc) - -# install Python APIs to correct import path -# The filename might vary depending on your Python and host version. -cp -f backends/qualcomm/PyQnnManagerAdaptor.cpython-310-x86_64-linux-gnu.so $EXECUTORCH_ROOT/backends/qualcomm/python -cp -f backends/qualcomm/PyQnnWrapperAdaptor.cpython-310-x86_64-linux-gnu.so $EXECUTORCH_ROOT/backends/qualcomm/python - -# Workaround for .fbs files in exir/_serialize -cp $EXECUTORCH_ROOT/schema/program.fbs $EXECUTORCH_ROOT/exir/_serialize/program.fbs -cp $EXECUTORCH_ROOT/schema/scalar_type.fbs $EXECUTORCH_ROOT/exir/_serialize/scalar_type.fbs -``` - -### Runtime: - -An example `qnn_executor_runner` executable would be used to run the compiled `pte` model. - -Commands to build `qnn_executor_runner` for Android: - -```bash -cd $EXECUTORCH_ROOT -mkdir build-android -cd build-android -# build executorch & qnn_executorch_backend -cmake .. \ - -DCMAKE_INSTALL_PREFIX=$PWD \ - -DEXECUTORCH_BUILD_QNN=ON \ - -DQNN_SDK_ROOT=$QNN_SDK_ROOT \ - -DEXECUTORCH_BUILD_DEVTOOLS=ON \ - -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \ - -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \ - -DEXECUTORCH_ENABLE_EVENT_TRACER=ON \ - -DPYTHON_EXECUTABLE=python3 \ - -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_ROOT/build/cmake/android.toolchain.cmake \ - -DANDROID_ABI='arm64-v8a' \ - -DANDROID_PLATFORM=android-30 - -# nproc is used to detect the number of available CPU. -# If it is not applicable, please feel free to use the number you want. -cmake --build $PWD --target install -j$(nproc) - -cmake ../examples/qualcomm \ - -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_ROOT/build/cmake/android.toolchain.cmake \ - -DANDROID_ABI='arm64-v8a' \ - -DANDROID_PLATFORM=android-30 \ - -DCMAKE_PREFIX_PATH="$PWD/lib/cmake/ExecuTorch;$PWD/third-party/gflags;" \ - -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH \ - -DPYTHON_EXECUTABLE=python3 \ - -Bexamples/qualcomm - -cmake --build examples/qualcomm -j$(nproc) - -# qnn_executor_runner can be found under examples/qualcomm -# The full path is $EXECUTORCH_ROOT/build-android/examples/qualcomm/executor_runner/qnn_executor_runner -ls examples/qualcomm -``` - -**Note:** If you want to build for release, add `-DCMAKE_BUILD_TYPE=Release` to the `cmake` command options. - ## Deploying and running on device @@ -365,7 +284,7 @@ The model, inputs, and output location are passed to `qnn_executorch_runner` by ## Supported model list -Please refer to `$EXECUTORCH_ROOT/examples/qualcomm/scripts/` and `EXECUTORCH_ROOT/examples/qualcomm/oss_scripts/` to the list of supported models. +Please refer to `$EXECUTORCH_ROOT/examples/qualcomm/scripts/` and `$EXECUTORCH_ROOT/examples/qualcomm/oss_scripts/` to the list of supported models. ## How to Support a Custom Model in HTP Backend diff --git a/examples/qualcomm/README.md b/examples/qualcomm/README.md index 355209f43a7..31443f2d356 100644 --- a/examples/qualcomm/README.md +++ b/examples/qualcomm/README.md @@ -111,12 +111,13 @@ This section outlines the essential APIs and utilities provided to streamline th Creates a clean directory for storing model outputs or intermediate results. If the directory already exists, it will be deleted and recreated to ensure a consistent environment for each run. ## Additional Dependency +This example requires the following Python packages: +- pandas and scikit-learn: used in the mobilebert multi-class text classification example. +- graphviz (optional): used for visualizing QNN graphs during debugging. -The mobilebert multi-class text classification example requires `pandas` and `sklearn`. Please install them by something like - ```bash -pip install scikit-learn pandas +pip install scikit-learn pandas graphviz ``` ## Limitation diff --git a/examples/qualcomm/oss_scripts/llama/README.md b/examples/qualcomm/oss_scripts/llama/README.md index 9bb76142362..be25324d63d 100644 --- a/examples/qualcomm/oss_scripts/llama/README.md +++ b/examples/qualcomm/oss_scripts/llama/README.md @@ -38,6 +38,7 @@ We offer the following modes to execute the model: ### Step 1: Setup 1. Follow the [tutorial](https://pytorch.org/executorch/main/getting-started-setup) to set up ExecuTorch. 2. Follow the [tutorial](https://pytorch.org/executorch/main/backends-qualcomm) to build Qualcomm AI Engine Direct Backend. +3. Please install the llm eval dependency via [examples/models/llama/install_requirements.sh](https://github.com/pytorch/executorch/blob/main/examples/models/llama/install_requirements.sh) ### Step 2: Prepare Model