Skip to content

Qualcomm AI Engine Direct - docs fix #14881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 9, 2025
Merged

Qualcomm AI Engine Direct - docs fix #14881

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9d3fb27
Qualcomm AI Engine Direct - docs fix
DannyYuyang-quic Oct 8, 2025
41f2c2e
remove aot and runtime build comments, and add instruction for instal…
DannyYuyang-quic Oct 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 4 additions & 85 deletions docs/source/backends-qualcomm.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,10 +74,9 @@ This example is verified with SM8550 and SM8450.
- A compiler to compile AOT parts, e.g., the GCC compiler comes with Ubuntu LTS.
- [Android NDK](https://developer.android.com/ndk). This example is verified with NDK 26c.
- [Qualcomm AI Engine Direct SDK](https://developer.qualcomm.com/software/qualcomm-ai-engine-direct-sdk)
- Click the "Get Software" button to download a version of QNN SDK.
- However, at the moment of updating this tutorial, the above website doesn't provide QNN SDK newer than 2.22.6.
- The below is public links to download various QNN versions. Hope they can be publicly discoverable soon.
- [QNN 2.37.0](https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.37.0.250724/v2.37.0.250724.zip)
- Click the "Get Software" button to download the latest version of the QNN SDK.
- Although newer versions are available, we have verified and recommend using QNN 2.37.0 for stability.
- You can download it directly from the following link: [QNN 2.37.0](https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.37.0.250724/v2.37.0.250724.zip)
Comment on lines +77 to +79
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cccclai we need to update these instructions. We shouldnt need to download sdk with qnn pip wheel. Make this either optional or remove it entirely. I would prefer latter because it avoid confusion

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pip wheel release hasn't finished yet. The nightly build is based on viable/strict and it has been stuck for a week, which doesn't include our latest commit for the fix. After it's verified, I can add a session and move this part to build from source session


The directory with installed Qualcomm AI Engine Direct SDK looks like:
```
Expand Down Expand Up @@ -136,86 +135,6 @@ cd $EXECUTORCH_ROOT
./backends/qualcomm/scripts/build.sh --release
```

### AOT (Ahead-of-time) components:

Python APIs on x64 are required to compile models to Qualcomm AI Engine Direct binary.

```bash
cd $EXECUTORCH_ROOT
mkdir build-x86
cd build-x86
# Note that the below command might change.
# Please refer to the above build.sh for latest workable commands.
cmake .. \
-DCMAKE_INSTALL_PREFIX=$PWD \
-DEXECUTORCH_BUILD_QNN=ON \
-DQNN_SDK_ROOT=${QNN_SDK_ROOT} \
-DEXECUTORCH_BUILD_DEVTOOLS=ON \
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
-DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
-DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
-DPYTHON_EXECUTABLE=python3

# nproc is used to detect the number of available CPU.
# If it is not applicable, please feel free to use the number you want.
cmake --build $PWD --target "PyQnnManagerAdaptor" "PyQnnWrapperAdaptor" -j$(nproc)

# install Python APIs to correct import path
# The filename might vary depending on your Python and host version.
cp -f backends/qualcomm/PyQnnManagerAdaptor.cpython-310-x86_64-linux-gnu.so $EXECUTORCH_ROOT/backends/qualcomm/python
cp -f backends/qualcomm/PyQnnWrapperAdaptor.cpython-310-x86_64-linux-gnu.so $EXECUTORCH_ROOT/backends/qualcomm/python

# Workaround for .fbs files in exir/_serialize
cp $EXECUTORCH_ROOT/schema/program.fbs $EXECUTORCH_ROOT/exir/_serialize/program.fbs
cp $EXECUTORCH_ROOT/schema/scalar_type.fbs $EXECUTORCH_ROOT/exir/_serialize/scalar_type.fbs
```

### Runtime:

An example `qnn_executor_runner` executable would be used to run the compiled `pte` model.

Commands to build `qnn_executor_runner` for Android:

```bash
cd $EXECUTORCH_ROOT
mkdir build-android
cd build-android
# build executorch & qnn_executorch_backend
cmake .. \
-DCMAKE_INSTALL_PREFIX=$PWD \
-DEXECUTORCH_BUILD_QNN=ON \
-DQNN_SDK_ROOT=$QNN_SDK_ROOT \
-DEXECUTORCH_BUILD_DEVTOOLS=ON \
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
-DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
-DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
-DPYTHON_EXECUTABLE=python3 \
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_ROOT/build/cmake/android.toolchain.cmake \
-DANDROID_ABI='arm64-v8a' \
-DANDROID_PLATFORM=android-30

# nproc is used to detect the number of available CPU.
# If it is not applicable, please feel free to use the number you want.
cmake --build $PWD --target install -j$(nproc)

cmake ../examples/qualcomm \
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_ROOT/build/cmake/android.toolchain.cmake \
-DANDROID_ABI='arm64-v8a' \
-DANDROID_PLATFORM=android-30 \
-DCMAKE_PREFIX_PATH="$PWD/lib/cmake/ExecuTorch;$PWD/third-party/gflags;" \
-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH \
-DPYTHON_EXECUTABLE=python3 \
-Bexamples/qualcomm

cmake --build examples/qualcomm -j$(nproc)

# qnn_executor_runner can be found under examples/qualcomm
# The full path is $EXECUTORCH_ROOT/build-android/examples/qualcomm/executor_runner/qnn_executor_runner
ls examples/qualcomm
```

**Note:** If you want to build for release, add `-DCMAKE_BUILD_TYPE=Release` to the `cmake` command options.


## Deploying and running on device

Expand Down Expand Up @@ -365,7 +284,7 @@ The model, inputs, and output location are passed to `qnn_executorch_runner` by

## Supported model list

Please refer to `$EXECUTORCH_ROOT/examples/qualcomm/scripts/` and `EXECUTORCH_ROOT/examples/qualcomm/oss_scripts/` to the list of supported models.
Please refer to `$EXECUTORCH_ROOT/examples/qualcomm/scripts/` and `$EXECUTORCH_ROOT/examples/qualcomm/oss_scripts/` to the list of supported models.

## How to Support a Custom Model in HTP Backend

Expand Down
7 changes: 4 additions & 3 deletions examples/qualcomm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,12 +111,13 @@ This section outlines the essential APIs and utilities provided to streamline th
Creates a clean directory for storing model outputs or intermediate results. If the directory already exists, it will be deleted and recreated to ensure a consistent environment for each run.

## Additional Dependency
This example requires the following Python packages:
- pandas and scikit-learn: used in the mobilebert multi-class text classification example.
- graphviz (optional): used for visualizing QNN graphs during debugging.

The mobilebert multi-class text classification example requires `pandas` and `sklearn`.
Please install them by something like

```bash
pip install scikit-learn pandas
pip install scikit-learn pandas graphviz
Copy link
Contributor

@abhinaykukkadapu abhinaykukkadapu Oct 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to install lm_eval too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lm_eval is only for llama I think

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I see it is part of examples/models/llama/install_requirements.sh, may be we should mention this in the instructions to run llama models?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think it's reasonable to have it as part of the instructions to run llama models

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ve just added the instruction at this line

```

## Limitation
Expand Down
1 change: 1 addition & 0 deletions examples/qualcomm/oss_scripts/llama/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ We offer the following modes to execute the model:
### Step 1: Setup
1. Follow the [tutorial](https://pytorch.org/executorch/main/getting-started-setup) to set up ExecuTorch.
2. Follow the [tutorial](https://pytorch.org/executorch/main/backends-qualcomm) to build Qualcomm AI Engine Direct Backend.
3. Please install the llm eval dependency via [examples/models/llama/install_requirements.sh](https://github.com/pytorch/executorch/blob/main/examples/models/llama/install_requirements.sh)

### Step 2: Prepare Model

Expand Down
Loading