Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/source/android-examples.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Examples & Demos

- [Working with LLMs - Android Examples](https://github.com/meta-pytorch/executorch-examples/tree/main/llm/android)
- [Demo Apps](https://github.com/meta-pytorch/executorch-examples/tree/main/dl3/android/DeepLabV3Demo#executorch-android-demo-app)
- [Working with LLMs - Android Examples](https://github.com/meta-pytorch/executorch-examples/blob/main/llm/android/LlamaDemo/README.md) - ExecuTorch Llama Android Demo App
- [Demo Apps](https://github.com/meta-pytorch/executorch-examples/tree/main/dl3/android/DeepLabV3Demo#executorch-android-demo-app) - DeepLab v3 model for image segmentation
- {doc}`tutorial-arm-vgf` — Export a simple PyTorch model for the ExecuTorch VGF backend

```{toctree}
Expand Down
2 changes: 1 addition & 1 deletion docs/source/android-vulkan.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
```{include} backends-vulkan.md
```{include} backends/vulkan/vulkan-overview.md
11 changes: 5 additions & 6 deletions docs/source/backends/coreml/coreml-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Core ML delegate is the ExecuTorch solution to take advantage of Apple's [Core M
## Target Requirements

Below are the minimum OS requirements on various hardware for running a Core ML-delegated ExecuTorch model:

- [macOS](https://developer.apple.com/macos) >= 13.0
- [iOS](https://developer.apple.com/ios/) >= 16.0
- [iPadOS](https://developer.apple.com/ipados/) >= 16.0
Expand Down Expand Up @@ -61,7 +62,6 @@ See [Partitioner API](coreml-partitioner.md) for a reference on available partit

The Core ML delegate can also be used as a backend to execute quantized models. See [Core ML Quantization](coreml-quantization.md) for more information on available quantization schemes and APIs.


## Backward compatibility

Core ML supports backward compatibility via the [`minimum_deployment_target`](coreml-partitioner.md#coreml-compilespec) option. A model exported with a specific deployment target is guaranteed to work on all deployment targets >= the specified deployment target. For example, a model exported with `coremltools.target.iOS17` will work on iOS 17 or higher.
Expand Down Expand Up @@ -91,16 +91,15 @@ target_link_libraries(

No additional steps are necessary to use the backend beyond linking the target. A Core ML-delegated .pte file will automatically run on the registered backend.


## Reference

**→{doc}`coreml-troubleshooting` — Debug common issues.**
**→{doc}`/backends/coreml/coreml-troubleshooting` — Debug common issues.**

**→{doc}`coreml-partitioner` — Partitioner options.**
**→{doc}`/backends/coreml/coreml-partitioner` — Partitioner options.**

**→{doc}`coreml-quantization` — Supported quantization schemes.**
**→{doc}`/backends/coreml/coreml-quantization` — Supported quantization schemes.**

**→{doc}`coreml-op-support` — Supported operators.**
**→{doc}`/backends/coreml/coreml-op-support` — Supported operators.**

```{toctree}
:maxdepth: 2
Expand Down
6 changes: 3 additions & 3 deletions docs/source/backends/vulkan/vulkan-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,11 +144,11 @@ Any Vulkan-delegated .pte file will automatically run on the registered backend.

## Additional Resources

**→{doc}`vulkan-partitioner`**
**→{doc}`/backends/vulkan/vulkan-partitioner`**

**→{doc}`vulkan-quantization`**
**→{doc}`/backends/vulkan/vulkan-quantization`**

**→{doc}`vulkan-troubleshooting`**
**→{doc}`/backends/vulkan/vulkan-troubleshooting`**

```{toctree}
:maxdepth: 2
Expand Down
1 change: 1 addition & 0 deletions docs/source/edge-platforms-section.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ Key features:
## Next Steps

After choosing your platform:

- **{doc}`backends-section`** - Deep dive into backend selection and optimization
- **{doc}`llm/working-with-llms`** - Working with Large Language Models on edge devices

Expand Down
166 changes: 99 additions & 67 deletions docs/source/using-executorch-android.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,28 @@

# Using ExecuTorch on Android

To use from Android, ExecuTorch provides Java/Kotlin API bindings and Android platform integration, available as an AAR file.
🚀 Quick Start: __New to ExecuTorch__ ? Jump to [Using AAR from Maven Central](#using-aar-from-maven-central) for the fastest setup, then see the [Runtime Integration](#runtime-integration) example.

Note: This page covers Android app integration through the AAR library. The ExecuTorch C++ APIs can also be used from Android native, and the documentation can be found on [this page about cross compilation](using-executorch-building-from-source.md#cross-compilation).
To use from Android, ExecuTorch provides Java/Kotlin API bindings and Android platform integration, available as an AAR file.
Note: This page covers Android app integration through the AAR library. The ExecuTorch C++ APIs can also be used from Android native, and the documentation can be found on this page about cross compilation.

## Installation

All ExecuTorch Android libraries are packaged into an [Android library (AAR)](https://developer.android.com/studio/projects/android-library), `executorch.aar` for both generic (image/audio processing) and LLM (LLaMA) use case. In each release, prebuilt AAR artifacts are uploaded to [Maven](https://repo.maven.apache.org/maven2/org/pytorch/executorch-android/) and S3. Users can also build the AAR from source.
__Choose your installation method:__

- __[Maven Central](#using-aar-from-maven-central)__ (recommended): Easiest for most developers
- __[Direct AAR file](#using-aar-file-directly)__: For specific versions or offline development
- __[Build from source](#building-from-source)__: For custom backends or contributions

All ExecuTorch Android libraries are packaged into an Android library (AAR), executorch.aar for both generic (image/audio processing) and LLM (LLaMA) use case. In each release, prebuilt AAR artifacts are uploaded to Maven and S3. Users can also build the AAR from source.

### Contents of library

The AAR artifact contains the Java library for users to integrate with their Java/Kotlin application code, as well as the corresponding JNI library (.so file), which is loaded by the Java code during initialization.

- [Java library](https://github.com/pytorch/executorch/tree/main/extension/android/executorch_android/src/main/java/org/pytorch/executorch)
- JNI contains the JNI binding for the corresponding Java code, and ExecuTorch native library, including
- core ExecuTorch runtime libraries
- Core ExecuTorch runtime libraries
- XNNPACK backend
- Portable kernels
- Optimized kernels
Expand All @@ -24,42 +32,52 @@ The AAR artifact contains the Java library for users to integrate with their Jav

The AAR library can be used for generic Android device with arm64-v8a or x86_64 architecture. It can be used across form factors, including phones, tablets, tv boxes, etc, as it does not contain any UI components.

## Using AAR from Maven Central
XNNPACK backend

ExecuTorch is available on [Maven Central](https://mvnrepository.com/artifact/org.pytorch/executorch-android).
Portable kernels
Optimized kernels
Quantized kernels
LLaMa-specific Custom ops library.
Comes with two ABI variants, arm64-v8a and x86_64.
The AAR library can be used for generic Android device with arm64-v8a or x86_64 architecture. It can be used across form factors, including phones, tablets, tv boxes, etc, as it does not contain any UI components.

Simply add the target [`org.pytorch:executorch-android:${executorch_version}`](https://repo.maven.apache.org/maven2/org/pytorch/executorch-android/${executorch_version}/) to your Android app dependency (build.gradle), and build your app.
## Using AAR from Maven Central

For example:
```
# app/build.gradle.kts
✅ Recommended for most developers
ExecuTorch is available on Maven Central.
Simply add the target org.pytorch:executorch-android:${executorch_version} to your Android app dependency (build.gradle), and build your app. For example:

```kotlin
app/build.gradle.kts
dependencies {
implementation("org.pytorch:executorch-android:${executorch_version}")
implementation("org.pytorch:executorch-android:${executorch_version}")
}
```

Note: If you want to use release v0.5.0, please use dependency `org.pytorch:executorch-android:0.5.1`.

Click the screenshot below to watch the *demo video* on how to add the package and run a simple ExecuTorch model with Android Studio.
Note: If you want to use release v1.0.0, please use dependency org.pytorch:executorch-android:1.0.0.
Click the screenshot below to watch the demo video on how to add the package and run a simple ExecuTorch model with Android Studio.
<a href="_static/img/android_studio.mp4">
<img src="_static/img/android_studio.jpeg" width="800" alt="Integrating and Running ExecuTorch on Android">
<img src="_static/img/android_studio.jpeg" width="800" alt="Integrating and Running ExecuTorch on Android">
</a>

## Using AAR file directly

You can also directly specify an AAR file in the app. We upload pre-built AAR to S3 during each release, or as a snapshot.

### Released versions (recommended)
### Released versions (Recommended)

| Version | AAR | SHASUMS |
| ------- | --- | ------- |
| [${executorch_version}](https://github.com/pytorch/executorch/releases/tag/${executorch_version}) | [executorch.aar](https://ossci-android.s3.amazonaws.com/executorch/release/${executorch_version}/executorch.aar) | [executorch.aar.sha256sums](https://ossci-android.s3.amazonaws.com/executorch/release/${executorch_version}/executorch.aar.sha256sums) |
| [v1.0.0](https://github.com/pytorch/executorch/releases/tag/v1.0.0) | [executorch.aar](https://ossci-android.s3.amazonaws.com/executorch/release/v1.0.0/executorch.aar) | [executorch.aar.sha256sums](https://ossci-android.s3.amazonaws.com/executorch/release/v1.0.0/executorch.aar.sha256sums) |
| [v0.7.0](https://github.com/pytorch/executorch/releases/tag/v0.7.0) | [executorch.aar](https://ossci-android.s3.amazonaws.com/executorch/release/v0.7.0/executorch.aar) | [executorch.aar.sha256sums](https://ossci-android.s3.amazonaws.com/executorch/release/v0.7.0/executorch.aar.sha256sums) |
| [v0.6.0](https://github.com/pytorch/executorch/releases/tag/v0.6.0) | [executorch.aar](https://ossci-android.s3.amazonaws.com/executorch/release/v0.6.0/executorch.aar) | [executorch.aar.sha256sums](https://ossci-android.s3.amazonaws.com/executorch/release/v0.6.0/executorch.aar.sha256sums) |
| [v0.5.0](https://github.com/pytorch/executorch/releases/tag/v0.5.0) | [executorch.aar](https://ossci-android.s3.amazonaws.com/executorch/release/v0.5.0-rc3/executorch.aar) | [executorch.aar.sha256sums](https://ossci-android.s3.amazonaws.com/executorch/release/v0.5.0-rc3/executorch.aar.sha256sums) |


### Snapshots from main branch

Starting from 2025-04-12, you can download nightly `main` branch snapshots:

* `executorch.aar`: `https://ossci-android.s3.amazonaws.com/executorch/release/snapshot-{YYYYMMDD}/executorch.aar`
* `executorch.aar.sha256sums`: `https://ossci-android.s3.amazonaws.com/executorch/release/snapshot-{YYYYMMDD}/executorch.aar.sha256sums`
* Replace `YYYYMMDD` with the actual date you want to use.
Expand All @@ -77,83 +95,95 @@ We aim to make every daily snapshot available and usable. However, for best stab
## Using AAR file

To add the AAR file to your app:
1. Download the AAR.
2. Add it to your gradle build rule as a file path.
Download the AAR.
Add it to your gradle build rule as a file path.
An AAR file itself does not contain dependency info, unlike the Maven one which bundled with pom.xml. The Java package requires fbjni and soloader, and currently requires users to explicitly declare the dependency. Therefore, two more dependencies in gradle rule is required:

An AAR file itself does not contain dependency info, unlike the Maven one which bundled with pom.xml. The Java package requires `fbjni` and `soloader`, and currently requires users to explicitly declare the dependency. Therefore, two more `dependencies` in gradle rule is required:
```
```kotlin
implementation("com.facebook.soloader:soloader:0.10.5")
implementation("com.facebook.fbjni:fbjni:0.7.0")
```

### Example usage

In your app working directory, such as executorch-examples/llm/android/LlamaDemo,
```

```sh
mkdir -p app/libs
curl https://ossci-android.s3.amazonaws.com/executorch/release/${executorch_version}/executorch.aar -o app/libs/executorch.aar
```

And include it in gradle:
```
# app/build.gradle.kts

```kotlin
app/build.gradle.kts
dependencies {
implementation(files("libs/executorch.aar"))
implementation("com.facebook.soloader:soloader:0.10.5")
implementation("com.facebook.fbjni:fbjni:0.7.0")
implementation(files("libs/executorch.aar"))
implementation("com.facebook.soloader:soloader:0.10.5")
implementation("com.facebook.fbjni:fbjni:0.7.0")
}
```

Now you can compile your app with the ExecuTorch Android library.

## Building from Source

`scripts/build_android_library.sh` is a helper script to build the Java library (into .jar), native library (into .so), and the packaged AAR file.

You need Android [SDK](https://developer.android.com/studio) and [NDK](https://developer.android.com/ndk/downloads) to use it.
```text
scripts/build_android_library.sh
```

is a helper script to build the Java library (into .jar), native library (into .so), and the packaged AAR file.
You need Android SDK and NDK to use it.
Current NDK version used in ExecuTorch CI: r28c.
You need to set ANDROID_HOME to Android SDK home and ANDROID_NDK to the correct NDK root (containing NOTICE file).

You need to set `ANDROID_HOME` to Android SDK home and `ANDROID_NDK` to the correct NDK root (containing NOTICE file).

```
```sh
export ANDROID_HOME=/path/to/sdk
export ANDROID_NDK=/path/to/ndk
sh scripts/build_android_library.sh
```

Currently, XNNPACK backend is always built with the script.
NOTE: Currently, XNNPACK backend is always built with the script.

### Optional environment variables

Optionally, set these environment variables before running `build_android_library.sh`.
Optionally, set these environment variables before running build_android_library.sh.

#### ANDROID_ABIS
Set environment variable `ANDROID_ABIS` to either `arm64-v8a` or `x86_64` if you only need to build the native library for one ABI only.
```
- __ANDROID_ABIS__

Set environment variable ANDROID_ABIS to either arm64-v8a or x86_64 if you only need to build the native library for one ABI only.

```sh
export ANDROID_ABIS=arm64-v8a
# or
# export ANDROID_ABIS=x86_64
```

(Or)

```sh
export ANDROID_ABIS=x86_64
```

And then run the script.

```sh
sh scripts/build_android_library.sh
```

#### EXECUTORCH_CMAKE_BUILD_TYPE
Set environment variable `EXECUTORCH_CMAKE_BUILD_TYPE` to `Release` or `Debug` based on your needs.
- __EXECUTORCH_CMAKE_BUILD_TYPE__

Set environment variable EXECUTORCH_CMAKE_BUILD_TYPE to Release or Debug based on your needs.

#### Using MediaTek backend
- __Using MediaTek backend__

To use [MediaTek backend](backends-mediatek.md),
after installing and setting up the SDK, set `NEURON_BUFFER_ALLOCATOR_LIB` and `NEURON_USDK_ADAPTER_LIB` to the corresponding path.
To use MediaTek backend, after installing and setting up the SDK, set NEURON_BUFFER_ALLOCATOR_LIB and NEURON_USDK_ADAPTER_LIB to the corresponding path.

#### Using Qualcomm AI Engine Backend
- __Using Qualcomm AI Engine Backend__

To use [Qualcomm AI Engine Backend](backends-qualcomm.md#qualcomm-ai-engine-backend),
after installing and setting up the SDK, set `QNN_SDK_ROOT` to the corresponding path.
To use Qualcomm AI Engine Backend, after installing and setting up the SDK, set QNN_SDK_ROOT to the corresponding path.

#### Using Vulkan Backend
- __Using Vulkan Backend__

To use [Vulkan Backend](backends-vulkan.md#vulkan-backend),
set `EXECUTORCH_BUILD_VULKAN` to `ON`.
To use Vulkan Backend, set EXECUTORCH_BUILD_VULKAN to ON.

## Android Backends

Expand All @@ -166,6 +196,7 @@ The following backends are available for Android:
| [Qualcomm AI Engine](https://www.qualcomm.com/developer/software/qualcomm-ai-engine-direct-sdk) | NPU | [Doc](backends-qualcomm.md) |
| [Vulkan](https://www.vulkan.org/) | GPU | [Doc](backends-vulkan.md) |

Start with XNNPACK (CPU backend) for maximum compatibility, then add hardware-specific backends for optimization.

## Runtime Integration

Expand All @@ -175,26 +206,27 @@ Here is an example code sample in Java that demonstrates how to integrate ExecuT
import org.pytorch.executorch.EValue;
import org.pytorch.executorch.Module;
import org.pytorch.executorch.Tensor;

public class MainActivity extends Activity {
private Module module;

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// Load the ExecuTorch module
Module module = Module.load("/data/local/tmp/add.pte");
Tensor tensor1 = Tensor.fromBlob(new float[] {1.0f}, new long[] {1});
Tensor tensor2 = Tensor.fromBlob(new float[] {20.0f}, new long[] {1});

EValue eValue1 = EValue.from(tensor1);
EValue eValue2 = EValue.from(tensor2);
float result = module.forward(eValue1, eValue2)[0].toTensor().getDataAsFloatArray()[0];
}
private Module module;

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// Load the ExecuTorch module
Module module = Module.load("/data/local/tmp/add.pte");

Tensor tensor1 = Tensor.fromBlob(new float[] {1.0f}, new long[] {1});
Tensor tensor2 = Tensor.fromBlob(new float[] {20.0f}, new long[] {1});

EValue eValue1 = EValue.from(tensor1);
EValue eValue2 = EValue.from(tensor2);

float result = module.forward(eValue1, eValue2)[0].toTensor().getDataAsFloatArray()[0];
}
```

Push the corresponding pte file to the phone:
Push the corresponding pte file to your Android device:

```sh
adb push extension/module/test/resources/add.pte /data/local/tmp/
```
Expand Down
2 changes: 2 additions & 0 deletions docs/source/using-executorch-building-from-source.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Even if you don't use CMake directly, CMake can emit scripts for other format
like Make, Ninja or Xcode. For information, see [cmake-generators(7)](https://cmake.org/cmake/help/latest/manual/cmake-generators.7.html).

## System Requirements

### Operating System

ExecuTorch is tested on the following systems, although it should also work in similar environments.
Expand All @@ -20,6 +21,7 @@ ExecuTorch is tested on the following systems, although it should also work in s
* Windows 10+ with Visual Studio 2022+ (experimental)

### Software Requirements

* `conda` or another virtual environment manager
- `conda` is recommended as it provides cross-language
support and integrates smoothly with `pip` (Python's built-in package manager)
Expand Down
Loading