Run ML model on GPU of an Android run system

[besartgrabanica]

Hello,

I would like to know whether ONNX Runtime has GPU support in Android systems. In my particular example, I would need to run the inference of a PyTorch based Deep Learning model which utilizes transformers on the GPU of an Android system.

Thanks in advance!

[guoyu-wang]

We have the NNAPI execution provider which uses CPU/GPU/NPU for model execution on Android.
Please see NNAPI Execution Provider

[deephudka005]

Can you provide some example code to run inference of ort model on GPU for Android? I tried using session options and adding NNAPI to it but I am not aware if my method is correct or not.

val options = OrtSession.SessionOptions()
options.addNnapi()
ortSession = ortEnv?.createSession(onnxModel, options)

[Uwenisme]

Contributor
Do you mean we do not need to select specific device(cpu, gpu or npu) and NNAPI EP can use all of them automatically?