Query on ONNX Encoder Inference Time: Samsung A20 with Android 11

[karamazvo]

The performance demonstrated in the paper for the iPhone 14 is remarkable, and I'm currently attempting to evaluate the model on an entry-level Android phone.

In a basic implementation, the encoder_session.run() operation takes approximately 3,000-4,000 ms for 720x540 images on a Samsung A20 (using the CPU execution provider) as the official ONNX model files do not support NNAPI. NNAPI is the Neural Network API leveraging the GPU on Android.

I'm curious about the significant difference in encoder performance between the iPhone 14, achieving 70 FPS and 14 ms per image, and the Samsung A20, taking 3,000-4,000 ms per image.

Could you please provide some advice on how to address this performance gap?

[Environment]
My testing setup involves ONNX files from the official link: EdgeSAM Encoder and EdgeSAM Decoder.
The tests are conducted using ONNX Runtime version 1.16.3 on a Samsung A20 with Android 11 (CPU: Exynos 7884, GPU: Mali-G71 MP2).

I appreciate your assistance and thank you for your valuable contributions.

[minh-nguyenhoang]

Can you share the project? I've tried to bring this model to mobile but the model is behaving not as expected, I can't get any mask out of the model.