GPU usage 0% on M3 Macbook Air

[yyzguy]

Using Macbook Air with M3. Installed the onxruntime for Apple Silicon.
Used command line to start with coreml
CPU is running hard 766%
GPU is at 0%

What am I doing wrong?

[e-basaran]

We're hoping to get a fix. Seems like .onnx doesn't fully support CoreML. Here is what you can do to get a better performance (lowering the CPU usage, and temperature, getting higher fps):

For a better performance, please download inswapper_216.onnx (no fp16) from huggingface, rename it to inswapper_216.fp16.onnx, and replace this file with the one in the modules folder.

Also, instead of installing onnxruntime 1.13.1, try 1.16.3:

pip uninstall onnxruntime onnxruntime-silicon
pip install onnxruntime-silicon==1.16.3

After these steps, you should be getting +10 fps, and no temperature issues. It still does not fully utilize the GPU cores though.

[westNeighbor]

@Retha23 Can you share the onnx download link?

[e-basaran]

Here: https://huggingface.co/ezioruan/inswapper_128.onnx/resolve/main/inswapper_128.onnx?download=true

@Retha23 Can you share the onnx download link?

[westNeighbor]

@Retha23 , hmmm, I thought it's a new inswapper_216.onnx, so it's a typo? what's the difference compared to the host provided?

[yyzguy]

I forgot to rename it, but it seems to have installed and works faster. My clips are too small for me to give a quantitive answer as to speed increase. Also output is a smaller file.

[IMNotMax]

Works better with a M1Pro with 16Go
Thanx a lot !

[e-basaran]

You're welcome!

…

On Sun, 18 Aug 2024 at 22:43, Max ***@***.***> wrote: Works better with a M1Pro with 16Go Thanx a lot ! — Reply to this email directly, view it on GitHub <#357 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVJOBSVWLQYR7TBIODEAIXTZSD2OBAVCNFSM6AAAAABMTMTAWSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJVGM3DSMBYGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[TVikg]

I tried this but with my Macbook Pro M1 Pro and 16 Gb is still very slow...
It's correct to launch it with --execution-provider coreml ?

[IMNotMax]

Yes, even with coreml execution provider, it’s slow on a M1Pro 16Go. Not usable for a live for example. (When I tested it, even with the experimental branch btw)