Expected Performance on Mobile?

[werez-mgr]

Reading Issue #26 - #26

"It should work nicely with a Samsung S7..."

When i run the Raccoon Demo on a Samsung Note 10 i am only getting around 5-12 FPS with Chrome, and there arent any errors in the console. Im assuming i should be getting better performance if it should work nicely on a 3 generations old phone?

[xavierjs]

Hi

Unfortunately I don't have this device.
If you send me a galaxy note 10 I could take a look.

Best
Xavier

[werez-mgr]

Unfortunately i dont have a spare $1500 to supply a device.
I know there can be a 100 + 1 device specific issues/variable. Relating to the comment on issue #26 with it working nicely on an S7 ;
https://www.gadgetsnow.com/compare-mobile-phones/Samsung-Galaxy-Note-10-Plus-5G-vs-Samsung-Galaxy-S7

Technically speaking from what i can see, there is the hardware requirements available to run the library on a Note 10 when compared to the S7. So if there is a handful of old devices which get 30 FPS+ then i know i have a device specific issue. Otherwise if working nicely means around 10 FPS on older devices, im not any more clear on benchmarks

I will try to get a hold of a range of devices andabout posting what benchmarks i can

[werez-mgr]

Apologies for misunderstanding, i don't wish for you to solve my issues. Thats why i didn't ask for a fix, commit or a timeline of completion.
I was just wondering what benchmark you may have done/what the basis was for that comment 👍
"It should work nicely with a Samsung S7..."

Was that just a guess?

Just trying to get an understanding*, not asking for free work.

*Typo

[xavierjs]

Hi,

The source of the problem can be:

• the webRTC implementation: maybe the video resolution is too high. SInce many getUserMedia API are bad, especially handling media constraints, if we don't manage to get a low resolution image we try with other media constraints, then with no media constraints at all. In this latest case the video resolution can be high. I had this problem with a version of IOS 12 and some specific media constraints (the bug has been fixed since),
• the WebGL implementation may introduce bottlenecks, especially when refreshing the video texture. This operation may involve the CPU, which makes thing slows,
• 2 WebGL contexts are created: 1 for the neural network processing, it is also the same used to display the video with the user face and the frame, and the other with THREE.js. Maybe the WebGL implementation struggles with switching contexts,
• maybe the resolution of the THREE.js canvas is too large,
• maybe other applications are running in the background, or other tabs are opened,
• a faulty OpenGL implementation. On Chrome you can open chrome://gpu . Sometimes the driver is dumb so the browser has too add workarounds (see Problems Detected section) that can ruin the performance (especially when performance is strongly due to instancing like in our deep learning framework),
• or other thing...

So there are different ways to explore to find the problem. Without the device it is difficult to proceed. It is harder to solve than when there is a specific bug somewhere or if it was not working at all.

I think the GPU is powerful enough, there is a bottleneck somewhere.
If the cartman demo ( https://jeeliz.com/demos/weboji/demos/svg/ ) works well we can already exclude cause due to getUserMedia implementation or video texture refresh.

When I tested on the Samsung S7 it was not running at 10FPS, but faster enough to be usable (at least 30FPS), with Chrome. It was not my phone so I cannot retest now. On My 4 yo Sony Xperia ZPerformance, which is quite dumb, it works well. On Iphone6 it is a bit slow. On Iphone XR/Ipad pro it works nicely. On Samsung galaxy A5 it is too slow to give a good experience. These are the latest tests.

[werez-mgr]

Amazing! Thank you for the breakdown, gives me some great options to explore. Ill be sure to return anything interesting i come across looking into it.

Cheers!

[xavierjs]

Hi @werez-mgr

I have pushed a new neural network, faster and lighter than the previous one. It makes a good difference on my medium range mobile device (4yo sony Xperia, same computing power than Iphone6). I hope it can help.