-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The inference speed using the old version is 5-8 times faster than the latest version #202
Comments
I also noticed the inference increasing significantly making live detection with more than super primitive model very ugly. Do you remember which version started the inference to increase @yingeo ? |
I looked into it and it seems that issues in Flutter. Forked one older repo and tested it: EnvironmentThis was tested on the following environment:
Just detected same blank(ish) camera image. Not too scientific (e.g. phone might heat up and give lower results), so few ms difference should not be taken too seriously, but the 7x or 16x difference is significant. ResultsAverage time until settles (times in ms).
Conclusion
Flutter 3.19.3 seemed to be as slow as 3.16.9. So it seems that the problem is not yet fixed. My repo: https://github.com/timukasr/object_detection_flutter Something similar results can be reproduced with tflite_flutter sample app live_object_detection_ssd_mobilenet. This app seems a bit broken - does not display results on screen. But it reports inference time. With Flutter 3.13.9, it was 170-200ms, with Flutter 3.19.3, it was 500-550ms. For some reason it is slow even with older Flutter, but even slower with newer flutter. |
@timukasr Thanks a lot for you insights! I didnt even try out different flutter versions. I noticed a inference increase by just changing the tflite_flutter version when I upgraded to 0.10.4 (from 0.10.0) I think. I will continue to investigate this when I find time. |
I take it all back. I was not able to reproduce significant inference differences in 0.10.x versions. The massive spikes due to distinct flutter versions should be addressed at some point. |
It has nothing to do with the version of tflite_flutter, but to the version of Flutter. Currently, I use tflite_flutter: version 0.10.4, and Flutter version is 3.7.12. The inference time is consistent with the native speed of Android. If you upgrade flutter to the latest version, the inference speed will be slow. terrible. |
@yingeo Did you compare 3.7.12 with the other version(such as 3.13.9). Having a hard time after downgrade to 3.7.12 (several config should be change lol) |
tried tflite_flutter 0.10.4 + flutter 3.13.9 + video classification(MoviNet) + isolateInterpreter.runForMultipleInputs. My conclusion is that tflite_flutter is currently not optimized for some models/operators... one step of inference (on a single frame image) takes more than 3000ms on Samsung galaxy s8 / more than 2000ms on Samsung galaxy z flip 3. |
1
The text was updated successfully, but these errors were encountered: