Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tflite] add xnnpack delegate to label_image #37890

Conversation

freedomtan
Copy link
Contributor

rebase and resubmit #36749 to see if it works.

The XNNPACK Delegate uses XNNPACK, a fairly optimized floating point library, to run some inference operators (see the delegate's README for currently supported ops). With XNNPACK, I was able to get performance numbers similar to what @Maratyszcza described at XNNPACK's readme. I also got good numbers on Pixel 4 and Oppo Reno 3. Numbers on x86 machines are also good.

  • Single-threaded, label_image -m MODEL_NAME -x 1 -c 50 -t 1
Model Pixel 3a, ms Pixel 4, ms Oppo Reno 3, ms
MobileNet v1 1.0X 88.02 29.95 36.23
MobileNet v2 1.0X 55.13 19.13 21.69
MobileNet v3 Large 44.84 15.69 17.65
MobileNet v3 Small 14.23 5.58 5.66
  • multi-threaded, the number of threads is set to be the number of big cores. label_image -m MODEL_NAME -x 1 -c 50 -t NUMBER_OF_BIG_CORES
Model Pixel 3a, ms Pixel 4, ms Oppo Reno 3, ms
MobileNet v1 1.0X 46.00 10.60 13.23
MobileNet v2 1.0X 28.57 6.89 7.56
MobileNet v3 Large 24.62 6.52 6.88
MobileNet v3 Small 8.24 2.47 2.37

rebase and resubmit tensorflow#36749 to see if it works.
@tensorflow-bot tensorflow-bot bot added the size:M CL Change Size: Medium label Mar 25, 2020
@gbaned gbaned self-assigned this Mar 25, 2020
@gbaned gbaned added the comp:lite TF Lite related issues label Mar 25, 2020
@gbaned gbaned added this to Assigned Reviewer in PR Queue via automation Mar 25, 2020
@gbaned gbaned requested a review from lu-wang-g March 25, 2020 15:40
@lu-wang-g lu-wang-g requested review from multiverse-tf and removed request for lu-wang-g March 25, 2020 17:34
@gbaned gbaned added the awaiting review Pull request awaiting review label Mar 27, 2020
PR Queue automation moved this from Assigned Reviewer to Approved by Reviewer Apr 1, 2020
@tensorflow-bot tensorflow-bot bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Apr 1, 2020
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Apr 1, 2020
@gbaned gbaned added ready to pull PR ready for merge process and removed awaiting review Pull request awaiting review ready to pull PR ready for merge process labels Apr 1, 2020
@tensorflow-copybara tensorflow-copybara merged commit e4efc51 into tensorflow:master Apr 2, 2020
PR Queue automation moved this from Approved by Reviewer to Merged Apr 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes comp:lite TF Lite related issues ready to pull PR ready for merge process size:M CL Change Size: Medium
Projects
PR Queue
  
Merged
Development

Successfully merging this pull request may close these issues.

None yet

6 participants