[tflite] add xnnpack delegate to label_image

[freedomtan]

The XNNPACK Delegate uses XNNPACK, a fairly optimized floating point library, to run some inference operators (see the delegate's README for currently supported ops). With XNNPACK, I was able to get performance numbers similar to what @Maratyszcza described at XNNPACK's readme. I also got good numbers on Pixel 4 and Oppo Reno 3. Numbers on x86 machines are also good.

• Single-threaded, label_image -m MODEL_NAME -x 1 -c 50 -t 1

Model	Pixel 3a, ms	Pixel 4, ms	Oppo Reno 3, ms
MobileNet v1 1.0X	88.02	29.95	36.23
MobileNet v2 1.0X	55.13	19.13	21.69
MobileNet v3 Large	44.84	15.69	17.65
MobileNet v3 Small	14.23	5.58	5.66

• multi-threaded, the number of threads is set to be the number of big cores. label_image -m MODEL_NAME -x 1 -c 50 -t NUMBER_OF_BIG_CORES

Model	Pixel 3a, ms	Pixel 4, ms	Oppo Reno 3, ms
MobileNet v1 1.0X	46.00	10.60	13.23
MobileNet v2 1.0X	28.57	6.89	7.56
MobileNet v3 Large	24.62	6.52	6.88
MobileNet v3 Small	8.24	2.47	2.37

[srjoglekar246]

Could you add some results you observe with XNNPack on label_image, to the PR description?

[srjoglekar246]

Looks good now. Thanks!

As one last thing, could you update the README in examples/label_image, for other users to use the delegate?

Will approve once this change is in.

[gbaned]

@freedomtan Can you please check srjoglekar246's comments and keep us posted? Thanks!

[freedomtan]

@srjoglekar246 and @gbaned: thanks. I updated it.

[srjoglekar246]

@gbaned Can you help get this in with the new change?

[gunan]

I am not sure how it passed the presubmits, but it looks like the build on macos is now failing after this cl:

ERROR: /Volumes/BuildData/tmpfs/tensor_flow/tensorflow/lite/tools/evaluation/stages/BUILD:171:1: Couldn't build file tensorflow/lite/tools/evaluation/stages/inference_profiler_stage_test: Linking of rule '//tensorflow/lite/tools/evaluation/stages:inference_profiler_stage_test' failed (Exit 1)
ld: warning: option -s is obsolete and being ignored
Undefined symbols for architecture x86_64:
  "__cvtu32_mask16", referenced from:
      _xnn_f32_clamp_ukernel__avx512f in libavx512f_ukernels.a(avx512f_f5094ccab65aa097758ad595fc013aa6.o)
      _xnn_f32_dwconv_ukernel_up16x25__avx512f in libavx512f_ukernels.a(up16x25-avx512f_858bc72ba4ab0c22a96b33f8c986da4d.o)
      _xnn_f32_dwconv_ukernel_up16x4__avx512f in libavx512f_ukernels.a(up16x4-avx512f_bda6330d2e7849b699d64546d5554778.o)
      _xnn_f32_dwconv_ukernel_up16x9__avx512f in libavx512f_ukernels.a(up16x9-avx512f_b7bac09f3bd6e9da99c3bd67f089f13b.o)
      _xnn_f32_gemm_ukernel_1x16__avx512f_broadcast in libavx512f_ukernels.a(1x16-avx512f-broadcast_9b0d6ef6fa981981cc969d9d4c7172fe.o)
      _xnn_f32_gemm_ukernel_7x16__avx512f_broadcast in libavx512f_ukernels.a(7x16-avx512f-broadcast_453776902dcf985bc6b1450a4b846b2e.o)
      _xnn_f32_hswish_ukernel__avx512f_x32 in libavx512f_ukernels.a(avx512f-x32_f8deb97e68fad5860a602672bfd8cdf3.o)
      ...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I will revert this for now. we can then look into why the presubmits here did not catch the problem.

[freedomtan]

@gunan How do I reproduce the error? I guess some recent XNNPACK related changes broke it. With my freedomtan:add_xnnpack_to_label_image branch, I can build tensorflow/lite/tools/evaluation/stages/inference_profiler_stage_test, tensorflow/lite/examples/label_image:label_image, and others. It also works after rebasing my branch to recent 690c33e.

[srjoglekar246]

@Maratyszcza FYI

[Maratyszcza]

I suspect the issue is due to Apple Clang lacking support for _cvtu32_mask16 AVX512 intrinsic. XNNPACK has a polyfill for this case, but it is enabled only for Apple Clang pre-10, while it seems that Apple Clang 10 also lacks _cvtu32_mask16. Presumed fix landed in google/XNNPACK@b187835