TFLite have not support SquaredDifference/TFLite_Detection_PostProcess?

[aifollower]

Describe the problem

2018-08-07 13:21:07.345473: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1053] Converting unsupported operation: SquaredDifference
2018-08-07 13:21:07.361241: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1053] Converting unsupported operation: TFLite_Detection_PostProcess

[tensorflowbutler]

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
What is the top-level directory of the model you are using
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

[achowdhery]

Please provide the sample frozen graph for the model you are trying to convert and instructions to reproduce.
The custom op TFLite_Detection_PostProcess does exist in latest Tensorflow code : tensorflow/tensorflow/contrib/lite/kernels/detection_postprocess.cc

[kate-kate]

@achowdhery Hi, I have same problem with TFLite_Detection_PostProcess running this command from here

bazel run --config=opt tensorflow/contrib/lite/toco:toco -- \
--input_file=$OUTPUT_DIR/tflite_graph.pb \
--output_file=$OUTPUT_DIR/detect.tflite \
--input_shapes=1,300,300,3 \
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3'  \
--inference_type=FLOAT \
--allow_custom_ops

I used ssd_mobilenet_v1_coco pretrained model. I created frozen graph with export_tflite_ssd_graph.py command. At first I've tried running toco command for quantized model, as it was described in article Training and serving a realtime mobile object detector and faced the same error.
Thank you for help. Can provide my frozen graph also

[achowdhery]

@kate-kate The steps in the second part of the tutorial do not require SquaredDifference. The other custom op TFLite_Detection_PostProcess is already present in the Tensorflow code base. So, it will give a warning but will work properly to give the detect.tflite file.

What version of Tensorflow are you using?

[kate-kate]

@achowdhery I use 1.9.0 built from source
I also checked the file responsible for this op - it is really present. But as I also noticed in #5019 even if the command creates tflite file, then in Android is throws an error "Cannot create interpreter: Didn't find custom op for name 'TFLite_Detection_PostProcess'". So that's why I've decided the problem was during conversion. Model from ml kit tutorial works well(

[achowdhery]

@kate-kate Please try tensorflow 1.10 or build tensorflow from head - the code was not released in tensorflow 1.9.0

[kate-kate]

@achowdhery tried with tensorflow 1.10 built from source
Command output in console

INFO: Build completed successfully, 316 total actions
INFO: Running command line: bazel-bin/tensorflow/contrib/lite/toco/toco '--input_file=/Users/kate/python/codes2/tflite_res/tflite_graph.pb' '--output_file=/Users/kate/python/codes2/tflite_res/detect4.tflite' '--input_shapes=1,300,300,3' '--input_arrays=normalized_input_image_tensor' '--output_arrays=TFLite_DetectioINFO: Build completed successfully, 316 total actions
2018-08-12 11:57:39.906797: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1055] Converting unsupported operation: TFLite_Detection_PostProcess
...

Error while running on Android (via Firebase ML Kit)

08-12 12:10:05.891 10381-10381/com.google.firebase.codelab.mlkit_custommodel W/System.err: Caused by: com.google.firebase.ml.common.FirebaseMLException: Internal error has occurred when executing Firebase ML tasks
08-12 12:10:05.896 10381-10381/com.google.firebase.codelab.mlkit_custommodel W/System.err:     at com.google.android.gms.internal.firebase_ml.zzgp.zza(Unknown Source:15)
    	... 5 more
    Caused by: java.lang.IllegalArgumentException: Cannot create interpreter: Didn't find custom op for name 'TFLite_Detection_PostProcess'
    Registration failed.
        at org.tensorflow.lite.NativeInterpreterWrapper.createInterpreter(Native Method)

[achowdhery]

@kate-kate Thanks for updating us that there is some incompatibility with ML Kit atm. Are you able to try with bazel build and Android sdk/ndk ? We have made a Dockerfile available to make installation easier https://github.com/tensorflow/models/blob/master/research/object_detection/dockerfiles/android/Dockerfile

[kate-kate]

@achowdhery I've tried with bazel build and Android sdk/ndk. First, I've tried it without docker and faced an error like C++ compilation of rule '@double_conversion//:double-conversion' failed (Exit 1). Then I've decided to use your Dockerfile and after executing bazel build -c opt --local_resources 4096,4.0,1.0 -j 1 //tensorflow/examples/android:tensorflow_demo command that I found in this article it threw me an error

ERROR: /tensorflow/tensorflow/compiler/xla/service/cpu/BUILD:472:1: C++ compilation of rule '//tensorflow/compiler/xla/service/cpu:runtime_conv2d_mkl' failed (Exit 1)
In file included from tensorflow/compiler/xla/service/cpu/runtime_conv2d_mkl.cc:17:
In file included from ./tensorflow/compiler/xla/executable_run_options.h:20:
In file included from ./tensorflow/compiler/xla/types.h:22:
In file included from ./tensorflow/core/framework/numeric_types.h:20:
In file included from ./third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1:
In file included from external/eigen_archive/unsupported/Eigen/CXX11/Tensor:129:
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorBroadcasting.h:108:15: warning: in-class initialization of non-static data member is a C++11 extension [-Wc++11-extensions]
  bool nByOne = false, oneByN = false;

Then I've tried to run it again with --verbose_failures option and error description was

ERROR: /root/.cache/bazel/_bazel_root/68a62076e91007a7908bc42a32e4cff9/external/highwayhash/BUILD.bazel:23:1: C++ compilation of rule '@highwayhash//:arch_specific' failed (Exit 1): clang failed: error executing command
  (cd /root/.cache/bazel/_bazel_root/68a62076e91007a7908bc42a32e4cff9/execroot/org_tensorflow && \
  exec env - \
    ANDROID_BUILD_TOOLS_VERSION=27.0.3 \
    ANDROID_NDK_API_LEVEL=14 \
    ANDROID_NDK_HOME=/opt/android-ndk-r14b \
    ANDROID_SDK_API_LEVEL=27 \
    ANDROID_SDK_HOME=/opt/android-sdk-linux \
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/android-sdk-linux/tools:/opt/android-sdk-linux/tools/bin:/opt/android-sdk-linux/platform-tools \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/usr/bin/python \
    PYTHON_LIB_PATH=/usr/lib/python2.7/dist-packages \
    TF_DOWNLOAD_CLANG=0 \
    TF_NEED_CUDA=0 \
    TF_NEED_OPENCL_SYCL=0 \
  external/androidndk/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -gcc-toolchain external/androidndk/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -Wno-invalid-command-line-argument -Wno-unused-command-line-argument -no-canonical-prefixes -fno-integrated-as -target armv7-none-linux-androideabi '-march=armv7-a' '-mfloat-abi=softfp' '-mfpu=vfpv3-d16' -mthumb -Os -g -DNDEBUG -MD -MF bazel-out/android-armeabi-v7a-opt/bin/external/highwayhash/_objs/arch_specific/external/highwayhash/highwayhash/arch_specific.d '-frandom-seed=bazel-out/android-armeabi-v7a-opt/bin/external/highwayhash/_objs/arch_specific/external/highwayhash/highwayhash/arch_specific.o' -iquote external/highwayhash -iquote bazel-out/android-armeabi-v7a-opt/genfiles/external/highwayhash -iquote external/bazel_tools -iquote bazel-out/android-armeabi-v7a-opt/genfiles/external/bazel_tools '--sysroot=external/androidndk/ndk/platforms/android-14/arch-arm' -isystem external/androidndk/ndk/sources/cxx-stl/gnu-libstdc++/4.9/include -isystem external/androidndk/ndk/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/include -isystem external/androidndk/ndk/sources/cxx-stl/gnu-libstdc++/4.9/include/backward -c external/highwayhash/highwayhash/arch_specific.cc -o bazel-out/android-armeabi-v7a-opt/bin/external/highwayhash/_objs/arch_specific/external/highwayhash/highwayhash/arch_specific.o)
In file included from external/highwayhash/highwayhash/arch_specific.cc:15:
external/highwayhash/highwayhash/arch_specific.h:115:20: warning: alias declarations are a C++11 extension [-Wc++11-extensions]
using TargetBits = unsigned;
                   ^
external/highwayhash/highwayhash/arch_specific.cc:37:14: error: use of undeclared identifier 'nullptr'
      return nullptr;  // zero, multiple, or unknown bits
             ^
1 warning and 1 error generated.
Target //tensorflow/examples/android:tensorflow_demo failed to build

[jdduke]

Hi @kate-kate, with your custom TFLite build, are you using these instructions for pointing MLKit to that build? The 1.10 TFLite release should be out shortly, and MLKit will likely migrate shortly after that.

[jefby]

@kate-kate You can git checkout r1.10, and i build with command

bazel run --config=opt tensorflow/contrib/lite/toco:toco -- \
--input_file=$OUTPUT_DIR/tflite_graph.pb \
--output_file=$OUTPUT_DIR/detect.tflite \
--input_shapes=1,300,300,3 \
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3'  \
--inference_type=FLOAT \
--allow_custom_ops

works