Can't allocate memory for the interpreter in tflite

[shiftsayan]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Nope, using tensorflow-for-poets
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS High Sierra
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): 1.7.1
• Python version: Python 2.7.10
• Bazel version (if compiling from source): N/A
• GCC/Compiler version (if compiling from source): N/A
• CUDA/cuDNN version: N/A
• GPU model and memory: N/A
• Exact command to reproduce: I added a custom tflite model (converted .pb using TOCO) and replaced graph.lite with the new custom model and built the app, but it crashes on runtime.

Detailed Description

I created a custom TensorFlow model and converted it to TOCO (as described in the tensorflow-for-poets tutorial) and I replaced the old graph.lite file with my custom model and changed nothing else in the code. When I run the app, I get the following runtime error:

Process: android.example.com.tflitecamerademo, PID: 29160
    java.lang.RuntimeException: Unable to start activity ComponentInfo{android.example.com.tflitecamerademo/com.example.android.tflitecamerademo.CameraActivity}: java.lang.NullPointerException: Can not allocate memory for the interpreter

Fixes Already Tried

• Ensuring that Android Studio/Gradle pick up on tensorflow-lite:0.1.7 changes (After tensorflow-lite-0.1.7 update Image classification using inception-v3 model stops working #19051). Currently, my build.gradle file says compile 'org.tensorflow:tensorflow-lite:+', but changing it to compile 'org.tensorflow:tensorflow-lite:0.1.7' doesn't resolve the issue either.
• Running TOCO with the --change_concat_input_ranges=false flag.

[alanchiao]

Hey Sayan, thanks for following up on this.

Could you possibly update the build.gradle file to 'org.tensorflow:tensorflow-lite:0.0.0-nightly' and see if it fixes your issue? It contains a much more recent version of the TFLite library (built every day). Let me know if it does. I have a fix coming up that makes that the default.

[shiftsayan]

Hey Alan, thanks for taking up the issue!
When I make the change you suggested, I get the following error message: Multiple dex files define Lorg/tensorflow/lite/NativeInterpreterWrapper;, but I am not sure if this has something to do with the fact that I am running the nightly build.

[alanchiao]

I'll try to reproduce it locally - it could have something to do with updating the Gradle file.

#7470 seems relevant

[shiftsayan]

I tried compiling and running the app again today with the nightly build and I get the following error message now:
java.lang.RuntimeException: Unable to start activity ComponentInfo{android.example.com.tflitecamerademo/com.example.android.tflitecamerademo.CameraActivity}: java.lang.NullPointerException: Internal error: Cannot allocate memory for the interpreter: tensorflow/contrib/lite/kernels/conv.cc:191 input->dims->size != 4 (0 != 4)

[alanchiao]

Hey Sayan! There seems to have been a problem in the model conversion process (ex: TOCO) as opposed to the demo app.

As the error message indicates, one of your conv operators is provided with an input tensor with no dimensions.

Some Background for Debugging
For ops with dynamic inputs, we call the Prepare function (that your error appeared in) right before calling its Eval function. Suppose op A feeds its output into op B through tensor C. The Prepare call in dynamic op A will resize tensor C.

No we can suppose in your model, the conv op that's failing is op B. For some reason, op A has not resized tensor C and the conv op is seeing an input tensor with no dims.

Unfortunately, I don't currently have cycles to help debug a model-specific issue. If it's ok for you to share your frozen graph file, that'd be great for when someone gets cycles to look into it.

[tensorflowbutler]

Nagging Assignee @alanchiao: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

[sistr22]

I'm also having the issue

[tensorflowbutler]

Nagging Assignee @alanchiao: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

[karlzheng]

with the same problem.

[alanchiao]

@karlzheng: could you provide more details?
Which one of the issues that Sayan ran into are you running into?

1. "Can't allocate memory for the interpreter in tflite."
   -> Are you using the nightly build as suggested above? Please see if it fixes your issue.
2. "Multiple dex files define Lorg/tensorflow/lite/NativeInterpreterWrapper".
   -> See my response above.
3. "java.lang.RuntimeException: Unable to start activity ComponentInfo{android.example.com.tflitecamerademo/com.example.android.tflitecamerademo.CameraActivity}: java.lang.NullPointerException: Internal error: Cannot allocate memory for the interpreter: tensorflow/contrib/lite/kernels/conv.cc:191 input->dims->size != 4 (0 != 4)"

If it's a separate error message, please create a separate issue. Thanks!

[bayan100]

I had the same issue (Nr. 3). Its probably because you retrained you modell with a modified retrain.py script. To complete the tutorial I had to give a variable called input_layer another value (to solve a different issue).
When converting your model with toco you have to make sure to use the same value as parameter for input_array.
Solved the problem for me.

[alanchiao]

Thank you @bayan100 for sharing how you resolved Nr. 3!

Closing this issue since the original reporter hasn't responded for a while. @WhiteSEEKER and @karlzheng, feel free to create a separate issue and link to this one for context. If what Bayan said is true (you'd modified retrain.py), it'd be great if you mentioned it here so others know that that's the cause.

[luoolu]

java.lang.NullPointerException: Internal error: Cannot allocate memory for the interpreter: tensorflow/contrib/lite/kernels/conv.cc:191 input->dims->size != 4 (1 != 4)Node 0 failed to prepare.

how to solve?

[alanchiao]

@luolugithub : there isn't sufficient information from just that error. Does bayan100's comment apply to you?

We'd need the .tflite model file at a minimum (or something the reproduces the error), and if it's generated from a working model (e.g. via retraining via retrain.py), it'd be useful to see both the original model and the generation code. Thanks!

[minizon]

Same issue encountered. And I attached a converted model file at another issue

[CartierPierre]

Having the same issue when trying to run on Android.
Internal error: Unexpected failure when preparing tensor allocations: tensorflow/contrib/lite/kernels/conv.cc:225 input->dims->size != 4 (0 != 4)Node number 0 (CONV_2D) failed to prepare.
I used the last retrain.py from https://www.tensorflow.org/hub/tutorials/image_retraining with my own dataset.
Then converting using TOCO ./toco --input_file="$HOME/tensorflow-master/output_graph.pb" --output_file="$HOME/output_graph.tflite" --input_arrays=Placeholder --output_arrays=final_result
Then putted in the android lite demo next to mobilenet_quant.tflite.
And i don't understand what @bayan100 said with the input_layer

Edit : Tried to use mobilenet_v2_140_224 model, still the same error on my phone

[CartierPierre]

The .pb model looks good because still have 4 dimension with Placeholder but when i try to open .tflite converted mode, it doesn't find any input array. Still working on it, may be a problem with toco

[damhurmuller]

Same error here but only when Im trying to use the post-training quantization.

converter = tf.contrib.lite.TFLiteConverter.from_keras_model_file('models/mobilenet_tune.h5')
converter.post_training_quantize = True --> Without this line is Ok
tflite_model = converter.convert()
open("mobilenet_quantize_model.tflite", "wb").write(tflite_model)

[SreehariRamMohan]

@damhurmuller I'm doing the same thing as you and running into issues. Did you manage to find a solution?

[damhurmuller]

@SreehariRamMohan I was using an old version 1.10.* I guess. Now I'm using the nightly version and is ok.

[SreehariRamMohan]

@damhurmuller I switched to compiling the tf nightly version but I still get the following error:

Caused by: com.google.firebase.ml.common.FirebaseMLException: Internal error has occurred when executing Firebase ML tasks W/System.err: at com.google.android.gms.internal.firebase_ml.zzhg.zza(Unknown Source) W/System.err: ... 5 more W/System.err: Caused by: java.lang.NullPointerException: Can not allocate memory for the interpreter W/System.err: at org.tensorflow.lite.NativeInterpreterWrapper.createInterpreter(Native Method) W/System.err: at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:63) W/System.err: at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:51) W/System.err: at org.tensorflow.lite.Interpreter.<init>(Interpreter.java:90) W/System.err: at com.google.android.gms.internal.firebase_ml.zzif.zzfm(Unknown Source) W/System.err: at com.google.android.gms.internal.firebase_ml.zzhr.zzfp(Unknown Source) W/System.err: at com.google.android.gms.internal.firebase_ml.zzhr.call(Unknown Source) W/System.err: at com.google.android.gms.internal.firebase_ml.zzhg.zza(Unknown Source) W/System.err: ... 5 more

[aselle]

@alanchiao, what version of TFLite is required to use post_training_quant?

[mstc-xqp]

E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.trp.develop.tflitetrp, PID: 3763
java.lang.RuntimeException: Unable to start activity ComponentInfo{com.trp.develop.tflitetrp/com.trp.develop.tflitetrp.MainActivity}: java.lang.NullPointerException: Internal error: Cannot allocate memory for the interpreter: tensorflow/contrib/lite/kernels/concatenation.cc:67 t->dims->size != t0->dims->size (3 != 2)Node number 4 (CONCATENATION) failed to prepare.

    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2778)
    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2856)
    at android.app.ActivityThread.-wrap11(Unknown Source:0)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1589)
    at android.os.Handler.dispatchMessage(Handler.java:106)
    at android.os.Looper.loop(Looper.java:164)
    at android.app.ActivityThread.main(ActivityThread.java:6494)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
 Caused by: java.lang.NullPointerException: Internal error: Cannot allocate memory for the interpreter: tensorflow/contrib/lite/kernels/concatenation.cc:67 t->dims->size != t0->dims->size (3 != 2)Node number 4 (CONCATENATION) failed to prepare.

    at org.tensorflow.lite.NativeInterpreterWrapper.allocateTensors(Native Method)

I use tflite convert because it support multi inputs

below is the command:
tflite_convert --graph_def_file=/home/xqp/Desktop/model1/frozen_purecnn.pb --output_file=//home/xqp/Desktop/model1/purecnn.tflite --input_arrays=cnn_input,pcs_1level_input --output_arrays=predictions/Softmax --input_shapes=1,500:1,7

if i use toco it will report this error
2018-12-11 09:58:57.163903: F tensorflow/contrib/lite/toco/graph_transformations/propagate_fixed_sizes.cc:1091] Check failed: indices_shape.dimensions_count() == 1 (2 vs. 1)

below is the command:
toco --input_file=/home/xqp/Desktop/model12/frozen_purecnn.pb --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE --output_file=/home/xqp/Desktop/model12/purecnn.lite --inference_type=FLOAT --input_type=FLOAT --input_arrays=cnn_input,pcs_1level_input --output_arrays=predictions/Softmax --input_shapes=1,500:1,7

[alanchiao]

@mstc-xqp : please create a separate issue. While the error message is the same, the underlying cause is different.

@damhurmuller : that makes sense. I don't think the post_training_quantize was ready for use until the 1.11 release

[MATTYGILO]

I am having the same problem. Has anyone worked out a solution to this problem.

[alanchiao]

@MATTYGILO : could you please create a separate issue (and CC mstc-xqp if the error is exactly the same).

Note that for mstc's case, the error is for the concatenation.cc , whereas for Pierre, it's for the conv.cc. These are all model-specific and op-specific issues

[gustavz]

same problem here

[fcakir]

I encountered a similar problem when trying to convert pb to tflite using the toco converter (via bazel). I managed to get rid of the problem by passing in the input_shape argument in the command line, e.g., --input_shape=1,224,224,3. Just adding here as a reference.

[joonb14]

I tried compiling and running the app again today with the nightly build and I get the following error message now:
java.lang.RuntimeException: Unable to start activity ComponentInfo{android.example.com.tflitecamerademo/com.example.android.tflitecamerademo.CameraActivity}: java.lang.NullPointerException: Internal error: Cannot allocate memory for the interpreter: tensorflow/contrib/lite/kernels/conv.cc:191 input->dims->size != 4 (0 != 4)

In my case, I used multiple inputs. I solved a similar problem with the below instruction. Then trained the model from scratch changing input shape and training data shape.