null pointer dereference Error in TF2.3.0 with runforMultipleInputOutput

[DeepakG19]

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Android
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Samsung A51
TensorFlow installed from (source or binary): Maven
TensorFlow version (use command below): implementation('org.tensorflow:tensorflow-lite:2.3.0'){changing=true}
Python version: n/a
Bazel version (if compiling from source): n/a
GCC/Compiler version (if compiling from source): n/a
CUDA/cuDNN version: n/a
GPU model and memory: n/a

Describe the current behavior

.tflite generated using tf.lite.TFLiteConverter.from_saved_model
tf_version = 2.3.0
Python Implementation for Inference works without errors.

using the same model on Android for inference gives the error-
2020-09-24 07:31:59.732 17121-17121/? A/DEBUG: Cause: null pointer dereference

Debugging Log -

Model Input Tensor Details
2020-09-24 07:31:59.284 16761-16761/Processor: hws2:0 2 Input SHAPE- 0
2020-09-24 07:31:59.285 16761-16761/Processor: mask:0 1 1 1 1 Input SHAPE- 1
2020-09-24 07:31:59.285 16761-16761/Processor: image:0 1 1 1 3 Input SHAPE- 2
2020-09-24 07:31:59.285 16761-16761/Processor: hws:0 2 Input SHAPE- 3

2020-09-24 07:31:59.285 16761-16761/Processor: strided_slice_1:0 1 1 3 Output SHAPE- 0

Model Reallocating the Input Tensor
after :
tflite.resizeInput(1,dim);
tflite.resizeInput(2,dim);
tflite.allocateTensors();

Model Input Tensor Details After Reallocation
2020-09-24 07:31:59.286 16761-16761/Processor: hws2:0 2 Input SHAPE- 0
2020-09-24 07:31:59.286 16761-16761/Processor: mask:0 1 256 256 1 Input SHAPE- 1
2020-09-24 07:31:59.286 16761-16761/Processor: image:0 1 256 256 3 Input SHAPE- 2
2020-09-24 07:31:59.286 16761-16761/Processor: hws:0 2 Input SHAPE- 3

2020-09-24 07:31:59.286 16761-16761/Processor: strided_slice_1:0 1 1 3 Output SHAPE- 0

Notice : Output Shape doesnt changes, which I assume is the correct behavior

On Running :
tflite.runForMultipleInputsOutputs(inputs, outputs);

inputs and outputs are properly initialized and non-null
This error comes -

2020-09-24 07:31:59.442 16761-16761/com.package.deepak A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 16761 (service.deepak), pid 16761 (service.deepak)

2020-09-24 07:31:59.731 17121-17121/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2020-09-24 07:31:59.732 17121-17121/? A/DEBUG: Build fingerprint: Samsung-A50
2020-09-24 07:31:59.732 17121-17121/? A/DEBUG: Revision: '2'
2020-09-24 07:31:59.732 17121-17121/? A/DEBUG: ABI: 'arm64'
2020-09-24 07:31:59.732 17121-17121/? A/DEBUG: pid: 16761, tid: 16761, name: service.deepak >>> com.package.deepak <<<
2020-09-24 07:31:59.732 17121-17121/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
2020-09-24 07:31:59.732 17121-17121/? A/DEBUG: Cause: null pointer dereference

2020-09-24 07:41:50.415 18015-18015/? A/DEBUG: backtrace:
2020-09-24 07:41:50.415 18015-18015/? A/DEBUG: #00 pc 000000000001dd6c /system/lib64/libc.so (memcpy+124)
2020-09-24 07:41:50.415 18015-18015/? A/DEBUG: #1 pc 0000000000133560 /data/app/com.package.deepak-fuEaz_w7MUZ9fy7vI4iAfA==/lib/arm64/libtensorflowlite_jni.so
2020-09-24 07:41:50.415 18015-18015/? A/DEBUG: #2 pc 00000000001331e8 /data/app/com.package.deepak-fuEaz_w7MUZ9fy7vI4iAfA==/lib/arm64/libtensorflowlite_jni.so
2020-09-24 07:41:50.415 18015-18015/? A/DEBUG: #3 pc 00000000001b269c /data/app/com.package.deepak-fuEaz_w7MUZ9fy7vI4iAfA==/lib/arm64/libtensorflowlite_jni.so
2020-09-24 07:41:50.415 18015-18015/? A/DEBUG: #4 pc 00000000001b546c /data/app/com.package.deepak-fuEaz_w7MUZ9fy7vI4iAfA==/lib/arm64/libtensorflowlite_jni.so
2020-09-24 07:41:50.415 18015-18015/? A/DEBUG: #5 pc 0000000000046738 /data/app/com.package.deepak-fuEaz_w7MUZ9fy7vI4iAfA==/lib/arm64/libtensorflowlite_jni.so (Java_org_tensorflow_lite_NativeInterpreterWrapper_run+32)
2020-09-24 07:41:50.415 18015-18015/? A/DEBUG: #6 pc 0000000000563be0 /system/lib64/libart.so (art_quick_generic_jni_trampoline+144)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #7 pc 000000000055ae4c /system/lib64/libart.so (art_quick_invoke_static_stub+604)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #8 pc 00000000000d04e8 /system/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+232)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #9 pc 00000000002838ac /system/lib64/libart.so (art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread*, art::ArtMethod*, art::ShadowFrame*, unsigned short, art::JValue*)+344)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #10 pc 000000000027d8b4 /system/lib64/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*)+968)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #11 pc 000000000052b750 /system/lib64/libart.so (MterpInvokeStatic+204)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #12 pc 000000000054d394 /system/lib64/libart.so (ExecuteMterpImpl+14612)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #13 pc 000000000021fcf4 /dev/ashmem/dalvik-classes.dex extracted in memory from /data/app/com.package.deepak-fuEaz_w7MUZ9fy7vI4iAfA==/base.apk_17926_17926 (deleted) (org.tensorflow.lite.NativeInterpreterWrapper.run+156)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #14 pc 00000000002575b8 /system/lib64/libart.so (_ZN3art11interpreterL7ExecuteEPNS_6ThreadERKNS_20CodeItemDataAccessorERNS_11ShadowFrameENS_6JValueEb.llvm.1037722801+488)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #15 pc 000000000025d0ac /system/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*)+216)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #16 pc 000000000027d898 /system/lib64/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*)+940)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #17 pc 000000000052a24c /system/lib64/libart.so (MterpInvokeVirtual+588)
2020-09-24 07:41:50.416 18015-18015/? A/DEBUG: #18 pc 000000000054d214 /system/lib64/libart.so (ExecuteMterpImpl+14228)
2020-09-24 07:41:50.417 18015-18015/? A/DEBUG: #19 pc 000000000021f2fe /dev/ashmem/dalvik-classes.dex extracted in memory from /data/app/com.package.deepak-fuEaz_w7MUZ9fy7vI4iAfA==/base.apk_17926_17926 (deleted) (org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs+10)
2020-09-24 07:41:50.417 18015-18015/? A/DEBUG: #20 pc 00000000002575b8 /system/lib64/libart.so (_ZN3art11interpreterL7ExecuteEPNS_6ThreadERKNS_20CodeItemDataAccessorERNS_11ShadowFrameENS_6JValueEb.llvm.1037722801+488)
2020-09-24 07:41:50.417 18015-18015/? A/DEBUG: #21 pc 000000000051aae0 /system/lib64/libart.so (artQuickToInterpreterBridge+1020)
2020-09-24 07:41:50.417 18015-18015/? A/DEBUG: #22 pc 0000000000563cfc /system/lib64/libart.so (art_quick_to_interpreter_bridge+92)
2020-09-24 07:41:50.417 18015-18015/? A/DEBUG: #23 pc 0000000000019ba4 /dev/ashmem/dalvik-jit-code-cache_17926_17926 (deleted) (com.package.deepak.Processor.process+12052)
2020-09-24 07:41:50.417 18015-18015/? A/DEBUG: #24 pc 000000000055aedc /system/lib64/libart.so (art_quick_osr_stub+44)

Describe the expected behavior
the Android code should run.

Ps - Reproducible Code/Model is not available due to confidentiality reasons.

[jdduke]

Can you provide a link to the model? It's possible this model has dynamically shaped outputs, which means the output shape depends on input values. In which case, the output tensor's shape can't be known until you run evaluation. We're working on making this easier to support with Java, where you can do something like:

Feed Input tensor
Run inference
Fetch output tensor

Rather than providing input/output buffers in a single invocation.

[DeepakG19]

Can you provide a link to the model? It's possible this model has dynamically shaped outputs, which means the output shape depends on input values. In which case, the output tensor's shape can't be known until you run evaluation. We're working on making this easier to support with Java, where you can do something like:

Feed Input tensor
Run inference
Fetch output tensor

Rather than providing input/output buffers in a single invocation.

The model is custom made and might not be possible to share.
Further, as I know how the network will modify the output shape (equal to input image), i am allocating a buffer of that size before inference call.
Shape of output = Shape of input with name='image'
Now the input can have varied dimensions. MODEL is converted using (None,None) parameter.

PS - the same model works fine in Python.

[jdduke]

Can you attach the code you're using to allocate your inputs/outputs, and their respective (Java) buffers?

[DeepakG19]

Can you attach the code you're using to allocate your inputs/outputs, and their respective (Java) buffers?

float[] image_ = getPixelData(image); Log.d(TAG, "Image Float Array "+image_.length+" : "+image_[0]); ByteBuffer imageBuffer = ByteBuffer.allocateDirect(4 * modelH * modelW * 3); imageBuffer.rewind(); imageBuffer.order(ByteOrder.nativeOrder()); for (int i = 0; i < image_.length; i++) { imageBuffer.putFloat((float) image_[i]); } Log.d(TAG, "Image Buffer Array "+imageBuffer.getFloat(0)+" : "+imageBuffer.capacity());

private static float[] getPixelData(Bitmap imageBitmap) { if (imageBitmap == null) { return null; } int width = imageBitmap.getWidth(); int height = imageBitmap.getHeight(); int inputSize = 256; int[] pixels = new int[width * height]; float[] floatValues = new float[width * height * 3]; imageBitmap.getPixels(pixels, 0, imageBitmap.getWidth(), 0, 0, imageBitmap.getWidth(), imageBitmap.getHeight()); int pixel = 0, k = 0; for (int i = 0; i < height; ++i) { for (int j = 0; j < width; ++j) { final int val = pixels[pixel++]; floatValues[k++] = (float) ((val >> 16) & 0xFF);// - (float) 123.68; floatValues[k++] = (float) ((val >> 8) & 0xFF);// - (float) 116.779; floatValues[k++] = (float) (val & 0xFF);// - (float) 103.939; } } return floatValues; }

PS - I dont know why is it showing in single line in preview, while formatting seems fine in EDIT window

[DeepakG19]

@jdduke any update

[Saduf2019]

@DeepakG19
Could you please try on the latest version of tf and let us know if this is still an issue.

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No