TFlite android inference produces slightly different results for the same input, is it normal?

[catqaq]

Click to expand!

Issue Type

Bug

Source

binary

Tensorflow Version

2.3.0

Custom Code

Yes

OS Platform and Distribution

windows10

Mobile device

android

Python version

3.6

Bazel version

4.2.1

GCC/Compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current Behaviour?

I use tflite android api to get sentence embeddings from my tflite model.
`Map<Integer, Object> outputs = new HashMap<>();
outputs.put(0, embeddings);
tflite.runForMultipleInputsOutputs(new Object[]{inputIds, attentionMask}, outputs);`
TFlite android inference produces slightly different results for the same input, is it normal? What causes the small differences？
query: -0.03889307#-0.20992874#-0.012840451...
query: -0.038892966#-0.20992874#-0.012840495...
query: -0.03889298#-0.20992877#-0.012840421...

Standalone code to reproduce the issue

The code is inconvenient to share, I just want to know if this phenomenon is normal? If it is not normal, is there a way to avoid it?

Relevant log output

No response

[mohantym]

Hi @catqaq !
Results does not look deterministic from above results. Probably you are using float64 datatype.
Could you give it a try in Python 3.7/3.8 (Python 3.7 minimum version now) , Bazel 5.0.0 and TF 2.9 and let us know whether the issue still persists.

Thank you!

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[catqaq]

Hi, @mohantym, sorry for replying so late. I follow your demo code to get output from tflite model. Is there anything to related with float64?
float[][] embedding = new float[1][64]; Map<Integer, Object> outputs = new HashMap<>(); outputs.put(0, embeddings); tflite.runForMultipleInputsOutputs(new Object[]{inputIds, attentionMask}, outputs);

As for your advice about newer version, I will try it later.

[catqaq]

About newer versions, I tried python3.7 - tf 2.8 - bazel4.2.1, the issue still persists. I can't user newer bazel limited by network issues.

[mohantym]

No @catqaq !
Just guessed the datatype from your output . Yes! Please test with Python 3.7/3.8 , Bazel 4.2.1 and TF 2.7 and let us know.
Thank you!