Python Wheel generation of TensorFlow Lite 2.17 for ARMv7l 32 bits not working

[paguilar-pxc]

I have previously crosscompiled different versions of TensorFlow lite (2.14, 2.15.1, 2.16.2) for Python 3.10 using CMAKE and following the instructions from the website .

So far the only changes required were to just set my armhf flags as:

echo "ARMCC_FLAGS=\"-march=armv7-a -mfpu=neon-vfpv3 -funsafe-math-optimizations \
adjust the python version in the MakeFile (tensorflow/lite/tools/pip_package/Makefile) and run the make command as:

make -C tensorflow/lite/tools/pip_package docker-build \ TENSORFLOW_TARGET=armhf PYTHON_VERSION=3.10
Now, I have tried the same approach for the release 2.17.0 ( ad6d8cc ) and although the build is executed and the wheel is generated without errors, I keep getting the following errorat the moment of importing the interpreter:

python3 simpletest.py 
Traceback (most recent call last):
  File "/mnt/simpletest.py", line 2, in <module>
    import tflite_runtime.interpreter as tflite
  File "/usr/local/lib/python3.10/dist-packages/tflite_runtime/interpreter.py", line 33, in <module>
    from tflite_runtime import _pywrap_tensorflow_interpreter_wrapper as _interpreter_wrapper
ImportError: /usr/local/lib/python3.10/dist-packages/tflite_runtime/_pywrap_tensorflow_interpreter_wrapper.so: undefined symbol: TfLiteXNNPackDelegateOptionsDefault

[gaikwadrahul8]

Hi, @paguilar-pxc

I apologize for the delayed response, if possible could you please help us with exact steps which you followed to replicate the same behavior from our end to investigate this issue further from our end ?

Thank you for your cooperation and patience.

[paguilar-pxc]

Hello @gaikwadrahul8,

The steps are not that different from my original comment. I write here a more detailed description:

1. I downloaded the source code from the latest stable release (2.17) (https://github.com/tensorflow/tensorflow/releases/tag/v2.17.0

2. Extract the content to a folder.

3. Navigate to "Makefile" in tensorflow-2.17.0/tensorflow/lite/tools/pip_package/Makefile and modify the base image, python version and numpy version to match those that I will be using

5. Then I proceed to navigate to "download_toolchains.sh" located in tensorflow-2.17.0/tensorflow/lite/tools/cmake/download_toolchains.sh and modify the flags that come by default for armh

6. Finally, from within the folder tensorflow-2.17.0 I run the command make -C tensorflow/lite/tools/pip_package docker-build \ TENSORFLOW_TARGET=armhf PYTHON_VERSION=3.10

7. After the process is finished, I take the Python wheel from tensorflow-2.17.0/tensorflow/lite/tools/pip_package/gen/tflite_pip/python3.10/dist to my device

8. From the device, it doesnt really matter what I try to run because I cannot even import the interpreter (see message in original post).

SOME EXTRA INFORMATION: The target device is not a raspberry pi (hence the tweak of the flags). I have attempted further variations with exactly the same results (wheel succesfully built but cannot import the interpreter in the edge device). Some of the variants I tried were:

a. Using the latest source code instead of the stable release. Same results.
b. Using a different base image (Ubuntu 20.04). Same results.
c. Using different versions of numpy. Same results
d. Specifying further flags that I know are present in my target hardware. Same results.
e. Setting the cross compilation for Python 3.11 instead of 3.10. Same results.
f. Trying to import the interpreter from the source file instead of using simply the python command. Same results.
g.

Due to the impossibility of getting it to work, I resorted to ask here for support. Notice that, just as I mentioned before, the process had always worked flawlessly for previous versions (both for Python 3.10 and 3.11) and only with the 2.17 and the up-to-date code is that I get that error with the interpreter.

[gaikwadrahul8]

Hi, @paguilar-pxc

Thank you for providing the detailed steps to replicate similar behavior from our end, I see the wheel file is generating the successfully in the dist folder for reference I've added output log but while importing the interpreter I'm also getting the same error message so will have to dig more into this issue

(tf-test-1) gaikwadrahul@gaikwadrahul-n1-standard-1-gpu-t4x1-tflite-ubuntu-24:~/issue-75562/tensorflow-2.17.0/tensorflow/lite/tools/pip_package/gen/tflite_pip/python3.10/dist$ ls
tflite_runtime-2.17.0-cp310-cp310-linux_armv7l.whl  tflite_runtime-2.17.0.linux-armv7l.tar.gz
(tf-test-1) gaikwadrahul@gaikwadrahul-n1-standard-1-gpu-t4x1-tflite-ubuntu-24:~/issue-75562/tensorflow-2.17.0/tensorflow/lite/tools/pip_package/gen/tflite_pip/python3.10/dist$ 

Thank you for your cooperation and patience.

[paguilar-pxc]

Hello @gaikwadrahul8,

thank you for the feedback. It is good to see that you got to the same result when following my steps.

I keep looking forward to your continued progress in finding solution or workaround. 👍

Thank you.