tflite import broken on nvidia-tensorflow

[markostam]

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): no
• OS Platform and Distribution (e.g., Linux Ubuntu 20.04): Ubuntu 20.04 LTS
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): nvidia-TF 1.15.5
• Python version: 3.8.2
• Bazel version (if compiling from source):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version: 11.1 / 8.0.1
• GPU model and memory: RTX 3090 24GB

Describe the current behavior

nvidia-tensorflow errors out when importing tflite.

Describe the expected behavior

Not erroring out when importing tflite.

Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.

$ python3 -m virtualenv testvenv
$ source testvenv/bin/activate
$ pip install nvidia-pyindex
$ pip install nvidia-tensorflow
$ python
Python 3.8.2 (default, Jul 16 2020, 14:00:26)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'1.15.5'
>>> tf.lite
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/marko/testvenv/lib/python3.8/site-packages/tensorflow_core/python/util/module_wrapper.py", line 193, in __getattr__
    attr = getattr(self._tfmw_wrapped_module, name)
AttributeError: module 'tensorflow' has no attribute 'lite'

Other info / logs

This import is tested and works as expected in the standard pip tensorflow-gpu releases 1.15.0, 1.15.4, 1.15.5.

[jeremiahrose]

I am also having this same issue when using the nvcr.io/nvidia/tensorflow:21.05-tf1-py3 Docker image.

I'm trying to figure out how to recompile nvidia-tensorflow with lite enabled using the nvbuild.sh script in the Docker image. Is there a simple Bazel flag that makes sure tflite is included in the nvidia-tensorflow Python bindings?

[nouiz]

tflite was removed in our TF1 in 2020 to allow us to backport more upstream changes.
You can try older container and see if it have tflite. I think nvcr.io/nvidia/tensorflow:20.06-tf1-py3 was the last container with it.

[jeremiahrose]

Hi @nouiz , thanks for your reply. That's a shame to hear that TFLite has been intentionally removed, especially since there are some downstream projects that rely on it (e.g, Coqui STT uses this nvidia image but are moving exclusively to TFLite).

Using an older container is a good workaround, but unfortunately nullifies the key advantage of using nvidia Docker images in the first place i.e up-to-date driver support.

Which brings us back to my previous question - how could I alter (/recompile) one of the newer containers to re-enable TFLite support? Is this easily done?

[nouiz]

It was removed only for TF1. If you use our TF2 container, then TFLite is available.
The TFLite code was removed from our TF1 container to allow backporting some upstream changes. It wasn't possible to backport them without removing TFLite. The codebase changed too much.
So recompiling our TF source code in the container won't help you.
You need to download another TF version and compile it. This isn't something we support.

Is moving to TF2 possible for you? Our TF2 container have TFLite. Now many people are moving or have moved to TF2.

[jeremiahrose]

Thanks @nouiz. I believe Coqui have downgraded to the old nVidia image and will stick with that until TF2 is working.