Incompatibility with tensorflow>=2.13.0 due to flatbuffers version conflict

[stancld]

Describe the bug
Hey 👋 I admit this a bit early note, however, I found out tf2onnx will not be compatible with tensorflow==2.13 (based on their 1st release candidate) due to dependency conflict. tf2onnx currently requires flatbuffers<3.0, while tensorflow==2.13 will likely need flatbuffers>=23.1.21.

Edit:
The constraint will become even stricter for tensorflow=2.14, which will likely require flatbuffers>=23.5.26

Urgency
High (Impossible to upgrade tensorflow to versions >=2.13.0

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 18.04*): Any
• TensorFlow Version: 2.13
• Python version: 3.11

To Reproduce

$ docker run --rm -it python:3.11 bash
> pip install tensorflow==2.13.0rc0
> pip install tf2onnx

Additional context
https://github.com/tensorflow/tensorflow/releases/tag/v2.13.0-rc0

[fatcat-z]

Thanks for your reminder. We will take necessary work for future upgrade.

[seidnerj]

I am also getting this same conflict:
tensorflow-macos 2.13.0rc0 depends on flatbuffers>=23.1.21
tf2onnx 1.14.0 depends on flatbuffers<3.0 and >=1.12

Unfortunately I have to use the 2.13 RC version because tensorflow 2.12.0 has dependence on numpy<1.24 and I have another conflict because of that.

[dinghram]

Similar conflict:
Onnxruntime 1.15.0 (required for Python 3.11 upgrade) requires Numpy 1.24
Tensorflow 2.12.0 requires Numpy < 1.24, so must upgrade to 2.13 RC
Tensorflow 2.13 RC requires newer flatbuffers, making it incompatible with tf2onnx

[seidnerj]

Similar conflict:
Onnxruntime 1.15.0 (required for Python 3.11 upgrade) requires Numpy 1.24
Tensorflow 2.12.0 requires Numpy < 1.24, so must upgrade to 2.13 RC
Tensorflow 2.13 RC requires newer flatbuffers, making it incompatible with tf2onnx

My temporary solution for this was to fork tf2onnx and create a new branch based on 1.14.0, where I relaxed the requirements. You're welcome to use it in the meanwhile: https://github.com/seidnerj/tensorflow-onnx/tree/r1.14.0b

You can add it to a requirements.txt file like so:
tf2onnx @ git+https://github.com/seidnerj/tensorflow-onnx.git@r1.14.0b

Ugly, I know, but couldn't continue working without this.

[jferlez]

Similar conflict:
Onnxruntime 1.15.0 (required for Python 3.11 upgrade) requires Numpy 1.24
Tensorflow 2.12.0 requires Numpy < 1.24, so must upgrade to 2.13 RC
Tensorflow 2.13 RC requires newer flatbuffers, making it incompatible with tf2onnx

My temporary solution for this was to fork tf2onnx and create a new branch based on 1.14.0, where I relaxed the requirements. You're welcome to use it in the meanwhile: https://github.com/seidnerj/tensorflow-onnx/tree/r1.14.0b

You can add it to a requirements.txt file like so: tf2onnx @ git+https://github.com/seidnerj/tensorflow-onnx.git@r1.14.0b

Ugly, I know, but couldn't continue working without this.

Is it functional for you? I tried a similar patch, and while it allowed tf2onnx to build and install, the result was non-functional for me. In particular, it seems concrete_func.graph._captures was removed, and performing a naive substitution of the presumable replacement, concrete_func.graph.captures, does not yield correct ONNX output.

I'm far from an expert in this software, though, so I didn't try a more elaborate patch. But perhaps there is something different between our setups that explains my experience?

(FYI: I'm using flatbuffers 23.5.26 instead, if that makes a difference.)

[seidnerj]

For me it worked without a hitch, though I'm doing just the one call to tf2onnx.convert.from_keras method. Worth checking the flatbuffers version, can't tell you much more than that unfortunately.

[jferlez]

No worries, @seidnerj: any information is helpful.

I tried flatbuffers 23.5.8 with no success. The error I get on conversion is the same, and looks like this:

    onnxModel, _ = tf2onnx.convert.from_keras(nn, input_signature=(tf.TensorSpec((None, problem.n), tfDataType, name="input"),), opset=13)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/james/Library/Python/3.11/lib/python/site-packages/tf2onnx-1.14.0-py3.11.egg/tf2onnx/convert.py", line 475, in from_keras
    graph_captures = concrete_func.graph._captures  # pylint: disable=protected-access
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'FuncGraph' object has no attribute '_captures'. Did you mean: 'captures'?

The same code/NN convert without issue on python3.10, tf 2.12.0 and tf2onnx 1.14.0.

Upon some more digging, I see the release notes for tf 2.13.0 indicate that the behavior of "ConcreteFunction ... as generated through get_concrete_function" has changed, and this call is responsible for the variable concrete_func in the above. So this new behavior may be the cause of the problem, perhaps triggered by some aspect of the specific NN I'm trying to convert -- or something wrong with my code (although it seems to work in previous versions).

I'd be more than happy to provide a minimal example if that would be helpful, but I also don't want to waste developer time on a spurious or anticipated error. In any case, I will double check my code to make sure I'm not doing anything wrong there.

[jferlez]

It sounds like the developers/maintainers are aware of incompatibilities with TF 2.13.0, and a more substantial patch will be necessary. Thanks in advance for your hard work on this (and past versions of tf2onnx)!

Just for completeness, though, here's an example of code that doesn't work for Python3.11/TF 2.13.0/tf2onnx 1.14.0, the latter built using a newer version of flatbuffers (neuron weights don't seem to matter):

import tensorflow as tf
import keras
import tf2onnx

nn = keras.models.Sequential(layers=[
    keras.layers.Input(shape=(2,)), # Also fails with this layer removed
    keras.layers.Dense(1,activation=None,use_bias=True)
])
onnxModel, _ = tf2onnx.convert.from_keras(nn, input_signature=(tf.TensorSpec((None, 2), tf.float32, name="input"),), opset=14)

from_keras fails to produce an onnx model with the same error as above, i.e. the _captures attribute is missing.

However, the same code produces correct output with Python3.10/TF 2.12.0/tf2onnx 1.14.0.

Let me know if there's any other way I can help!

[yan12125]

I work around the _captures issue by replacing these two lines in tf2onnx/convert.py

        graph_captures = concrete_func.graph._captures  # pylint: disable=protected-access
        captured_inputs = [t_name.name for _, t_name in graph_captures.values()]

with

    try:
        graph_captures = concrete_func.graph._captures  # pylint: disable=protected-access
        captured_inputs = [t_name.name for _, t_name in graph_captures.values()]
    except AttributeError:
        graph_captures = concrete_func.graph.function_captures.by_val_internal
        captured_inputs = [t.name for t in graph_captures.values()]

This is done by checking relevant tensorflow commits:
tensorflow/tensorflow@db12f60
tensorflow/tensorflow@b92b675
tensorflow/tensorflow@488a324
I don't understand what's the meaning of those variables, though.

EDIT 2023/08/07: use try-except to keep compatibility with older Tensorflow.

[yan12125]

Probably tf2onnx tests should be fixed for tensorflow 2.12 first #2154

[yan12125]

This project is now tested with Tensorflow 1.15 and 2.8~2.13 [1]. Among those versions, Tensorflow 2.9 requires flatbuffers >= 1.12, <2 [2] and Tensorflow 2.13 wants flatbuffers >= 23.1.2 [3] (Tensorflow 2.8 has no upper bound and Tensorflow 1.15 does not require flatbuffers). As a result, a version range with both lower and upper bounds (like [4]) cannot fit both old and new Tensorflow. How about removing the upper bound for flatbuffers in this project? The dependency resolver in pip 20.3 [4] can already find suitable dependency versions according to all version constraints.

[1] https://github.com/onnx/tensorflow-onnx/blob/main/README.md
[2] https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/tools/pip_package/setup.py#L80
[3] https://github.com/tensorflow/tensorflow/blob/v2.13.0/tensorflow/tools/pip_package/setup.py#L87
[4]

tensorflow-onnx/setup.py

Line 85 in a0bd3e5

install_requires=['numpy>=1.14.1', 'onnx>=1.4.1', 'requests', 'six', 'flatbuffers<3.0,>=1.12', 'protobuf~=3.20.2'],

[5] https://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html

[stancld]

Hello @fatcat-z,

Bttw, it looks like tensorflow does not make it easier with tensorflow==2.14.0 since it requires flatbuffers >= 23.5.26 :(

Have you tried to remove the flatbuffers upper bound version in tf2oonx?

Thanks a lot for all your hard work!

[seidnerj]

This project is now tested with Tensorflow 1.15 and 2.8~2.13 [1]. Among those versions, tf 2.9 requires flatbuffers >= 1.12, <2 [2] and tf 2.13 wants flatbuffers >= 23.1.2 [3] (tf 2.8 has no upper bound and tf 1.15 does not require flatbuffers). As a result, a version range with both lower and upper bounds (like [4]) cannot fit both old and new Tensorflow. How about removing the upper bound for flatbuffers in this project? The dependency resolver in pip 20.3 [4] can already find suitable dependency versions according to all version constraints.

[1] https://github.com/onnx/tensorflow-onnx/blob/main/README.md

[2] https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/tools/pip_package/setup.py#L80

[3] https://github.com/tensorflow/tensorflow/blob/v2.13.0/tensorflow/tools/pip_package/setup.py#L87

[4]

tensorflow-onnx/setup.py

Line 85 in a0bd3e5

install_requires=['numpy>=1.14.1', 'onnx>=1.4.1', 'requests', 'six', 'flatbuffers<3.0,>=1.12', 'protobuf~=3.20.2'],

[5] https://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html

Your comment is confusing, when you refer to "Tensorflow 1.15", I assume you are referring to "tensorflow-onnx", correct? If so, I suggest editing your comment so it reflects the accurate package names, tensorflow-onnx/tensorflow, as applicable (and not short it to "tf").

[yan12125]

Your comment is confusing, when you refer to "Tensorflow 1.15", I assume you are referring to "tensorflow-onnx", correct? If so, I suggest editing your comment so it reflects the accurate package names, tensorflow-onnx/tensorflow, as applicable (and not short it to "tf").

Good point. I do mean Tensorflow 1.15, an old version released years ago - edited to avoid confusion.

[fatcat-z]

Hello @fatcat-z,

Bttw, it looks like tensorflow does not make it easier with tensorflow==2.14.0 since it requires flatbuffers >= 23.5.26 :(

Have you tried to remove the flatbuffers upper bound version in tf2oonx?

Thanks a lot for all your hard work!

That limitation has been removed since tf2onnx 1.15.0. Please try the latest 1.16.0 version and hopefully it could help you.