Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tensorflow-text not compatible with tensorflow-macos and tensorflow-metal #654

Open
dbl001 opened this issue Jul 13, 2021 · 3 comments
Open

tensorflow-text not compatible with tensorflow-macos and tensorflow-metal #654

dbl001 opened this issue Jul 13, 2021 · 3 comments
Labels
duplicate This issue or pull request already exists

Comments

@dbl001
Copy link

dbl001 commented Jul 13, 2021 •
edited

I am able to get my AMD Radeon Pro 5700 XT gpu to run on an iMac 27" with Big Sur tensorflow with 2.5.

Screen Shot 2021-07-13 at 11 49 32 AM

If I add tensorflow-text, I get this exception:

% python
Python 3.8.2 (default, Apr  8 2021, 23:19:18) 
[Clang 12.0.5 (clang-1205.0.22.9)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/__init__.py", line 449, in <module>
    _ll.load_library(_plugin_dir)
  File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/framework/load_library.py", line 154, in load_library
    py_tf.TF_LoadLibrary(lib)
tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow-plugins/libmetal_plugin.dylib, 6): Symbol not found: _TF_AssignUpdateVariable
  Referenced from: /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow-plugins/libmetal_plugin.dylib
  Expected in: flat namespace

>>>

Here's what was added with pip install tensorflow-text in the Python 3.8.2 virtual environment:

 pip install tensorflow_text
Collecting tensorflow_text
  Using cached tensorflow_text-2.5.0-cp38-cp38-macosx_10_9_x86_64.whl (3.6 MB)
Collecting tensorflow<2.6,>=2.5.0
  Using cached tensorflow-2.5.0-cp38-cp38-macosx_10_11_x86_64.whl (195.7 MB)
Collecting tensorflow-hub>=0.8.0
  Using cached tensorflow_hub-0.12.0-py2.py3-none-any.whl (108 kB)
Requirement already satisfied: tensorflow-estimator<2.6.0,>=2.5.0rc0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (2.5.0)
Requirement already satisfied: numpy~=1.19.2 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (1.19.5)
Requirement already satisfied: tensorboard~=2.5 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (2.5.0)
Requirement already satisfied: termcolor~=1.1.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (1.1.0)
Requirement already satisfied: opt-einsum~=3.3.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (3.3.0)
Requirement already satisfied: protobuf>=3.9.2 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (3.17.3)
Requirement already satisfied: flatbuffers~=1.12.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (1.12)
Requirement already satisfied: google-pasta~=0.2 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (0.2.0)
Requirement already satisfied: grpcio~=1.34.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (1.34.1)
Requirement already satisfied: astunparse~=1.6.3 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (1.6.3)
Requirement already satisfied: six~=1.15.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (1.15.0)
Requirement already satisfied: keras-preprocessing~=1.1.2 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (1.1.2)
Requirement already satisfied: typing-extensions~=3.7.4 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (3.7.4.3)
Requirement already satisfied: gast==0.4.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (0.4.0)
Requirement already satisfied: keras-nightly~=2.5.0.dev in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (2.5.0.dev2021032900)
Requirement already satisfied: absl-py~=0.10 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (0.12.0)
Requirement already satisfied: wrapt~=1.12.1 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (1.12.1)
Requirement already satisfied: h5py~=3.1.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (3.1.0)
Requirement already satisfied: wheel~=0.35 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow_text) (0.36.2)
Requirement already satisfied: google-auth<2,>=1.6.3 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (1.32.1)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (0.6.1)
Requirement already satisfied: requests<3,>=2.21.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (2.26.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (0.4.4)
Requirement already satisfied: werkzeug>=0.11.15 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (2.0.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (1.8.0)
Requirement already satisfied: setuptools>=41.0.0 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (41.2.0)
Requirement already satisfied: markdown>=2.6.8 in ./tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (3.3.4)
Requirement already satisfied: pyasn1-modules>=0.2.1 in ./tensorflow-metal/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (0.2.8)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in ./tensorflow-metal/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (4.2.2)
Requirement already satisfied: rsa<5,>=3.1.4 in ./tensorflow-metal/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (4.7.2)
Requirement already satisfied: requests-oauthlib>=0.7.0 in ./tensorflow-metal/lib/python3.8/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (1.3.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in ./tensorflow-metal/lib/python3.8/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (0.4.8)
Requirement already satisfied: certifi>=2017.4.17 in ./tensorflow-metal/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (2021.5.30)
Requirement already satisfied: idna<4,>=2.5 in ./tensorflow-metal/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (3.2)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./tensorflow-metal/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (1.26.6)
Requirement already satisfied: charset-normalizer~=2.0.0 in ./tensorflow-metal/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (2.0.1)
Requirement already satisfied: oauthlib>=3.0.0 in ./tensorflow-metal/lib/python3.8/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow_text) (3.1.1)
Installing collected packages: tensorflow-hub, tensorflow, tensorflow-text
Successfully installed tensorflow-2.5.0 tensorflow-hub-0.12.0 tensorflow-text-2.5.0

Here's what the environment looked like before adding tensorflow-text:

r@x86_64-apple-darwin13 ~ % pip list
Package                  Version
------------------------ -------------------
absl-py                  0.12.0
appnope                  0.1.2
astunparse               1.6.3
attrs                    21.2.0
backcall                 0.2.0
cachetools               4.2.2
certifi                  2021.5.30
charset-normalizer       2.0.1
Cython                   0.29.23
debugpy                  1.3.0
decorator                5.0.9
dill                     0.3.4
flatbuffers              1.12
future                   0.18.2
gast                     0.4.0
google-auth              1.32.1
google-auth-oauthlib     0.4.4
google-pasta             0.2.0
googleapis-common-protos 1.53.0
grpcio                   1.34.1
h5py                     3.1.0
idna                     3.2
importlib-resources      5.2.0
ipykernel                6.0.1
ipython                  7.25.0
ipython-genutils         0.2.0
jedi                     0.18.0
jupyter-client           6.1.12
jupyter-core             4.7.1
keras-nightly            2.5.0.dev2021032900
Keras-Preprocessing      1.1.2
Markdown                 3.3.4
matplotlib-inline        0.1.2
numpy                    1.19.5
oauthlib                 3.1.1
opt-einsum               3.3.0
parso                    0.8.2
pexpect                  4.8.0
pickleshare              0.7.5
pip                      21.1.3
promise                  2.3
prompt-toolkit           3.0.19
protobuf                 3.17.3
ptyprocess               0.7.0
pyasn1                   0.4.8
pyasn1-modules           0.2.8
Pygments                 2.9.0
python-dateutil          2.8.1
pyzmq                    22.1.0
requests                 2.26.0
requests-oauthlib        1.3.0
rsa                      4.7.2
setuptools               41.2.0
six                      1.15.0
tensorboard              2.5.0
tensorboard-data-server  0.6.1
tensorboard-plugin-wit   1.8.0
tensorflow-datasets      4.3.0
tensorflow-estimator     2.5.0
tensorflow-macos         2.5.0
tensorflow-metadata      1.1.0
tensorflow-metal         0.1.1
termcolor                1.1.0
tornado                  6.1
tqdm                     4.61.2
traitlets                5.0.5
typing-extensions        3.7.4.3
urllib3                  1.26.6
wcwidth                  0.2.5
Werkzeug                 2.0.1
wheel                    0.36.2
wrapt                    1.12.1
zipp                     3.5.0

Here's with the environment looks like after installing tensorflow-text:

% pip list
Package                  Version
------------------------ -------------------
absl-py                  0.12.0
appnope                  0.1.2
astunparse               1.6.3
attrs                    21.2.0
backcall                 0.2.0
cachetools               4.2.2
certifi                  2021.5.30
charset-normalizer       2.0.1
Cython                   0.29.23
debugpy                  1.3.0
decorator                5.0.9
dill                     0.3.4
flatbuffers              1.12
future                   0.18.2
gast                     0.4.0
google-auth              1.32.1
google-auth-oauthlib     0.4.4
google-pasta             0.2.0
googleapis-common-protos 1.53.0
grpcio                   1.34.1
h5py                     3.1.0
idna                     3.2
importlib-resources      5.2.0
ipykernel                6.0.1
ipython                  7.25.0
ipython-genutils         0.2.0
jedi                     0.18.0
jupyter-client           6.1.12
jupyter-core             4.7.1
keras-nightly            2.5.0.dev2021032900
Keras-Preprocessing      1.1.2
Markdown                 3.3.4
matplotlib-inline        0.1.2
numpy                    1.19.5
oauthlib                 3.1.1
opt-einsum               3.3.0
parso                    0.8.2
pexpect                  4.8.0
pickleshare              0.7.5
pip                      21.1.3
promise                  2.3
prompt-toolkit           3.0.19
protobuf                 3.17.3
ptyprocess               0.7.0
pyasn1                   0.4.8
pyasn1-modules           0.2.8
Pygments                 2.9.0
python-dateutil          2.8.1
pyzmq                    22.1.0
requests                 2.26.0
requests-oauthlib        1.3.0
rsa                      4.7.2
setuptools               41.2.0
six                      1.15.0
tensorboard              2.5.0
tensorboard-data-server  0.6.1
tensorboard-plugin-wit   1.8.0
tensorflow               2.5.0
tensorflow-datasets      4.3.0
tensorflow-estimator     2.5.0
tensorflow-hub           0.12.0
tensorflow-macos         2.5.0
tensorflow-metadata      1.1.0
tensorflow-metal         0.1.1
tensorflow-text          2.5.0
termcolor                1.1.0
tornado                  6.1
tqdm                     4.61.2
traitlets                5.0.5
typing-extensions        3.7.4.3
urllib3                  1.26.6
wcwidth                  0.2.5
Werkzeug                 2.0.1
wheel                    0.36.2
wrapt                    1.12.1
zipp                     3.5.0
@dbl001 dbl001 changed the title tensor flow-text not compatible with tensorflow-macos and tensorflow-metal tensorflow-text not compatible with tensorflow-macos and tensorflow-metal Jul 13, 2021
@dbl001
Copy link
Author

dbl001 commented Jul 13, 2021

I tried building tensorflow-text from source but that didn't work either.
I edited line 40 of oss_scripts/configure.sh to:
if [[ $(pip show tensorflow-macos) == tensorflow ]] || [[ $(pip show tf-nightl

but, ...

./oss_scripts/run_build.sh
++ uname -s
+ osname=Darwin
+ [[ Darwin == \D\a\r\w\i\n ]]
+ sed -i '' 's/".so"/".dylib"/' tensorflow_text/tftext.bzl
+ sed -i '' 's/*.so/*.dylib/' oss_scripts/pip_package/MANIFEST.in
++ find tensorflow_text/python -type f
+ perl -pi -e 's/(load_library.load_op_library.*)\.so'\''/$1.dylib'\''/' tensorflow_text/python/keras/layers/__init__.py tensorflow_text/python/keras/layers/__pycache__/__init__.cpython-38.pyc tensorflow_text/python/keras/layers/__pycache__/todense.cpython-38.pyc tensorflow_text/python/keras/layers/todense.py tensorflow_text/python/keras/layers/todense_test.py tensorflow_text/python/keras/__init__.py tensorflow_text/python/keras/__pycache__/__init__.cpython-38.pyc tensorflow_text/python/metrics/text_similarity_metric_ops.py tensorflow_text/python/metrics/__init__.py tensorflow_text/python/metrics/__pycache__/__init__.cpython-38.pyc tensorflow_text/python/metrics/__pycache__/text_similarity_metric_ops.cpython-38.pyc tensorflow_text/python/metrics/text_similarity_metric_ops_test.py tensorflow_text/python/metrics/_text_similarity_metric_ops.dylib tensorflow_text/python/__init__.py tensorflow_text/python/__pycache__/__init__.cpython-38.pyc tensorflow_text/python/numpy/viterbi_decode.py tensorflow_text/python/numpy/__init__.py tensorflow_text/python/numpy/viterbi_decode_test.py tensorflow_text/python/benchmarks/ops_benchmarks.py tensorflow_text/python/benchmarks/test_data/uncased_L-12_H-768_A-12/vocab.txt tensorflow_text/python/benchmarks/__init__.py tensorflow_text/python/benchmarks/tokenizers_benchmarks.py tensorflow_text/python/benchmarks/benchmark_utils.py tensorflow_text/python/ops/string_ops.py tensorflow_text/python/ops/viterbi_constrained_sequence_op_test.py tensorflow_text/python/ops/sentencepiece_tokenizer.py tensorflow_text/python/ops/normalize_ops_test.py tensorflow_text/python/ops/trimmer_ops_test.py tensorflow_text/python/ops/masking_ops.py tensorflow_text/python/ops/pointer_ops.py tensorflow_text/python/ops/hub_module_tokenizer_test.py tensorflow_text/python/ops/regex_split_ops_test.py tensorflow_text/python/ops/bert_tokenizer_test.py tensorflow_text/python/ops/_split_merge_from_logits_tokenizer.dylib tensorflow_text/python/ops/pad_model_inputs_ops.py tensorflow_text/python/ops/_mst_ops.dylib tensorflow_text/python/ops/item_selector_ops.py tensorflow_text/python/ops/state_based_sentence_breaker_op.py tensorflow_text/python/ops/split_merge_from_logits_tokenizer.py tensorflow_text/python/ops/pad_along_dimension_op_test.py tensorflow_text/python/ops/_sentencepiece_tokenizer.dylib tensorflow_text/python/ops/segment_combiner_ops_test.py tensorflow_text/python/ops/_split_merge_tokenizer.dylib tensorflow_text/python/ops/unicode_char_tokenizer_test.py tensorflow_text/python/ops/test_data/test_oss_model.model tensorflow_text/python/ops/test_data/segmenter_hub_module/variables/variables.data-00000-of-00001 tensorflow_text/python/ops/test_data/segmenter_hub_module/variables/variables.index tensorflow_text/python/ops/test_data/segmenter_hub_module/saved_model.pb tensorflow_text/python/ops/test_data/segmenter_hub_module/tfhub_module.pb tensorflow_text/python/ops/wordpiece_tokenizer_test.py tensorflow_text/python/ops/ngrams_op_test.py tensorflow_text/python/ops/greedy_constrained_sequence_op_test.py tensorflow_text/python/ops/_whitespace_tokenizer.dylib tensorflow_text/python/ops/splitter.py tensorflow_text/python/ops/viterbi_constrained_sequence_op.py tensorflow_text/python/ops/__init__.py tensorflow_text/python/ops/split_merge_from_logits_tokenizer_test.py tensorflow_text/python/ops/tokenization.py tensorflow_text/python/ops/span_alignment_op_test.py tensorflow_text/python/ops/whitespace_tokenizer_test.py tensorflow_text/python/ops/split_merge_tokenizer.py tensorflow_text/python/ops/__pycache__/__init__.cpython-38.pyc tensorflow_text/python/ops/__pycache__/unicode_char_tokenizer.cpython-38.pyc tensorflow_text/python/ops/__pycache__/greedy_constrained_sequence_op.cpython-38.pyc tensorflow_text/python/ops/__pycache__/pad_along_dimension_op.cpython-38.pyc tensorflow_text/python/ops/__pycache__/wordshape_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/ngrams_op.cpython-38.pyc tensorflow_text/python/ops/__pycache__/split_merge_tokenizer.cpython-38.pyc tensorflow_text/python/ops/__pycache__/trimmer_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/mst_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/sentencepiece_tokenizer.cpython-38.pyc tensorflow_text/python/ops/__pycache__/splitter.cpython-38.pyc tensorflow_text/python/ops/__pycache__/wordpiece_tokenizer.cpython-38.pyc tensorflow_text/python/ops/__pycache__/hub_module_tokenizer.cpython-38.pyc tensorflow_text/python/ops/__pycache__/bert_tokenizer.cpython-38.pyc tensorflow_text/python/ops/__pycache__/normalize_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/pointer_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/whitespace_tokenizer.cpython-38.pyc tensorflow_text/python/ops/__pycache__/sentence_breaking_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/string_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/split_merge_from_logits_tokenizer.cpython-38.pyc tensorflow_text/python/ops/__pycache__/viterbi_constrained_sequence_op.cpython-38.pyc tensorflow_text/python/ops/__pycache__/regex_split_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/tokenization.cpython-38.pyc tensorflow_text/python/ops/__pycache__/item_selector_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/masking_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/unicode_script_tokenizer.cpython-38.pyc tensorflow_text/python/ops/__pycache__/state_based_sentence_breaker_op.cpython-38.pyc tensorflow_text/python/ops/__pycache__/hub_module_splitter.cpython-38.pyc tensorflow_text/python/ops/__pycache__/pad_model_inputs_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/segment_combiner_ops.cpython-38.pyc tensorflow_text/python/ops/__pycache__/create_feature_bitmask_op.cpython-38.pyc tensorflow_text/python/ops/__pycache__/sliding_window_op.cpython-38.pyc tensorflow_text/python/ops/bert_tokenizer.py tensorflow_text/python/ops/hub_module_splitter_test.py tensorflow_text/python/ops/wordshape_ops_test.py tensorflow_text/python/ops/span_overlaps_op_test.py tensorflow_text/python/ops/_sentence_breaking_ops.dylib tensorflow_text/python/ops/sentence_breaking_ops.py tensorflow_text/python/ops/wordpiece_tokenizer.py tensorflow_text/python/ops/sliding_window_op_test.py tensorflow_text/python/ops/trimmer_ops.py tensorflow_text/python/ops/segment_combiner_ops.py tensorflow_text/python/ops/unicode_script_tokenizer.py tensorflow_text/python/ops/normalize_ops.py tensorflow_text/python/ops/hub_module_tokenizer.py tensorflow_text/python/ops/whitespace_tokenizer.py tensorflow_text/python/ops/mst_ops.py tensorflow_text/python/ops/state_based_sentence_breaker_op_test.py tensorflow_text/python/ops/sliding_window_op.py tensorflow_text/python/ops/sentence_breaking_ops_test.py tensorflow_text/python/ops/masking_ops_test.py tensorflow_text/python/ops/greedy_constrained_sequence_op.py tensorflow_text/python/ops/unicode_script_tokenizer_test.py tensorflow_text/python/ops/_constrained_sequence_op.dylib tensorflow_text/python/ops/gather_with_default_op_test.py tensorflow_text/python/ops/split_merge_tokenizer_test.py tensorflow_text/python/ops/create_feature_bitmask_op.py tensorflow_text/python/ops/pad_along_dimension_op.py tensorflow_text/python/ops/create_feature_bitmask_op_test.py tensorflow_text/python/ops/ngrams_op.py tensorflow_text/python/ops/_unicode_script_tokenizer.dylib tensorflow_text/python/ops/sentencepiece_tokenizer_test.py tensorflow_text/python/ops/item_selector_ops_test.py tensorflow_text/python/ops/regex_split_ops.py tensorflow_text/python/ops/coerce_to_valid_utf8_op_test.py tensorflow_text/python/ops/hub_module_splitter.py tensorflow_text/python/ops/unicode_char_tokenizer.py tensorflow_text/python/ops/_regex_split_ops.dylib tensorflow_text/python/ops/wordshape_ops.py tensorflow_text/python/ops/_wordpiece_tokenizer.dylib tensorflow_text/python/ops/_normalize_ops.dylib tensorflow_text/python/ops/_state_based_sentence_breaker_op.dylib tensorflow_text/python/ops/pad_model_inputs_ops_test.py tensorflow_text/python/ops/mst_ops_test.py
+ export CC_OPT_FLAGS=-mavx
+ CC_OPT_FLAGS=-mavx
+ ./oss_scripts/configure.sh
darwin
Using installed tensorflow.
+ /Users/davidlaxer/Downloads/bazel-3.7.2/bazel-bin/src/bazel build --enable_runfiles oss_scripts/pip_package:build_pip_package
INFO: Repository local_config_tf instantiated at:
  /Users/davidlaxer/text/WORKSPACE:119:13: in <toplevel>
Repository rule tf_configure defined at:
  /Users/davidlaxer/text/third_party/tensorflow/tf_configure.bzl:207:31: in <toplevel>
ERROR: An error occurred during the fetch of repository 'local_config_tf':
   Traceback (most recent call last):
	File "/Users/davidlaxer/text/third_party/tensorflow/tf_configure.bzl", line 182, column 46, in _tf_pip_impl
		tf_header_rule = _symlink_genrule_for_dir(
	File "/Users/davidlaxer/text/third_party/tensorflow/tf_configure.bzl", line 156, column 43, in _symlink_genrule_for_dir
		files = "\n".join(sorted(_read_dir(repository_ctx, src_dir).splitlines()))
	File "/Users/davidlaxer/text/third_party/tensorflow/tf_configure.bzl", line 89, column 31, in _read_dir
		find_result = _execute(
	File "/Users/davidlaxer/text/third_party/tensorflow/tf_configure.bzl", line 56, column 14, in _execute
		_fail("\n".join([
	File "/Users/davidlaxer/text/third_party/tensorflow/tf_configure.bzl", line 24, column 9, in _fail
		fail("%sPython Configuration Error:%s %s\n" % (red, no_color, msg))
Error in fail: Python Configuration Error: Repository command failed
find: it: No such file or directory

ERROR: /Users/davidlaxer/text/tensorflow_text/BUILD:674:19: //tensorflow_text:python/ops/_state_based_sentence_breaker_op.dylib depends on @local_config_tf//:libtensorflow_framework in repository @local_config_tf which failed to fetch. no such package '@local_config_tf//': Python Configuration Error: Repository command failed
find: it: No such file or directory

ERROR: /Users/davidlaxer/text/tensorflow_text/BUILD:674:19: //tensorflow_text:python/ops/_state_based_sentence_breaker_op.dylib depends on @local_config_tf//:tf_header_lib in repository @local_config_tf which failed to fetch. no such package '@local_config_tf//': Python Configuration Error: Repository command failed
find: it: No such file or directory

ERROR: Analysis of target '//oss_scripts/pip_package:build_pip_package' failed; build aborted: Analysis failed
INFO: Elapsed time: 1.101s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (30 packages loaded, 269 targets c\
onfigured)

@broken broken added the duplicate This issue or pull request already exists label Jul 13, 2021
@broken
Copy link
Member

broken commented Jul 13, 2021

tensorflow-macos is provided by Apple. Our builds only work with the core TF builds. Until they begin to offer these, we cannot either. There are more details on #538

@dbl001
Copy link
Author

dbl001 commented Jul 14, 2021 •
edited

I tried cloning a branch 2.5 version of tensorflow_text, and
correcting some problems in 'oss_scripts/configure.sh, then building with Bazel 3.7.2
E.g.

if [[ $(pip show tensorflow-macos) == *tensorflow* ]] || [[ $(pip show tf-nightl
...
#TF_CFLAGS=( $(python -c "import tensorflow as tf; print(' '.join(tf.sysconfig.get_compile_flags()))" | awk '{print $1}') )
#TF_LFLAGS=( $(python -c "import tensorflow as tf; print(' '.join(tf.sysconfig.get_link_flags()))" | awk '{print $1}') )
#TF_LFLAGS_2=( $(python -c "import tensorflow as tf; print(' '.join(tf.sysconfig.get_link_flags()))" | awk '{print $2}') )
#TF_ABIFLAG=$(python -c "import tensorflow as tf; print(tf.sysconfig.CXX11_ABI_FLAG)")

TF_CFLAGS="/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/include"
TF_LFLAGS="/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow"
TF_FLAGS_2="-ltensorflow_framework.2"

echo $TF_CFLAGS
echo $TF_LFLAGS
echo $TF_LFLAGS_2

#HEADER_DIR=${TF_CFLAGS:2}
#SHARED_LIBRARY_DIR=${TF_LFLAGS:2}
#SHARED_LIBRARY_NAME=$(echo $TF_LFLAGS_2 | rev | cut -d":" -f1 | rev)

HEADER_DIR=$TF_CFLAGS
SHARED_LIBRARY_DIR=$TF_LFLAGS
SHARED_LIBRARY_NAME=$TF_LFLAGS_2

echo $HEADER_DIR
echo $SHARED_LIBRARY_DIR
echo $SHARED_LIBRARY_NAME
...

I was able to build a wheel
E.g.
-rw-r--r-- 1 davidlaxer staff 4043186 Jul 14 09:48 tensorflow_text-2.5.0-cp38-cp38-macosx_10_9_x86_64.whl

text % pip install tensorflow_text-2.5.0-cp38-cp38-macosx_10_9_x86_64.whl

But, after installing the wheel into my tensorflow-metal virtual environment, it broke tensorflow:

% pip install tensorflow_text-2.5.0-cp38-cp38-macosx_10_9_x86_64.whl 
Processing ./tensorflow_text-2.5.0-cp38-cp38-macosx_10_9_x86_64.whl
Collecting tensorflow<2.6,>=2.5.0
  Using cached tensorflow-2.5.0-cp38-cp38-macosx_10_11_x86_64.whl (195.7 MB)
Collecting tensorflow-hub>=0.8.0
  Using cached tensorflow_hub-0.12.0-py2.py3-none-any.whl (108 kB)
Requirement already satisfied: h5py~=3.1.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (3.1.0)
Requirement already satisfied: wheel~=0.35 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (0.36.2)
Requirement already satisfied: keras-nightly~=2.5.0.dev in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (2.5.0.dev2021032900)
Requirement already satisfied: tensorflow-estimator<2.6.0,>=2.5.0rc0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (2.5.0)
Requirement already satisfied: tensorboard~=2.5 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (2.5.0)
Requirement already satisfied: grpcio~=1.34.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.34.1)
Requirement already satisfied: six~=1.15.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.15.0)
Requirement already satisfied: termcolor~=1.1.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.1.0)
Requirement already satisfied: protobuf>=3.9.2 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (3.17.3)
Requirement already satisfied: numpy~=1.19.2 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.19.5)
Requirement already satisfied: astunparse~=1.6.3 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.6.3)
Requirement already satisfied: typing-extensions~=3.7.4 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (3.7.4.3)
Requirement already satisfied: gast==0.4.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (0.4.0)
Requirement already satisfied: opt-einsum~=3.3.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (3.3.0)
Requirement already satisfied: absl-py~=0.10 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (0.12.0)
Requirement already satisfied: flatbuffers~=1.12.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.12)
Requirement already satisfied: google-pasta~=0.2 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (0.2.0)
Requirement already satisfied: wrapt~=1.12.1 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.12.1)
Requirement already satisfied: keras-preprocessing~=1.1.2 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.1.2)
Requirement already satisfied: werkzeug>=0.11.15 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (2.0.1)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (0.6.1)
Requirement already satisfied: setuptools>=41.0.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (41.2.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (0.4.4)
Requirement already satisfied: requests<3,>=2.21.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (2.26.0)
Requirement already satisfied: markdown>=2.6.8 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (3.3.4)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.8.0)
Requirement already satisfied: google-auth<2,>=1.6.3 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.32.1)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (4.7.2)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (4.2.2)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.3.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (0.4.8)
Requirement already satisfied: certifi>=2017.4.17 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (2021.5.30)
Requirement already satisfied: charset-normalizer~=2.0.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (2.0.1)
Requirement already satisfied: idna<4,>=2.5 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (3.2)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (1.26.6)
Requirement already satisfied: oauthlib>=3.0.0 in /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.5->tensorflow<2.6,>=2.5.0->tensorflow-text==2.5.0) (3.1.1)
Installing collected packages: tensorflow-hub, tensorflow, tensorflow-text
Successfully installed tensorflow-2.5.0 tensorflow-hub-0.12.0 tensorflow-text-2.5.0
(tensorflow-metal) (base) davidlaxer@x86_64-apple-darwin13 text % python
Python 3.8.2 (default, Apr  8 2021, 23:19:18) 
[Clang 12.0.5 (clang-1205.0.22.9)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/__init__.py", line 449, in <module>
    _ll.load_library(_plugin_dir)
  File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/framework/load_library.py", line 154, in load_library
    py_tf.TF_LoadLibrary(lib)
tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow-plugins/libmetal_plugin.dylib, 6): Symbol not found: _TF_AssignUpdateVariable
  Referenced from: /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow-plugins/libmetal_plugin.dylib
  Expected in: flat namespace

>>> quit()

I will wait until Apple integrates their tensorflow-macos & tensorflow-metal work into the Tensorflow ecosystem.

@broken broken mentioned this issue Jan 18, 2022
Closed
@broken broken mentioned this issue Feb 1, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
duplicate This issue or pull request already exists
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@broken @dbl001