Unable to build tensorflow from source on Apple M1

[sun1638650145]

Click to expand!

Issue Type

Build/Install

Source

source

Tensorflow Version

tf 2.10

Custom Code

No

OS Platform and Distribution

macOS 12.6

Mobile device

None

Python version

3.9

Bazel version

5.1.1-homebrew

GCC/Compiler version

Apple clang version 14.0.0 (clang-1400.0.29.102)

CUDA/cuDNN version

None

GPU model and memory

None

Current Behaviour?

I need to use tensorflow-text, because tensorflow does not provide pre-compiled packages for Apple silicon, text compiled by tensorflow-macos has a symbol error, so now I need to compile tensorflow myself, and then go to compile text. I'm a newbie to bazel, and this is my first time compiling tensorflow, I have tried reinstalling Xcode and using conda and virtualenv virtual environments respectively, but all have this error.

Standalone code to reproduce the issue

Default settings used for all options.

 bazel build //tensorflow/tools/pip_package:build_pip_package

Relevant log output

INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=80
INFO: Reading rc options for 'build' from /Users/sunruiqi/Desktop/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/sunruiqi/Desktop/tensorflow/.bazelrc:
  'build' options: --define framework_shared_object=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false
INFO: Reading rc options for 'build' from /Users/sunruiqi/Desktop/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/Users/sunruiqi/tf/bin/python3 --action_env PYTHON_LIB_PATH=/Users/sunruiqi/tf/lib/python3.9/site-packages --python_path=/Users/sunruiqi/tf/bin/python3
INFO: Reading rc options for 'build' from /Users/sunruiqi/Desktop/tensorflow/.bazelrc:
  'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /Users/sunruiqi/Desktop/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/sunruiqi/Desktop/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:macos in file /Users/sunruiqi/Desktop/tensorflow/.bazelrc: --apple_platform_type=macos --copt=-DGRPC_BAZEL_BUILD --copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (1 packages loaded, 4 targets configured).
INFO: Found 1 target...
ERROR: /Users/sunruiqi/Desktop/tensorflow/tensorflow/python/profiler/internal/BUILD:80:27: Linking tensorflow/python/profiler/internal/_pywrap_traceme.so failed: (Aborted): cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh @bazel-out/darwin_arm64-opt/bin/tensorflow/python/profiler/internal/_pywrap_traceme.so-2.params
ld: malformed trie, childNodeOffset==0 file 'bazel-out/darwin_arm64-opt/bin/_solib_darwin_arm64/_U_S_Stensorflow_Spython_C_Upywrap_Utensorflow_Uinternal_Umacos___Utensorflow_Spython/lib_pywrap_tensorflow_internal.dylib'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error in child process '/usr/bin/xcrun'. 1
external/local_config_cc/cc_wrapper.sh: line 69:  2048 Abort trap: 6           "$(/usr/bin/dirname "$0")"/wrapped_clang "$@"
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2939.365s, Critical Path: 233.21s
INFO: 13427 processes: 141 internal, 13286 local.
FAILED: Build did NOT complete successfully

[tilakrayal]

@sun1638650145,
Could you please take a look at the comment from the developer for the similar issue while installing the Tensorflow on Apple M1. Thank you!

[sun1638650145]

I've looked at this issue, but this issue doesn't provide any installation help.

[SuryanarayanaY]

Hi @sun1638650145 ,

Referring to above comment till the official support for Apple Silicon (Aarch64 based) made available, could you please try with alternate approaches suggested by other users for similar issues from here. Also I observed you are using Clang version of 14.0 which is not yet tested on Tensorflow officially.Please refer the supported configurations here.

Thankyou!

[sun1638650145]

Hi, because I contributed to the build of tensorflow-text on Apple silicon, but now we found an issue with tensorflow-macos 2.10, and tensorflow-macos is not open source. So the current idea is to build a tensorflow from source code.

[freedomtan]

Xcode 14.x come with "problematic" linker (ld), cf. #58368 (comment)

[SuryanarayanaY]

Hi @sun1638650145 ,

Could you please confirm the sequence of steps you followed to Build from source? Please refer to the attached documentation here and confirm the steps you have followed. Also please confirm have you used ./configure option to change any default settings. Please ignore GPU related steps as MacOS not supporting TF-GPU for now.

Thankyou!

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[sun1638650145]

@SuryanarayanaY Hi, I've verified my steps (I didn't use any additional configuration options), and I've also tried TensorFlow 2.11 with the same problem.

[sun1638650145]

@freedomtan I suspect this is the problem now, thanks for reminding me.

[freedomtan]

@freedomtan I suspect this is the problem now, thanks for reminding me.

Yes, I am pretty sure that's the ld problem (or maybe compiler+ld problem). It's not an M1 only issue. I ran into same problems on MacBook Pro 2018 + Ventura + Xcode 14.1

[SuryanarayanaY]

Hi @sun1638650145 ,
Developer team looking into the similar issue mentioned in above comment and hopefully this also resolved with it.
Thankyou!

[SuryanarayanaY]

Hi @sun1638650145 ,

I confirm that with MacOS 13.3 and XCode 14.3 configurations worked successfully for build from source on Apple M1. Xcode14.3 should resolve this error related to linker id.

Please refer this source for confirmation. Please update the configurations mentioned above and let us know if still having problem in build. Thanks!

[sun1638650145]

@SuryanarayanaY Yes, I have been able to successfully build, thank you very much for your help.

[sun1638650145]

Here is the whl file that I have built for everyone to use. Additionally, I have also compiled a tutorial that I have organized.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No