Failed to build libtensorflowlite_c.so on M1 mac

[jasonw247]

Click to expand!

Issue Type

Build/Install

Source

source

Tensorflow Version

2.10.0

Custom Code

No

OS Platform and Distribution

darwin_arm64

Mobile device

No response

Python version

3.9.12

Bazel version

5.3.0

GCC/Compiler version

Apple clang version 14.0.0

CUDA/cuDNN version

No response

GPU model and memory

No response

Current Behaviour?

When attempting to build `libtensorflowlite_c.so` on an M1 mac using the commands:


bazel build --config=monolithic --cpu=darwin_arm64 -c opt //tensorflow/lite/c:libtensorflowlite_c.so --verbose_failures --jobs=4

bazel build --config=macos_arm64 --cpu=darwin_arm64 -c opt //tensorflow/lite/c:libtensorflowlite_c.so --verbose_failures --jobs=4

bazel build --config=monolithic -c opt //tensorflow/lite/c:libtensorflowlite_c.so --verbose_failures --jobs=4

I get the following error:

ERROR: /private/var/tmp/_bazel_jason.white/5d9bcbdac1f97eda46b1818c207beabc/external/flatbuffers/BUILD.bazel:84:10: Linking external/flatbuffers/flatc [for host] failed: (Exit 1): cc_wrapper.sh failed: error executing command
  (cd /private/var/tmp/_bazel_jason.white/5d9bcbdac1f97eda46b1818c207beabc/execroot/org_tensorflow && \
  exec env - \
    PATH='/Users/jason.white/Library/Caches/bazelisk/downloads/bazelbuild/bazel-5.3.0-darwin-arm64/bin:/usr/local/opt/llvm/bin:/opt/homebrew/opt/llvm/bin:/opt/homebrew/opt/llvm/bin:/Users/jason.white/anaconda3/bin:/Users/jason.white/anaconda3/condabin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/jason.white/.cargo/bin:/Applications/CMake.app/Contents/bin
' \
    PWD=/proc/self/cwd \
  external/local_config_cc/cc_wrapper.sh @bazel-out/host/bin/external/flatbuffers/flatc-2.params)
# Configuration: bb154819987dda44281f89fe0a702c318c5f92b418ba76d644851296222133e1
# Execution platform: @local_execution_config_platform//:platform
clang: error: invalid linker name in argument '-fuse-ld=-debugger-tuning=lldb'
Target //tensorflow/lite/c:libtensorflowlite_c.so failed to build
INFO: Elapsed time: 0.187s, Critical Path: 0.04s
INFO: 5 processes: 5 internal.
FAILED: Build did NOT complete successfully

Expected behavior: the libtensorflowlite_c.so object is built.

### Standalone code to reproduce the issue

```shell
From the root TensorFlow directory:


bazel build --config=monolithic --cpu=darwin_arm64 -c opt //tensorflow/lite/c:libtensorflowlite_c.so --verbose_failures --jobs=4

### Relevant log output

```shell
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=160
INFO: Reading rc options for 'build' from /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc:
  'build' options: --define framework_shared_object=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:monolithic in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --define framework_shared_object=false --experimental_link_static_libraries_once=false
INFO: Found applicable config definition build:macos in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --apple_platform_type=macos --copt=-DGRPC_BAZEL_BUILD --copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/0fb2676c248ef1b76a4ef86c934e69ff80796ba7.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/6c8cdbbdb02f2ec36e3440d6edc34006368f0ba3.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_go/releases/download/v0.34.0/rules_go-v0.34.0.zip failed: class java.io.FileNotFoundException GET returned 404 Not Found
INFO: Analyzed target //tensorflow/lite/c:libtensorflowlite_c.so (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_jason.white/5d9bcbdac1f97eda46b1818c207beabc/external/flatbuffers/BUILD.bazel:84:10: Linking external/flatbuffers/flatc [for host] failed: (Exit 1): cc_wrapper.sh failed: error executing command
  (cd /private/var/tmp/_bazel_jason.white/5d9bcbdac1f97eda46b1818c207beabc/execroot/org_tensorflow && \
  exec env - \
    PATH='/Users/jason.white/Library/Caches/bazelisk/downloads/bazelbuild/bazel-5.3.0-darwin-arm64/bin:/usr/local/opt/llvm/bin:/opt/homebrew/opt/llvm/bin:/opt/homebrew/opt/llvm/bin:/Users/jason.white/anaconda3/bin:/Users/jason.white/anaconda3/condabin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/jason.white/.cargo/bin:/Applications/CMake.app/Contents/bin
' \
    PWD=/proc/self/cwd \
  external/local_config_cc/cc_wrapper.sh @bazel-out/host/bin/external/flatbuffers/flatc-2.params)
# Configuration: bb154819987dda44281f89fe0a702c318c5f92b418ba76d644851296222133e1
# Execution platform: @local_execution_config_platform//:platform
clang: error: invalid linker name in argument '-fuse-ld=-debugger-tuning=lldb'
Target //tensorflow/lite/c:libtensorflowlite_c.so failed to build
INFO: Elapsed time: 0.187s, Critical Path: 0.04s
INFO: 5 processes: 5 internal.
FAILED: Build did NOT complete successfully

[mohantym]

Hi @jasonw247 !
.so extension for linux machines and
.dylib extension for macos

Ref
Could you just go without .so extension and let us know .
bazel build --config=monolithic --cpu=darwin_arm64 -c opt //tensorflow/lite/c:libtensorflowlite_c --verbose_failures --jobs=4
Thank you!

[jasonw247]

@mohantym

running bazel build --config=monolithic --cpu=darwin_arm64 -c opt //tensorflow/lite/c:libtensorflowlite_c --verbose_failures --jobs=4

results in:

INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=149
INFO: Reading rc options for 'build' from /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc:
  'build' options: --define framework_shared_object=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:monolithic in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --define framework_shared_object=false --experimental_link_static_libraries_once=false
INFO: Found applicable config definition build:macos in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --apple_platform_type=macos --copt=-DGRPC_BAZEL_BUILD --copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/0fb2676c248ef1b76a4ef86c934e69ff80796ba7.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/6c8cdbbdb02f2ec36e3440d6edc34006368f0ba3.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_go/releases/download/v0.34.0/rules_go-v0.34.0.zip failed: class java.io.FileNotFoundException GET returned 404 Not Found
ERROR: Skipping '//tensorflow/lite/c:libtensorflowlite_c': no such target '//tensorflow/lite/c:libtensorflowlite_c': target 'libtensorflowlite_c' not declared in package 'tensorflow/lite/c' (did you mean 'libtensorflowlite_c.so'?) defined by /Users/jason.white/workspace/p21-library-builder/tensorflow_src/tensorflow/lite/c/BUILD
WARNING: Target pattern parsing failed.
ERROR: no such target '//tensorflow/lite/c:libtensorflowlite_c': target 'libtensorflowlite_c' not declared in package 'tensorflow/lite/c' (did you mean 'libtensorflowlite_c.so'?) defined by /Users/jason.white/workspace/p21-library-builder/tensorflow_src/tensorflow/lite/c/BUILD
INFO: Elapsed time: 0.138s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)

running: bazel build --config=monolithic --cpu=darwin_arm64 -c opt //tensorflow/lite/c:libtensorflowlite_c.dylib --verbose_failures --jobs=4

Results in:

bazel build --config=monolithic --cpu=darwin_arm64 -c opt //tensorflow/lite/c:libtensorflowlite_c.dylib --verbose_failures --jobs=4
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=149
INFO: Reading rc options for 'build' from /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc:
  'build' options: --define framework_shared_object=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:monolithic in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --define framework_shared_object=false --experimental_link_static_libraries_once=false
INFO: Found applicable config definition build:macos in file /Users/jason.white/workspace/p21-library-builder/tensorflow_src/.bazelrc: --apple_platform_type=macos --copt=-DGRPC_BAZEL_BUILD --copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/0fb2676c248ef1b76a4ef86c934e69ff80796ba7.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/6c8cdbbdb02f2ec36e3440d6edc34006368f0ba3.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_go/releases/download/v0.34.0/rules_go-v0.34.0.zip failed: class java.io.FileNotFoundException GET returned 404 Not Found
INFO: Analyzed target //tensorflow/lite/c:libtensorflowlite_c.dylib (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_jason.white/5d9bcbdac1f97eda46b1818c207beabc/external/flatbuffers/BUILD.bazel:84:10: Linking external/flatbuffers/flatc [for host] failed: (Exit 1): cc_wrapper.sh failed: error executing command
  (cd /private/var/tmp/_bazel_jason.white/5d9bcbdac1f97eda46b1818c207beabc/execroot/org_tensorflow && \
  exec env - \
    PATH='/Users/jason.white/Library/Caches/bazelisk/downloads/bazelbuild/bazel-5.3.0-darwin-arm64/bin:/Users/jason.white/.local/bin:/opt/homebrew/opt/llvm/bin:/opt/homebrew/opt/llvm/bin:/usr/local/opt/llvm/bin:/opt/homebrew/opt/llvm/bin:/opt/homebrew/opt/llvm/bin:/Users/jason.white/anaconda3/bin:/Users/jason.white/anaconda3/condabin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/jason.white/.cargo/bin:/Applications/CMake.app/Contents/bin
:/Applications/CMake.app/Contents/bin' \
    PWD=/proc/self/cwd \
  external/local_config_cc/cc_wrapper.sh @bazel-out/host/bin/external/flatbuffers/flatc-2.params)
# Configuration: bb154819987dda44281f89fe0a702c318c5f92b418ba76d644851296222133e1
# Execution platform: @local_execution_config_platform//:platform
clang: error: invalid linker name in argument '-fuse-ld=-debugger-tuning=lldb'
Target //tensorflow/lite/c:libtensorflowlite_c.dylib failed to build
INFO: Elapsed time: 0.225s, Critical Path: 0.04s
INFO: 5 processes: 5 internal.
FAILED: Build did NOT complete successfully

[jasonw247]

For reference, I was able to run both:

bazel build --config=monolithic -c opt //tensorflow/lite/c:libtensorflowlite_c.so --verbose_failures --jobs=4

and

bazel build --config=monolithic -c opt //tensorflow/lite/c:libtensorflowlite_c.dylib --verbose_failures --jobs=4

on an x86 mac without failure.

[mohantym]

Ok @jasonw247 !
I think you were trying to build libtensorflowlite_c for Mac M1 (Darwin Arm64). It succeeded with below command in my Mac M1 with Bazel 5.0.0 and TF 2.9 branch. Could you test in 2.10 version with Same command .

bazel build --config=monolithic -c opt tensorflow/lite/c:libtensorflowlite_c.dylib --verbose_failures --jobs=4

Thank you!

[jasonw247]

@mohantym I can confirm this now works with current master and r2.10 branches.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No