Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apple M1 support #33

Closed
ibulu opened this issue Apr 23, 2022 · 17 comments
Closed

Apple M1 support #33

ibulu opened this issue Apr 23, 2022 · 17 comments

Comments

@ibulu
Copy link

ibulu commented Apr 23, 2022

is it possible to install on Apple M1? I wasn't able to install via pip

@jbms
Copy link
Collaborator

jbms commented Apr 23, 2022

It should build on Apple M1, however we do not yet have binary wheels pre-built for Apple M1 so it will be necessary to build, which requires xcode and may take a few minutes.

What error are you getting?

The current release 0.1.18 on PyPI has an issue with the zlib dependency --- a new release 0.1.19 should be available within an hour that solves the zlib issue (but you will still need to build on M1).

You can also just pip install git+https://github.,com/google/tensorstore rather than wait for 0.1.19.

@ibulu
Copy link
Author

ibulu commented Apr 23, 2022

below is the error I am getting

pip install git+https://github.com/google/tensorstore

Collecting git+https://github.com/google/tensorstore
Cloning https://github.com/google/tensorstore to /private/var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/pip-req-build-ek2na8c4
Running command git clone --filter=blob:none --quiet https://github.com/google/tensorstore /private/var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/pip-req-build-ek2na8c4
Resolved https://github.com/google/tensorstore to commit 7f45003
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy>=1.16.0 in /Users/ibulu/opt/miniconda3/lib/python3.8/site-packages (from tensorstore==0.1.19) (1.21.5)
Building wheels for collected packages: tensorstore
Building wheel for tensorstore (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for tensorstore (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [118 lines of output]
running bdist_wheel
running build
running build_py
creating /var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/tmpsop1e0bx/lib.macosx-11.0-arm64-cpython-38
creating /var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/tmpsop1e0bx/lib.macosx-11.0-arm64-cpython-38/tensorstore
copying python/tensorstore/cc_test_driver_main.py -> /var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/tmpsop1e0bx/lib.macosx-11.0-arm64-cpython-38/tensorstore
copying python/tensorstore/init.py -> /var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/tmpsop1e0bx/lib.macosx-11.0-arm64-cpython-38/tensorstore
running build_ext
/Users/ibulu/opt/miniconda3/bin/python -u bazelisk.py build -c opt //python/tensorstore:_tensorstore__shared_objects --verbose_failures --macos_minimum_os=11.0
Starting local Bazel server and connecting to it...
Loading:
Loading: 0 packages loaded
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (1 packages loaded, 0 targets configured)
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (34 packages loaded, 131 targets configured)
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (34 packages loaded, 131 targets configured)
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (34 packages loaded, 131 targets configured)
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (34 packages loaded, 131 targets configured)
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (80 packages loaded, 349 targets configured)
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (112 packages loaded, 2137 targets configured)
DEBUG: Rule 'org_aomedia_avif' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "6dce70530fa750544ba842e40555825e8894e8f805cf5c458ee0642a60d160e5"
DEBUG: Repository org_aomedia_avif instantiated at:
/private/var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/pip-req-build-ek2na8c4/WORKSPACE:7:25: in
/private/var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/pip-req-build-ek2na8c4/external.bzl:20:29: in tensorstore_dependencies
/private/var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/pip-req-build-ek2na8c4/third_party/third_party.bzl:54:26: in third_party_dependencies
/private/var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/pip-req-build-ek2na8c4/third_party/org_aomedia_avif/workspace.bzl:26:10: in repo
/private/var/tmp/_bazel_ibulu/fa7d43dc29afa7aa0093ed0eb5c0237a/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule third_party_http_archive defined at:
/private/var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/pip-req-build-ek2na8c4/third_party/repo.bzl:116:43: in
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (121 packages loaded, 3781 targets configured)
INFO: Analyzed target //python/tensorstore:_tensorstore__shared_objects (122 packages loaded, 5461 targets configured).
INFO: Found 1 target...
[0 / 292] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[395 / 697] RTCD aom_dsp_rtcd @org_aomedia_aom//:aom_dsp_rtcd; 0s darwin-sandbox ... (10 actions, 9 running)
[486 / 788] Compiling tensorstore/driver/downsample/grid_occupancy_map.cc; 1s darwin-sandbox ... (10 actions, 9 running)
[572 / 857] Compiling tensorstore/driver/json/driver.cc; 3s darwin-sandbox ... (10 actions, 9 running)
[662 / 933] Compiling tensorstore/driver/zarr/spec.cc; 2s darwin-sandbox ... (10 actions, 9 running)
[767 / 1,021] Compiling python/tensorstore/context.cc; 4s darwin-sandbox ... (10 actions, 9 running)
[898 / 1,153] Compiling python/tensorstore/tensorstore_class.cc; 4s darwin-sandbox ... (10 actions, 9 running)
[994 / 1,363] Compiling python/tensorstore/tensorstore_class.cc; 8s darwin-sandbox ... (10 actions, 9 running)
[1,093 / 1,363] Compiling lib/urlapi.c; 0s darwin-sandbox ... (10 actions, 9 running)
[1,235 / 1,438] Compiling tensorstore/internal/oauth2/oauth_utils.cc; 2s darwin-sandbox ... (10 actions, 9 running)
[1,364 / 1,539] Compiling python/tensorstore/chunk_layout.cc; 6s darwin-sandbox ... (10 actions, 9 running)
INFO: From Compiling python/tensorstore/kvstore.cc:
In file included from python/tensorstore/kvstore.cc:21:
In file included from ./python/tensorstore/future.h:63:
./python/tensorstore/garbage_collection.h:269:38: warning: offset of on non-standard-layout type 'tensorstore::internal_python::PythonKvStoreObject' [-Winvalid-offsetof]
python_type->tp_weaklistoffset = offsetof(DerivedPythonObject, weakrefs);
^ ~~~~~~~~
/Library/Developer/CommandLineTools/usr/lib/clang/13.0.0/include/stddef.h:104:24: note: expanded from macro 'offsetof'
#define offsetof(t, d) builtin_offsetof(t, d)
^ ~
python/tensorstore/kvstore.cc:181:35: note: in instantiation of member function 'tensorstore::internal_python::GarbageCollectedPythonObject<tensorstore::internal_python::PythonKvStoreObject, tensorstore::kvstore::KvStore>::Define' requested here
auto cls = PythonKvStoreObject::Define(R"(
^
1 warning generated.
[1,595 / 1,745] Compiling tensorstore/driver/neuroglancer_precomputed/driver.cc; 4s darwin-sandbox ... (10 actions, 9 running)
[1,776 / 1,892] Compiling python/tensorstore/index_space.cc; 8s darwin-sandbox ... (10 actions, 9 running)
INFO: From Compiling blosc/shuffle.c:
external/org_blosc_cblosc/blosc/shuffle.c:306:4: warning: Hardware-acceleration detection not implemented for the target architecture. Only the generic shuffle/unshuffle routines will be available. [-W#warnings]
#warning Hardware-acceleration detection not implemented for the target architecture. Only the generic shuffle/unshuffle routines will be available.
^
external/org_blosc_cblosc/blosc/shuffle.c:316:22: warning: unused variable 'cpu_features' [-Wunused-variable]
blosc_cpu_features cpu_features = blosc_get_cpu_features();
^
In file included from external/org_blosc_cblosc/blosc/shuffle.c:12:
external/org_blosc_cblosc/blosc/shuffle-generic.h:32:13: warning: unused function 'shuffle_generic_inline' [-Wunused-function]
static void shuffle_generic_inline(const size_t type_size,
^
external/org_blosc_cblosc/blosc/shuffle-generic.h:61:13: warning: unused function 'unshuffle_generic_inline' [-Wunused-function]
static void unshuffle_generic_inline(const size_t type_size,
^
4 warnings generated.
[1,950 / 2,021] Compiling python/tensorstore/downsample.cc; 2s darwin-sandbox ... (10 actions, 9 running)
ERROR: /private/var/tmp/bazel_ibulu/fa7d43dc29afa7aa0093ed0eb5c0237a/external/org_aomedia_aom/BUILD.bazel:580:11: Compiling aom_dsp/arm/subpel_variance_neon.c failed: (Exit 1): cc_wrapper.sh failed: error executing command
(cd /private/var/tmp/bazel_ibulu/fa7d43dc29afa7aa0093ed0eb5c0237a/sandbox/darwin-sandbox/980/execroot/com_google_tensorstore &&
exec env -
PATH=/bin:/usr/bin:/usr/local/bin
PWD=/proc/self/cwd
external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wno-free-nonheap-object -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections -MD -MF bazel-out/darwin_arm64-opt/bin/external/org_aomedia_aom/objs/dsp/subpel_variance_neon.pic.d '-frandom-seed=bazel-out/darwin_arm64-opt/bin/external/org_aomedia_aom/objs/dsp/subpel_variance_neon.pic.o' -fPIC -iquote external/org_aomedia_aom -iquote bazel-out/darwin_arm64-opt/bin/external/org_aomedia_aom '-fdiagnostics-color=always' -Wno-deprecated-declarations -Wno-sign-compare '-std=c99' -DNEON_INTRINSICS -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE="redacted"' '-D__TIMESTAMP
="redacted"' '-D__TIME
="redacted"' -c external/org_aomedia_aom/aom_dsp/arm/subpel_variance_neon.c -o bazel-out/darwin_arm64-opt/bin/external/org_aomedia_aom/_objs/dsp/subpel_variance_neon.pic.o)
# Configuration: 245e3f3149faf5759d46c93b8151f91ec30af73f4a7c5da9a071b398cad67b5a
# Execution platform: @local_config_platform//:host

  Use --sandbox_debug to see verbose messages from the sandbox
  external/org_aomedia_aom/aom_dsp/arm/subpel_variance_neon.c:121:10: error: implicit declaration of function 'aom_variance8x8_neon' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    return aom_variance8x8_neon(temp2, 8, dst, dst_stride, sse);
           ^
  external/org_aomedia_aom/aom_dsp/arm/subpel_variance_neon.c:121:10: note: did you mean 'aom_variance8x8_c'?
  bazel-out/darwin_arm64-opt/bin/external/org_aomedia_aom/config/aom_dsp_rtcd.h:4729:14: note: 'aom_variance8x8_c' declared here
  unsigned int aom_variance8x8_c(const uint8_t *src_ptr, int source_stride, const uint8_t *ref_ptr, int ref_stride, unsigned int *sse);
               ^
  external/org_aomedia_aom/aom_dsp/arm/subpel_variance_neon.c:136:10: error: implicit declaration of function 'aom_variance16x16_neon' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    return aom_variance16x16_neon(temp2, 16, dst, dst_stride, sse);
           ^
  external/org_aomedia_aom/aom_dsp/arm/subpel_variance_neon.c:136:10: note: did you mean 'aom_variance16x16_c'?
  bazel-out/darwin_arm64-opt/bin/external/org_aomedia_aom/config/aom_dsp_rtcd.h:4663:14: note: 'aom_variance16x16_c' declared here
  unsigned int aom_variance16x16_c(const uint8_t *src_ptr, int source_stride, const uint8_t *ref_ptr, int ref_stride, unsigned int *sse);
               ^
  external/org_aomedia_aom/aom_dsp/arm/subpel_variance_neon.c:151:10: error: implicit declaration of function 'aom_variance32x32_neon' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    return aom_variance32x32_neon(temp2, 32, dst, dst_stride, sse);
           ^
  external/org_aomedia_aom/aom_dsp/arm/subpel_variance_neon.c:151:10: note: did you mean 'aom_variance32x32_c'?
  bazel-out/darwin_arm64-opt/bin/external/org_aomedia_aom/config/aom_dsp_rtcd.h:4687:14: note: 'aom_variance32x32_c' declared here
  unsigned int aom_variance32x32_c(const uint8_t *src_ptr, int source_stride, const uint8_t *ref_ptr, int ref_stride, unsigned int *sse);
               ^
  external/org_aomedia_aom/aom_dsp/arm/subpel_variance_neon.c:166:10: error: implicit declaration of function 'aom_variance64x64_neon' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    return aom_variance64x64_neon(temp2, 64, dst, dst_stride, sse);
           ^
  external/org_aomedia_aom/aom_dsp/arm/subpel_variance_neon.c:166:10: note: did you mean 'aom_variance64x64_c'?
  bazel-out/darwin_arm64-opt/bin/external/org_aomedia_aom/config/aom_dsp_rtcd.h:4717:14: note: 'aom_variance64x64_c' declared here
  unsigned int aom_variance64x64_c(const uint8_t *src_ptr, int source_stride, const uint8_t *ref_ptr, int ref_stride, unsigned int *sse);
               ^
  4 errors generated.
  Target //python/tensorstore:_tensorstore__shared_objects failed to build
  INFO: Elapsed time: 81.578s, Critical Path: 11.63s
  INFO: 2003 processes: 1023 internal, 980 darwin-sandbox.
  FAILED: Build did NOT complete successfully
  FAILED: Build did NOT complete successfully
  error: command '/Users/ibulu/opt/miniconda3/bin/python' failed with exit code 1
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tensorstore
Failed to build tensorstore
ERROR: Could not build wheels for tensorstore, which is required to install pyproject.toml-based projects

@jbms
Copy link
Collaborator

jbms commented Apr 23, 2022

Thanks --- we'll look into that issue. Can you share your version of xcode and macos ?

@jbms
Copy link
Collaborator

jbms commented Apr 23, 2022

With Python 3.8.9 on macOS 12.2.1 and xcode 13.3.1 I am able to build it successfully.

@ibulu
Copy link
Author

ibulu commented Apr 23, 2022

my python version is 3.8.13, macOS is 12.2.1 and Xcode command line utils is 13.3.1

@laramiel
Copy link
Collaborator

laramiel commented Apr 23, 2022

Another data point; installation works for me on MacOS 12.3.1 and xcode 13.2.1

$ python3 --version
Python 3.8.9

$ uname -v
Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000

$ clang --version
Apple clang version 13.1.6 (clang-1316.0.21.2.3)
Target: arm64-apple-darwin21.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

You could try a non pip build:

git clone https://github.com/google/tensorstore.git
cd tensorstore
./bazelisk.py build //tensorstore

@ibulu
Copy link
Author

ibulu commented Apr 23, 2022

build was successful with the Bazelisk.py. Do I need to do anything else to install the package after build?

@jbms
Copy link
Collaborator

jbms commented Apr 23, 2022

Building just the //tensorstore target does not actually build the Python module nor does it pull in the third-party AOM dependency that was causing you a problem.

Until we can sort out the AOM build failure, one workaround is to disable the image/avif driver that pulls in the AOM dependency. To do that, delete this line:

Then you can install the package with:

pip install .

from the root of the checkout.

@laramiel
Copy link
Collaborator

laramiel commented Apr 23, 2022

Before you move on, try this:

bazelisk.py build //tensorstore/driver:all_drivers

If that fails, yeah, remove the line and do as jeremy suggests.

@ibulu
Copy link
Author

ibulu commented Apr 23, 2022

Building just the //tensorstore target does not actually build the Python module nor does it pull in the third-party AOM dependency that was causing you a problem.

Until we can sort out the AOM build failure, one workaround is to disable the image/avif driver that pulls in the AOM dependency. To do that, delete this line:

Then you can install the package with:

pip install .

from the root of the checkout.

yep..that worked

@laramiel
Copy link
Collaborator

laramiel commented Apr 26, 2022

I tried to install the xcode 13.3.1 command line tools, but that doesn't appear to have changed anything for me.

Looking at this, the missing symbol is from the file variance_neon.cc in the AOM codec.

This file should be selected into the build when the config :arm64 is selected, which resolves to the root BUILD.bazel file where the rule for "arm64" is defined based on the config_setting { "cpu" } targets. We might want to adjust those constraints to use @platforms:aarch64.

Nevertheless it would be nice to know which --cpu target the bazel build has deduced. Maybe it's attempting a fat build with M1 + x86?

You can probably get the full build working by forcing it to build with --cpu=darwin_arm64. Editing the .bazelrc file could do that.

cat >> .bazelrc <EOF
build:macos --cpu=darwin_arm64
EOF

git restore tensorstore/tensorstore/driver/BUILD  
pip install .

@ibulu
Copy link
Author

ibulu commented Apr 29, 2022

I ll give this a try

@ibulu
Copy link
Author

ibulu commented Apr 30, 2022

getting the following error:

Building wheels for collected packages: tensorstore
Building wheel for tensorstore (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for tensorstore (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [187 lines of output]
running bdist_wheel
running build
running build_py
creating /var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/tmpd_1_5sbp/lib.macosx-11.0-arm64-cpython-39
creating /var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/tmpd_1_5sbp/lib.macosx-11.0-arm64-cpython-39/tensorstore
copying python/tensorstore/cc_test_driver_main.py -> /var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/tmpd_1_5sbp/lib.macosx-11.0-arm64-cpython-39/tensorstore
copying python/tensorstore/init.py -> /var/folders/xs/thxh4s7s4m7dl5p9qn1bq4ym0000gp/T/tmpd_1_5sbp/lib.macosx-11.0-arm64-cpython-39/tensorstore
running build_ext
/Users/ib/opt/miniconda3/bin/python3.9 -u bazelisk.py build -c opt //python/tensorstore:_tensorstore__shared_objects --verbose_failures --macos_minimum_os=11.0
Starting local Bazel server and connecting to it...
Loading:
Loading: 0 packages loaded
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (1 packages loaded, 0 targets configured)
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (119 packages loaded, 3627 targets configured)
Analyzing: target //python/tensorstore:_tensorstore__shared_objects (121 packages loaded, 5444 targets configured)
INFO: Analyzed target //python/tensorstore:_tensorstore__shared_objects (122 packages loaded, 5463 targets configured).
INFO: Found 1 target...
[0 / 490] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[199 / 654] [Sched] Executing genrule @local_config_python//:python_include
[202 / 654] Executing genrule @local_config_python//:python_include; 1s darwin-sandbox
[644 / 931] Compiling tensorstore/kvstore/gcs/validate.cc; 0s darwin-sandbox ... (7 actions running)
ERROR: /private/var/tmp/_bazel_ib/65418f1368a1334ac0a3082efdb71bb6/external/com_google_absl/absl/debugging/BUILD.bazel:30:11: Compiling absl/debugging/stacktrace.cc failed: undeclared inclusion(s) in rule '@com_google_absl//absl/debugging:stacktrace':
this rule is missing dependency declarations for the following files included by 'absl/debugging/stacktrace.cc':
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/limits.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__config'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__config_site'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/pthread.h'
'/Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/limits.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/limits.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_symbol_aliasing.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_posix_availability.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/limits.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/limits.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/_limits.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/syslimits.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/cstddef'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/version'
'/Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/stddef.h'
'/Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/__stddef_max_align_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__nullptr'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/Availability.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/AvailabilityVersions.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/TargetConditionals.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/ciso646'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/any'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/optional'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/variant'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/string_view'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/atomic'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__availability'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__thread/poll_with_backoff.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/chrono'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/compare'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/type_traits'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__utility/forward.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__utility/move.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__undef_macros'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/ctime'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/time.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/_types.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/_types.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_types.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_clock_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/types.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/types.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int8_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int16_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int32_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int64_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_u_int8_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_u_int16_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_u_int32_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_u_int64_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_intptr_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_uintptr_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_null.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_size_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_time_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_timespec.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/limits'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/ratio'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/climits'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/cstdint'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/stdint.h'
'/Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/stdint.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdint.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_uint8_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_uint16_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_uint32_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_uint64_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_intmax_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_uintmax_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/cstring'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/string.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_ssize_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__threading_support'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/errno.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/errno.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/errno.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/iosfwd'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__mbstate_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/wchar.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/stddef.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_mbstate_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_ct_rune_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_rune_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_wchar_t.h'
'/Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/stdarg.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/stdio.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_va_list.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/stdio.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_ctermid.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_off_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_wctype.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/__wctype.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_wint_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_wctype_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/ctype.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/ctype.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_ctype.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/runetype.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/pthread/sched.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/pthread/pthread_impl.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_attr_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_cond_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_condattr_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_key_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_mutex_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_mutexattr_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_once_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_rwlock_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_rwlockattr_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/pthread/qos.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/qos.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_mach_port_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_sigset_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sched.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/assert.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/stdlib.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/wait.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_pid_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_id_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/signal.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/appleapiopts.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/signal.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/signal.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/_mcontext.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/_mcontext.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach/machine/_structs.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach/arm/_structs.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_sigaltstack.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_ucontext.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_uid_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/resource.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_timeval.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/endian.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/endian.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_endian.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/libkern/_OSByteOrder.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/libkern/arm/OSByteOrder.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/arch.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/alloca.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/malloc/_malloc.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_dev_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_mode_t.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/execinfo.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/os/base.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/os/availability.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/uuid/uuid.h'
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_uuid_t.h'
Target //python/tensorstore:_tensorstore__shared_objects failed to build
INFO: Elapsed time: 9.943s, Critical Path: 4.08s
INFO: 15 processes: 13 internal, 2 darwin-sandbox.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
error: command '/Users/ib/opt/miniconda3/bin/python3.9' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tensorstore
Failed to build tensorstore
ERROR: Could not build wheels for tensorstore, which is required to install pyproject.toml-based projects

@ShourjaMukherjee
Copy link

any updates on a direct fix for the build problem?

@jbms
Copy link
Collaborator

jbms commented Jul 30, 2022

The issue described in #33 (comment) appears to be a problem in Bazel itself, since it is considering those system headers to be undeclared inclusions. As a starting point, if you can try just building abseil itself using Bazel on the machine that triggers that issue, that would indicate whether it is a more general bazel issue or something specific to the bazel configuration used by tensorstore.

It is also possible that upgrading to a newer version of Bazel (change it in .bazelversion) could fix the issue, if this issue happened to already have been fixed in Bazel.

However, there is good news: we are now able to build macOS arm64 binary packages on Github actions (expected to be released in the next couple days), which in many cases will eliminate the need to build at all and thereby avoid possible build issues. In fact you can download a prerelease version of the binary packages from here:

https://github.com/google/tensorstore/actions/runs/2763457110

Download and extract the arm64 zip file, and then you can install the package for your Python version using pip install.

@jbms
Copy link
Collaborator

jbms commented Aug 2, 2022

As of 0.1.22, there are now macOS arm64 binary packages available on PyPI.

@jbms
Copy link
Collaborator

jbms commented Sep 26, 2022

I believe the "undeclared inclusions" issue reported by @ibulu may be a Bazel bug that is fixed simply by deleting the bazel cache:

python3 bazelisk.py clean --expunge

@jbms jbms closed this as completed Sep 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants