Fix build on Apple Silicon Mac #219

sticnarf · 2020-12-10T16:39:25Z

Backport Fix compilation on Apple Silicon facebook/rocksdb#7714 from upstream.
uname -m on the new Apple Silicon Mac outputs arm64. The ARMv8 CRC check in the cmake file is changed.
Do not treat arm darwin as iOS.

With this PR, cmake .. -DWITH_GFLAGS=OFF -DWITH_TESTS=OFF -DWITH_TOOLS=OFF outputs:

-- The C compiler identification is AppleClang 12.0.0.12000032
-- The CXX compiler identification is AppleClang 12.0.0.12000032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The ASM compiler identification is Clang
-- Found assembler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Found Git: /usr/bin/git (found version "2.24.3 (Apple Git-128)") 
-- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER
-- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER - Success
-- Performing Test HAS_ARMV8_CRC
-- Performing Test HAS_ARMV8_CRC - Success
--  HAS_ARMV8_CRC yes
-- Performing Test HAVE_SSE42
-- Performing Test HAVE_SSE42 - Failed
-- Performing Test HAVE_THREAD_LOCAL
-- Performing Test HAVE_THREAD_LOCAL - Success
-- Enabling RTTI in Debug builds only (default)
-- Performing Test HAVE_FALLOCATE
-- Performing Test HAVE_FALLOCATE - Failed
-- Performing Test HAVE_SYNC_FILE_RANGE_WRITE
-- Performing Test HAVE_SYNC_FILE_RANGE_WRITE - Failed
-- Performing Test HAVE_PTHREAD_MUTEX_ADAPTIVE_NP
-- Performing Test HAVE_PTHREAD_MUTEX_ADAPTIVE_NP - Failed
-- Looking for malloc_usable_size
-- Looking for malloc_usable_size - not found
-- Looking for sched_getcpu
-- Looking for sched_getcpu - not found
-- Looking for getauxval
-- Looking for getauxval - not found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- JNI library is disabled
-- Configuring done
-- Generating done

And then, make rocksdb -j4 succeeds.

Signed-off-by: Yilin Chen <sticnarf@gmail.com>

sticnarf · 2020-12-11T05:40:52Z

PTAL @yiwu-arbug

Connor1996 · 2020-12-11T06:14:43Z

CMakeLists.txt

@@ -217,14 +217,14 @@ if(CMAKE_SYSTEM_PROCESSOR MATCHES "ppc64le")
  endif(HAS_ALTIVEC)
 endif(CMAKE_SYSTEM_PROCESSOR MATCHES "ppc64le")

-if(CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64|AARCH64")
+if(CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64|AARCH64|arm64")


why the original PR of rocksdb doesn't have this problem?

The developer of that PR uses an early DTK. The CPU of the DTK is different from an M1.

I don't know if it's due to system or CPU updates, however.

Signed-off-by: yiwu-arbug <yiwu@pingcap.com>

yiwu-arbug · 2020-12-14T02:10:07Z

arm build fails, haven't check what's the cause.

Do the same cmake change in tikv/rocksdb#219 Signed-off-by: Yilin Chen <sticnarf@gmail.com>

Signed-off-by: Yilin Chen <sticnarf@gmail.com>

sticnarf · 2020-12-14T06:19:09Z

Backport facebook#5745 to fix the CI failure.

sticnarf · 2020-12-15T05:57:17Z

db/db_wal_test.cc:1464: Failure

Expected: (GetAllocatedFileSize(dbname_ + file_before->PathName())) >= (preallocated_size), actual: 512 vs 18427904

terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException'

  what():  db/db_wal_test.cc:1464: Failure

Expected: (GetAllocatedFileSize(dbname_ + file_before->PathName())) >= (preallocated_size), actual: 512 vs 18427904

This error still exists in upstream: facebook#6236. Can we ignore running platform-dependent tests on CI now like what upstream does? @yiwu-arbug

BusyJay · 2020-12-24T05:44:49Z

Friendly ping @yiwu-arbug @Connor1996

dragonly · 2021-02-05T08:47:58Z

Please merge this for Apple M1 🌚

Signed-off-by: Yilin CHen <sticnarf@gmail.com>

sticnarf · 2021-02-06T09:33:00Z

The test failed on CI probably because Travis uses LXD for arm64 by default. Now I switch the arm64 tests to use virtual machine. The test should now pass.

@yiwu-arbug PTAL again, thanks!

1. Backport facebook#7714 from upstream. 2. `uname -m` on the new Apple Silicon Mac outputs `arm64`. The ARMv8 CRC check in the cmake file is changed. 3. Do not treat arm darwin as iOS. With this PR, ` cmake .. -DWITH_GFLAGS=OFF -DWITH_TESTS=OFF -DWITH_TOOLS=OFF` outputs: ``` -- The C compiler identification is AppleClang 12.0.0.12000032 -- The CXX compiler identification is AppleClang 12.0.0.12000032 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- The ASM compiler identification is Clang -- Found assembler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- Found Git: /usr/bin/git (found version "2.24.3 (Apple Git-128)") -- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER -- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER - Success -- Performing Test HAS_ARMV8_CRC -- Performing Test HAS_ARMV8_CRC - Success -- HAS_ARMV8_CRC yes -- Performing Test HAVE_SSE42 -- Performing Test HAVE_SSE42 - Failed -- Performing Test HAVE_THREAD_LOCAL -- Performing Test HAVE_THREAD_LOCAL - Success -- Enabling RTTI in Debug builds only (default) -- Performing Test HAVE_FALLOCATE -- Performing Test HAVE_FALLOCATE - Failed -- Performing Test HAVE_SYNC_FILE_RANGE_WRITE -- Performing Test HAVE_SYNC_FILE_RANGE_WRITE - Failed -- Performing Test HAVE_PTHREAD_MUTEX_ADAPTIVE_NP -- Performing Test HAVE_PTHREAD_MUTEX_ADAPTIVE_NP - Failed -- Looking for malloc_usable_size -- Looking for malloc_usable_size - not found -- Looking for sched_getcpu -- Looking for sched_getcpu - not found -- Looking for getauxval -- Looking for getauxval - not found -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- JNI library is disabled -- Configuring done -- Generating done ``` And then, `make rocksdb -j4` succeeds. Signed-off-by: Yilin CHen <sticnarf@gmail.com> Co-authored-by: yiwu-arbug <yiwu@pingcap.com>

Do the same cmake change in tikv/rocksdb#219 Signed-off-by: Yilin Chen <sticnarf@gmail.com>

1. Backport facebook#7714 from upstream. 2. `uname -m` on the new Apple Silicon Mac outputs `arm64`. The ARMv8 CRC check in the cmake file is changed. 3. Do not treat arm darwin as iOS. With this PR, ` cmake .. -DWITH_GFLAGS=OFF -DWITH_TESTS=OFF -DWITH_TOOLS=OFF` outputs: ``` -- The C compiler identification is AppleClang 12.0.0.12000032 -- The CXX compiler identification is AppleClang 12.0.0.12000032 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- The ASM compiler identification is Clang -- Found assembler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- Found Git: /usr/bin/git (found version "2.24.3 (Apple Git-128)") -- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER -- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER - Success -- Performing Test HAS_ARMV8_CRC -- Performing Test HAS_ARMV8_CRC - Success -- HAS_ARMV8_CRC yes -- Performing Test HAVE_SSE42 -- Performing Test HAVE_SSE42 - Failed -- Performing Test HAVE_THREAD_LOCAL -- Performing Test HAVE_THREAD_LOCAL - Success -- Enabling RTTI in Debug builds only (default) -- Performing Test HAVE_FALLOCATE -- Performing Test HAVE_FALLOCATE - Failed -- Performing Test HAVE_SYNC_FILE_RANGE_WRITE -- Performing Test HAVE_SYNC_FILE_RANGE_WRITE - Failed -- Performing Test HAVE_PTHREAD_MUTEX_ADAPTIVE_NP -- Performing Test HAVE_PTHREAD_MUTEX_ADAPTIVE_NP - Failed -- Looking for malloc_usable_size -- Looking for malloc_usable_size - not found -- Looking for sched_getcpu -- Looking for sched_getcpu - not found -- Looking for getauxval -- Looking for getauxval - not found -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- JNI library is disabled -- Configuring done -- Generating done ``` And then, `make rocksdb -j4` succeeds. Signed-off-by: Yilin CHen <sticnarf@gmail.com> Co-authored-by: yiwu-arbug <yiwu@pingcap.com>

Fix build on Apple Silicon Mac

db62805

Signed-off-by: Yilin Chen <sticnarf@gmail.com>

This was referenced Dec 10, 2020

Adapt for apple silicon Mac tikv/titan#199

Merged

Support build on Apple Silicon/M1 tikv/tikv#9192

Closed

Connor1996 reviewed Dec 11, 2020

View reviewed changes

yiwu-arbug added 3 commits December 13, 2020 15:01

Update .travis.yml

c394484

Signed-off-by: yiwu-arbug <yiwu@pingcap.com>

Update .travis.yml

52d6bc6

Signed-off-by: yiwu-arbug <yiwu@pingcap.com>

Update .travis.yml

4faabe7

Signed-off-by: yiwu-arbug <yiwu@pingcap.com>

yiwu-arbug self-requested a review December 14, 2020 02:10

yiwu-arbug pushed a commit to tikv/titan that referenced this pull request Dec 14, 2020

Adapt for apple silicon Mac (#199)

531a25e

Do the same cmake change in tikv/rocksdb#219 Signed-off-by: Yilin Chen <sticnarf@gmail.com>

backport facebook#5745

a2ffcaa

Signed-off-by: Yilin Chen <sticnarf@gmail.com>

Merge branch '6.4.tikv' into apple-silicon

aae513c

shhdgit mentioned this pull request Feb 2, 2021

Add support for darwin/arm64 (Apple M1) pingcap/tiup#1122

Closed

sticnarf added 2 commits February 6, 2021 15:14

Merge remote-tracking branch 'upstream/6.4.tikv' into apple-silicon

ea15e5f

Signed-off-by: Yilin CHen <sticnarf@gmail.com>

use vm arm64 for CI

7f24690

Signed-off-by: Yilin CHen <sticnarf@gmail.com>

sticnarf force-pushed the apple-silicon branch from 6c42618 to 7f24690 Compare February 6, 2021 08:28

yiwu-arbug merged commit 0b76f87 into tikv:6.4.tikv Feb 6, 2021

ti-srebot mentioned this pull request Feb 6, 2021

rocksdb: Fix build on Apple Silicon Mac tikv/rust-rocksdb#604

Closed

Thegaram pushed a commit to Thegaram/titan that referenced this pull request Apr 2, 2021

Adapt for apple silicon Mac (tikv#199)

4a09973

Do the same cmake change in tikv/rocksdb#219 Signed-off-by: Yilin Chen <sticnarf@gmail.com>

tabokie mentioned this pull request May 9, 2022

Upgrade to facebook 6.29 #277

Closed

39 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix build on Apple Silicon Mac #219

Fix build on Apple Silicon Mac #219

sticnarf commented Dec 10, 2020 •

edited

Loading

sticnarf commented Dec 11, 2020

Connor1996 Dec 11, 2020

sticnarf Dec 11, 2020

yiwu-arbug commented Dec 14, 2020

sticnarf commented Dec 14, 2020

sticnarf commented Dec 15, 2020

BusyJay commented Dec 24, 2020

dragonly commented Feb 5, 2021

sticnarf commented Feb 6, 2021

Fix build on Apple Silicon Mac #219

Fix build on Apple Silicon Mac #219

Conversation

sticnarf commented Dec 10, 2020 • edited Loading

sticnarf commented Dec 11, 2020

Connor1996 Dec 11, 2020

Choose a reason for hiding this comment

sticnarf Dec 11, 2020

Choose a reason for hiding this comment

yiwu-arbug commented Dec 14, 2020

sticnarf commented Dec 14, 2020

sticnarf commented Dec 15, 2020

BusyJay commented Dec 24, 2020

dragonly commented Feb 5, 2021

sticnarf commented Feb 6, 2021

sticnarf commented Dec 10, 2020 •

edited

Loading