Can't compile KataGo with Metal backend

[egri-nagy]

XCode is installed

❯ xcode-select -p
/Applications/Xcode.app/Contents/Developer

but I might be missing something as cmake blows up:

❯ cmake -G Ninja -DUSE_BACKEND=METAL
CMake Warning:
  No source or binary directory provided.  Both will be assumed to be the
  same as the current working directory, but note that this warning will
  become a fatal error in future CMake releases.


-- The CXX compiler identification is AppleClang 17.0.0.17000013
-- The Swift compiler identification is Apple 6.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working Swift compiler: /usr/bin/swiftc
-- Check for working Swift compiler: /usr/bin/swiftc - works
-- Building 'katago' executable for GTP engine and other tools.
-- -DUSE_BACKEND=METAL, using Metal backend.
warning: no such SDK: -print-target-info
error: no input files
CMake Error at external/macos/cmake/modules/InitializeSwift.cmake:42 (string):
  string sub-command JSON failed parsing json string: * Line 1, Column 1

    Syntax error: value, object or array expected.

  .
Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)


CMake Error at external/macos/cmake/modules/InitializeSwift.cmake:44 (string):
  string sub-command JSON failed parsing json string: * Line 1, Column 1

    Syntax error: value, object or array expected.

  .
Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)


CMake Error at external/macos/cmake/modules/InitializeSwift.cmake:45 (string):
  string sub-command JSON failed parsing json string: * Line 1, Column 1

    Syntax error: value, object or array expected.

  .
Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)


CMake Warning (dev) at external/macos/cmake/modules/InitializeSwift.cmake:46 (math):
  Unexpected character in expression at position 1: N

  Unexpected character in expression at position 2: O

  Unexpected character in expression at position 3: T

  Unexpected character in expression at position 4: F

  Unexpected character in expression at position 5: O

  Unexpected character in expression at position 6: U

  Unexpected character in expression at position 7: N

  Unexpected character in expression at position 8: D

Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Error at external/macos/cmake/modules/InitializeSwift.cmake:48 (string):
  string sub-command JSON failed parsing json string: * Line 1, Column 1

    Syntax error: value, object or array expected.

  .
Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)


CMake Error at external/macos/cmake/modules/InitializeSwift.cmake:49 (string):
  string sub-command JSON failed parsing json string: * Line 1, Column 1

    Syntax error: value, object or array expected.

  .
Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)


CMake Warning (dev) at external/macos/cmake/modules/InitializeSwift.cmake:50 (math):
  Unexpected character in expression at position 1: N

  Unexpected character in expression at position 2: O

  Unexpected character in expression at position 3: T

  Unexpected character in expression at position 4: F

  Unexpected character in expression at position 5: O

  Unexpected character in expression at position 6: U

  Unexpected character in expression at position 7: N

  Unexpected character in expression at position 8: D

Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Error at external/macos/cmake/modules/InitializeSwift.cmake:55 (string):
  string sub-command JSON failed parsing json string: * Line 1, Column 1

    Syntax error: value, object or array expected.

  .
Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)


CMake Error at external/macos/cmake/modules/InitializeSwift.cmake:55 (string):
  string sub-command JSON failed parsing json string: * Line 1, Column 1

    Syntax error: value, object or array expected.

  .
Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)


CMake Error at external/macos/cmake/modules/InitializeSwift.cmake:60 (string):
  string sub-command JSON failed parsing json string: * Line 1, Column 1

    Syntax error: value, object or array expected.

  .
Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)


CMake Error at external/macos/cmake/modules/InitializeSwift.cmake:60 (string):
  string sub-command JSON failed parsing json string: * Line 1, Column 1

    Syntax error: value, object or array expected.

  .
Call Stack (most recent call first):
  external/macos/cmake/modules/InitializeSwift.cmake:89 (_setup_swift_paths)
  CMakeLists.txt:108 (include)


-- Including Git revision in the compiled executable, specify -DNO_GIT_REVISION=1 to disable
-- Found Git: /usr/bin/git (found version "2.39.5 (Apple Git-154)")
-- Found ZLIB: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/libz.tbd (found version "1.2.12")
-- Setting up build for GNU or Clang.
-- ARM architecture detected: adding -fsigned-char flag
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Enabling AppleClang-specific build options.
-- Configuring incomplete, errors occurred!

Any ideas?

[ChinChangYang]

This is what I see in my environment:

% git clone https://github.com/lightvector/KataGo.git KataGo-temp
Cloning into 'KataGo-temp'...
remote: Enumerating objects: 32198, done.
remote: Counting objects: 100% (163/163), done.
remote: Compressing objects: 100% (71/71), done.
remote: Total 32198 (delta 102), reused 96 (delta 92), pack-reused 32035 (from 2)
Receiving objects: 100% (32198/32198), 75.57 MiB | 16.35 MiB/s, done.
Resolving deltas: 100% (25789/25789), done.
(base) chinchangyang@MacBook-Pro-M3-Max Code % cd KataGo-temp 
(base) chinchangyang@MacBook-Pro-M3-Max KataGo-temp % cd cpp
(base) chinchangyang@MacBook-Pro-M3-Max cpp % cmake . -G Ninja -DUSE_BACKEND=METAL
-- The CXX compiler identification is AppleClang 17.0.0.17000013
-- The Swift compiler identification is Apple 6.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working Swift compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc
-- Check for working Swift compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc - works
-- Building 'katago' executable for GTP engine and other tools.
-- -DUSE_BACKEND=METAL, using Metal backend.
-- Including Git revision in the compiled executable, specify -DNO_GIT_REVISION=1 to disable
-- Found Git: /opt/homebrew/bin/git (found version "2.43.0")
-- Found ZLIB: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.4.sdk/usr/lib/libz.tbd (found version "1.2.12")
-- Setting up build for GNU or Clang.
-- ARM architecture detected: adding -fsigned-char flag
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Enabling AppleClang-specific build options.
-- Configuring done (1.9s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/chinchangyang/Code/KataGo-temp/cpp
(base) chinchangyang@MacBook-Pro-M3-Max cpp % xcode-select -p
/Applications/Xcode.app/Contents/Developer

[egri-nagy]

Ah, that's helpful.

You have
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc
while I have
/usr/bin/swiftc
for the Swift compiler. I don't know why, but I will investigate.

[dfannius]

I had the same issue and was able to fix it by setting the CMAKE_OSX_SYSROOT environment variable. In my case, the command line was

$ cmake . -G Ninja -DUSE_BACKEND=METAL -DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk

[egri-nagy]

Great! This solved the problem. Thank you very much!

I see about 10% speed increase with Metal compared to OpenCL on M1 Max.

Update: the performance gain is lot bigger than that, like 80%. The CPU cores were all taken by another process when I first looked at it.