Apple Silicon not supported #17

truedichotomy · 2021-11-29T22:50:06Z

I was trying to install GibbsSeaWater on an Apple M1 laptop and got the following error:

(@ocean) pkg> add GibbsSeaWater
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
   Installed BinaryProvider ─ v0.5.10
   Installed GibbsSeaWater ── v0.1.1
    Updating `~/.julia/environments/ocean/Project.toml`
  [9a22fb26] + GibbsSeaWater v0.1.1
    Updating `~/.julia/environments/ocean/Manifest.toml`
  [b99e7846] + BinaryProvider v0.5.10
  [9a22fb26] + GibbsSeaWater v0.1.1
    Building GibbsSeaWater → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/9a48963e25eb85ace3fa50706f10be2a29bc84bc/build.log`
ERROR: Error building `GibbsSeaWater`: 
┌ Warning: Platform `arm64-apple-darwin21.1.0` is not an officially supported platform
└ @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:450
ERROR: LoadError: KeyError: key "unknown" not found
Stacktrace:
  [1] getindex
    @ ./dict.jl:481 [inlined]
  [2] parse_dl_name_version
    @ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:511 [inlined]
  [3] detect_libgfortran_abi(libgfortran_name::String, platform::BinaryProvider.UnknownPlatform) (repeats 2 times)
    @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:555
  [4] detect_libgfortran_abi
    @ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:584 [inlined]
  [5] detect_compiler_abi()
    @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:667
  [6] top-level scope
    @ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:685
  [7] include(mod::Module, _path::String)
    @ Base ./Base.jl:418
  [8] include(x::String)
    @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:1
  [9] top-level scope
    @ ~/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:12
 [10] include
    @ ./Base.jl:418 [inlined]
 [11] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1318
 [12] top-level scope
    @ none:1
 [13] eval
    @ ./boot.jl:373 [inlined]
 [14] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [15] top-level scope
    @ none:1
in expression starting at /Users/gong/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:685
in expression starting at /Users/gong/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:1
ERROR: LoadError: Failed to precompile BinaryProvider [b99e7846-7c00-51b0-8f62-c81ae34c0232] to /Users/gong/.julia/compiled/v1.7/BinaryProvider/jl_awjTD7.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base ./loading.jl:1466
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1410
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1120
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997
 [7] include(fname::String)
   @ Base.MainInclude ./client.jl:451
 [8] top-level scope
   @ none:5
in expression starting at /Users/gong/.julia/packages/GibbsSeaWater/vq3GE/deps/build.jl:1

Julia Version 1.7.0-rc3
Commit 3348de4ea6 (2021-11-15 08:22 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin21.1.0)
  CPU: Apple M1
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, cyclone)

The text was updated successfully, but these errors were encountered:

truedichotomy · 2022-06-08T16:06:04Z

Is it possible to remove GibbsSeaWater.jl's dependence on BinaryProvider.jl? It has not been updated for over 2 years. This prevents GibbsSeaWater.jl from working on any recent Macs. Most binary dependencies are built as JLL using BinaryBuilder.jl nowadays right? Can GibbsSeaWater.jl work that way as well? My work around is using GSW in python and do a PyCall. It works but I'd like to see a Julia native package.

kouketsu · 2022-06-08T22:53:16Z

As I have no M1 Mac (I want it :)), I have not tested and I'm sorry that I respond to this too late.
As @truedichotomy pointed out, at least I should fix the dependency of the binary library.

I have done now (but I can't test it on M1 Mac), please try @truedichotomy (pkg> add GibbsSeaWater#master).

truedichotomy · 2022-06-09T02:08:13Z

I just tried the updated master branch and encountered the following:

(@ocean) pkg> add https://github.com/TEOS-10/GibbsSeaWater.jl
    Updating git-repo `https://github.com/TEOS-10/GibbsSeaWater.jl`
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/.julia/environments/ocean/Project.toml`
  [9a22fb26] ~ GibbsSeaWater v0.1.1 `https://github.com/truedichotomy/GibbsSeaWater.jl#master` ⇒ v0.1.1 `https://github.com/TEOS-10/GibbsSeaWater.jl#master`
    Updating `~/.julia/environments/ocean/Manifest.toml`
  [b99e7846] + BinaryProvider v0.5.10
  [9a22fb26] ~ GibbsSeaWater v0.1.1 `https://github.com/truedichotomy/GibbsSeaWater.jl#master` ⇒ v0.1.1 `https://github.com/TEOS-10/GibbsSeaWater.jl#master`
Precompiling project...
  ✗ BinaryProvider
  ✗ GibbsSeaWater
  0 dependencies successfully precompiled in 3 seconds. 577 already precompiled.
  2 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages

julia> import Pkg

julia> Pkg.precompile()
Precompiling project...
  ✗ BinaryProvider
  ✗ GibbsSeaWater
  0 dependencies successfully precompiled in 2 seconds. 577 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

GibbsSeaWater [9a22fb26-0b63-4589-b28e-8f9d0b5c3d05]

Failed to precompile GibbsSeaWater [9a22fb26-0b63-4589-b28e-8f9d0b5c3d05] to /Users/gong/.julia/compiled/v1.9/GibbsSeaWater/jl_ipG8JM.
ERROR: LoadError: UndefVarError: libgswteos_path not defined
Stacktrace:
 [1] getproperty(x::Module, f::Symbol)
   @ Base ./Base.jl:31
 [2] top-level scope
   @ ~/.julia/packages/GibbsSeaWater/Wfekc/src/GibbsSeaWater.jl:7
 [3] include
   @ ./Base.jl:428 [inlined]
 [4] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base ./loading.jl:1475
 [5] top-level scope
   @ stdin:1
in expression starting at /Users/gong/.julia/packages/GibbsSeaWater/Wfekc/src/GibbsSeaWater.jl:1
in expression starting at stdin:1
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types ~/GitHub/julia/usr/share/julia/stdlib/v1.9/Pkg/src/Types.jl:68
 [2] precompile(ctx::Pkg.Types.Context, pkgs::Vector{String}; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API ~/GitHub/julia/usr/share/julia/stdlib/v1.9/Pkg/src/API.jl:1427
 [3] precompile
   @ ~/GitHub/julia/usr/share/julia/stdlib/v1.9/Pkg/src/API.jl:1058 [inlined]
 [4] #precompile#225
   @ ~/GitHub/julia/usr/share/julia/stdlib/v1.9/Pkg/src/API.jl:1057 [inlined]
 [5] precompile (repeats 2 times)
   @ ~/GitHub/julia/usr/share/julia/stdlib/v1.9/Pkg/src/API.jl:1057 [inlined]
 [6] top-level scope
   @ REPL[4]:1

I see a couple of issues, one is the continued specified dependence on BinaryProvider.jl which presumably can simply be removed as I don't see any real dependence on it at a glance? Second, there is no binaries for aarch64 for macOS provided. Now I can build the binaries directly by compiling the GSW-C library manually on my mac and produce libgswteos-10.so, however I don't really know how to use BinaryBuilder to produce the necessary binary tarball or produce the build_tarballs.jl script. It quits on me at this step:

ERROR: The build has produced no binary artifacts.
       This is generally because an error occurred during the build
       or because you forgot to `make install` or equivalent.

libgswteos-10.so is already produced and residing in the directory so I don't know where make install would want to move it to. Anyway, since you are familiar with BinaryBuilder.jl and I have access to a M1 mac, we can try to figure this out?

Alexander-Barth · 2022-06-09T07:00:19Z

For NCDatasets, we use the NetCDF binaries from by https://github.com/JuliaPackaging/Yggdrasil. Here is an initial test to include GSW:

https://github.com/JuliaPackaging/Yggdrasil/blob/master/G/GibbsSeaWater/build_tarballs.jl

Everytime you make a PR to this file, the script is run and new binaries are created here:

https://github.com/JuliaBinaryWrappers/GibbsSeaWater_jll.jl/releases/tag/GibbsSeaWater-v3.5.0%2B0

(currently without M1 binary, because it was not a supported platform at the time).

GibbsSeaWater.jl needs then only dependent on GibbsSeaWater_jll.jl which will define libgswteos_path:

https://github.com/JuliaBinaryWrappers/GibbsSeaWater_jll.jl/blob/master/src/wrappers/x86_64-linux-gnu.jl#L33

It might be sufficient to add a new line to this file to get a Mac M1 binary (which is now in BinaryBuilder.supported_platforms())

kouketsu · 2022-06-10T01:20:26Z

@truedichotomy
I've updated GibbsSeaWater_jll. I prepared the binary aarch64 for macOS (with gcc), but I haven't tested it.

truedichotomy · 2022-06-10T01:52:10Z

@kouketsu I'm able to install now but it wouldn't precompile.

(@ocean) pkg> add https://github.com/TEOS-10/GibbsSeaWater.jl
    Updating git-repo `https://github.com/TEOS-10/GibbsSeaWater.jl`
   Resolving package versions...
    Updating `~/.julia/environments/ocean/Project.toml`
  [9a22fb26] + GibbsSeaWater v0.1.1 `https://github.com/TEOS-10/GibbsSeaWater.jl#master`
    Updating `~/.julia/environments/ocean/Manifest.toml`
  [b99e7846] + BinaryProvider v0.5.10
  [9a22fb26] + GibbsSeaWater v0.1.1 `https://github.com/TEOS-10/GibbsSeaWater.jl#master`
  [6727f6b2] + GibbsSeaWater_jll v3.5.0+0

julia> using GibbsSeaWater
[ Info: Precompiling GibbsSeaWater [9a22fb26-0b63-4589-b28e-8f9d0b5c3d05]
ERROR: LoadError: UndefVarError: libgswteos_path not defined
Stacktrace:
 [1] getproperty(x::Module, f::Symbol)
   @ Base ./Base.jl:31
 [2] top-level scope
   @ ~/.julia/packages/GibbsSeaWater/Wfekc/src/GibbsSeaWater.jl:7
 [3] include
   @ ./Base.jl:418 [inlined]
 [4] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base ./loading.jl:1495
 [5] top-level scope
   @ stdin:1
in expression starting at /Users/gong/.julia/packages/GibbsSeaWater/Wfekc/src/GibbsSeaWater.jl:1
in expression starting at stdin:1
ERROR: Failed to precompile GibbsSeaWater [9a22fb26-0b63-4589-b28e-8f9d0b5c3d05] to /Users/gong/.julia/compiled/v1.9/GibbsSeaWater/jl_cOlIei.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base ./loading.jl:1646
 [3] compilecache
   @ ./loading.jl:1590 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1291
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1164
 [6] macro expansion
   @ ./loading.jl:1144 [inlined]
 [7] macro expansion
   @ ./lock.jl:267 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1108

kouketsu · 2022-06-11T04:11:28Z

Sorry, I've made a mistake in updating.
Please try again @truedichotomy.

truedichotomy · 2022-06-11T10:43:21Z

Thank you!

kouketsu closed this as completed Jun 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apple Silicon not supported #17

Apple Silicon not supported #17

truedichotomy commented Nov 29, 2021

truedichotomy commented Jun 8, 2022

kouketsu commented Jun 8, 2022

truedichotomy commented Jun 9, 2022

Alexander-Barth commented Jun 9, 2022

kouketsu commented Jun 10, 2022

truedichotomy commented Jun 10, 2022

kouketsu commented Jun 11, 2022

truedichotomy commented Jun 11, 2022

Apple Silicon not supported #17

Apple Silicon not supported #17

Comments

truedichotomy commented Nov 29, 2021

truedichotomy commented Jun 8, 2022

kouketsu commented Jun 8, 2022

truedichotomy commented Jun 9, 2022

Alexander-Barth commented Jun 9, 2022

kouketsu commented Jun 10, 2022

truedichotomy commented Jun 10, 2022

kouketsu commented Jun 11, 2022

truedichotomy commented Jun 11, 2022