Building for wasm on macOS 10.14

[howardwu]

I was able to build rust-secp256k1 in a Ubuntu vagrant box, however am still running into issues on macOS 10.14.

The error originates from depend/secp256k1/contrib/lax_der_parsing.c:7 and is caused by /usr/include/machine/_types.h:34:2: error: architecture not supported.

Clang version:

$ clang --version
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I have tried separately installing and compiling with Clang 8 via Homebrew, and receive the same compilation issue.

Below is the full output:

$ wasm-pack build
[INFO]: 🎯  Checking for the Wasm target...
[INFO]: 🌀  Compiling to Wasm...
   Compiling secp256k1 v0.14.1
error: failed to run custom build command for `secp256k1 v0.14.1`

Caused by:
  process didn't exit successfully: `/wasm-demo/target/release/build/secp256k1-babdfc45bf44ba99/build-script-build` (exit code: 101)
--- stdout
TARGET = Some("wasm32-unknown-unknown")
OPT_LEVEL = Some("3")
HOST = Some("x86_64-apple-darwin")
CC_wasm32-unknown-unknown = None
CC_wasm32_unknown_unknown = None
TARGET_CC = None
CC = None
CFLAGS_wasm32-unknown-unknown = None
CFLAGS_wasm32_unknown_unknown = None
TARGET_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
CC_wasm32-unknown-unknown = None
CC_wasm32_unknown_unknown = None
TARGET_CC = None
CC = None
CFLAGS_wasm32-unknown-unknown = None
CFLAGS_wasm32_unknown_unknown = None
TARGET_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
running: "clang" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=wasm32-unknown-unknown" "-I" "depend/secp256k1/" "-I" "depend/secp256k1/include" "-I" "depend/secp256k1/src" "-Wall" "-Wextra" "-DSECP256K1_BUILD=1" "-DUSE_NUM_NONE=1" "-DUSE_FIELD_INV_BUILTIN=1" "-DUSE_SCALAR_INV_BUILTIN=1" "-DENABLE_MODULE_ECDH=1" "-DUSE_EXTERNAL_DEFAULT_CALLBACKS=1" "-DECMULT_WINDOW_SIZE=15" "-DUSE_FIELD_10X26=1" "-DUSE_SCALAR_8X32=1" "-o" "/wasm-demo/target/wasm32-unknown-unknown/release/build/secp256k1-c0bc0977ed6381c0/out/depend/secp256k1/contrib/lax_der_parsing.o" "-c" "depend/secp256k1/contrib/lax_der_parsing.c"
cargo:warning=In file included from depend/secp256k1/contrib/lax_der_parsing.c:7:
cargo:warning=In file included from /usr/include/string.h:61:
cargo:warning=In file included from /usr/include/_types.h:27:
cargo:warning=In file included from /usr/include/sys/_types.h:32:
cargo:warning=/usr/include/sys/cdefs.h:784:2: error: Unsupported architecture
cargo:warning=#error Unsupported architecture
cargo:warning= ^
cargo:warning=In file included from depend/secp256k1/contrib/lax_der_parsing.c:7:
cargo:warning=In file included from /usr/include/string.h:61:
cargo:warning=In file included from /usr/include/_types.h:27:
cargo:warning=In file included from /usr/include/sys/_types.h:33:
cargo:warning=/usr/include/machine/_types.h:34:2: error: architecture not supported
cargo:warning=#error architecture not supported
cargo:warning= ^
cargo:warning=In file included from depend/secp256k1/contrib/lax_der_parsing.c:7:
cargo:warning=In file included from /usr/include/string.h:61:
cargo:warning=In file included from /usr/include/_types.h:27:
cargo:warning=/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'; did you mean '__int128_t'?
cargo:warning=typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */
cargo:warning=        ^
cargo:warning=note: '__int128_t' declared here
cargo:warning=/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
cargo:warning=typedef __int32_t       __darwin_blksize_t;     /* preferred block size */
cargo:warning=        ^
cargo:warning=note: '__int128_t' declared here
cargo:warning=/usr/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
cargo:warning=typedef __int32_t       __darwin_dev_t;         /* dev_t */
cargo:warning=        ^
cargo:warning=note: '__int128_t' declared here
cargo:warning=/usr/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
cargo:warning=typedef __uint32_t      __darwin_gid_t;         /* [???] process and group IDs */
cargo:warning=        ^
cargo:warning=note: '__uint128_t' declared here
cargo:warning=/usr/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
cargo:warning=typedef __uint32_t      __darwin_id_t;          /* [XSI] pid_t, uid_t, or gid_t*/
cargo:warning=        ^
cargo:warning=note: '__uint128_t' declared here
cargo:warning=/usr/include/sys/_types.h:62:9: error: unknown type name '__uint64_t'; did you mean '__uint128_t'?
cargo:warning=typedef __uint64_t      __darwin_ino64_t;       /* [???] Used for 64 bit inodes */
cargo:warning=        ^
cargo:warning=note: '__uint128_t' declared here
cargo:warning=/usr/include/sys/_types.h:68:9: error: unknown type name '__darwin_natural_t'
cargo:warning=typedef __darwin_natural_t __darwin_mach_port_name_t; /* Used by mach */
cargo:warning=        ^
cargo:warning=/usr/include/sys/_types.h:70:9: error: unknown type name '__uint16_t'; did you mean '__uint128_t'?
cargo:warning=typedef __uint16_t      __darwin_mode_t;        /* [???] Some file attributes */
cargo:warning=        ^
cargo:warning=note: '__uint128_t' declared here
cargo:warning=/usr/include/sys/_types.h:71:9: error: unknown type name '__int64_t'; did you mean '__int128_t'?
cargo:warning=typedef __int64_t       __darwin_off_t;         /* [???] Used for file sizes */
cargo:warning=        ^
cargo:warning=note: '__int128_t' declared here
cargo:warning=/usr/include/sys/_types.h:72:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
cargo:warning=typedef __int32_t       __darwin_pid_t;         /* [???] process and group IDs */
cargo:warning=        ^
cargo:warning=note: '__int128_t' declared here
cargo:warning=/usr/include/sys/_types.h:73:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
cargo:warning=typedef __uint32_t      __darwin_sigset_t;      /* [???] signal set */
cargo:warning=        ^
cargo:warning=note: '__uint128_t' declared here
cargo:warning=/usr/include/sys/_types.h:74:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
cargo:warning=typedef __int32_t       __darwin_suseconds_t;   /* [???] microseconds */
cargo:warning=        ^
cargo:warning=note: '__int128_t' declared here
cargo:warning=/usr/include/sys/_types.h:75:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
cargo:warning=typedef __uint32_t      __darwin_uid_t;         /* [???] user IDs */
cargo:warning=        ^
cargo:warning=note: '__uint128_t' declared here
cargo:warning=/usr/include/sys/_types.h:76:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
cargo:warning=typedef __uint32_t      __darwin_useconds_t;    /* [???] microseconds */
cargo:warning=        ^
cargo:warning=note: '__uint128_t' declared here
cargo:warning=In file included from depend/secp256k1/contrib/lax_der_parsing.c:7:
cargo:warning=In file included from /usr/include/string.h:64:
cargo:warning=/usr/include/sys/_types/_size_t.h:31:9: error: unknown type name '__darwin_size_t'
cargo:warning=typedef __darwin_size_t        size_t;
cargo:warning=        ^
cargo:warning=In file included from depend/secp256k1/contrib/lax_der_parsing.c:7:
cargo:warning=In file included from /usr/include/string.h:141:
cargo:warning=In file included from /usr/include/sys/_types/_rsize_t.h:30:
cargo:warning=/usr/include/machine/types.h:37:2: error: architecture not supported
cargo:warning=#error architecture not supported
cargo:warning= ^
cargo:warning=In file included from depend/secp256k1/contrib/lax_der_parsing.c:7:
cargo:warning=In file included from /usr/include/string.h:141:
cargo:warning=/usr/include/sys/_types/_rsize_t.h:31:9: error: unknown type name '__darwin_size_t'
cargo:warning=typedef __darwin_size_t        rsize_t;
cargo:warning=        ^
cargo:warning=fatal error: too many errors emitted, stopping now [-ferror-limit=]
cargo:warning=20 errors generated.
exit code: 1

--- stderr
thread 'main' panicked at '

Internal error occurred: Command "clang" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=wasm32-unknown-unknown" "-I" "depend/secp256k1/" "-I" "depend/secp256k1/include" "-I" "depend/secp256k1/src" "-Wall" "-Wextra" "-DSECP256K1_BUILD=1" "-DUSE_NUM_NONE=1" "-DUSE_FIELD_INV_BUILTIN=1" "-DUSE_SCALAR_INV_BUILTIN=1" "-DENABLE_MODULE_ECDH=1" "-DUSE_EXTERNAL_DEFAULT_CALLBACKS=1" "-DECMULT_WINDOW_SIZE=15" "-DUSE_FIELD_10X26=1" "-DUSE_SCALAR_8X32=1" "-o" "/wasm-demo/target/wasm32-unknown-unknown/release/build/secp256k1-c0bc0977ed6381c0/out/depend/secp256k1/contrib/lax_der_parsing.o" "-c" "depend/secp256k1/contrib/lax_der_parsing.c" with args "clang" did not execute successfully (status code exit code: 1).

'

warning: build failed, waiting for other jobs to finish...
error: build failed
Error: Compiling your crate to WebAssembly failed

[elichai]

This errors look like errors with the OS files, not with this library.
Try reinstalling the toolchains, compiler(clang and llvm), stdlibs etc.(Xcode?)

And google those cdefs.h:784:2: error: Unsupported architecture, _types.h:34:2: error: architecture not supported, _types.h:55:9: error: unknown type name '__int64_t'; did you mean '__int128_t'? Errors and try what people are saying.

Don't know enough about MacOS to really help you

[TheBlueMatt]

Also probably seems like an issue that will break almost any CC-built projects building for WASM, not just us. Maybe either file on the appropriate wasm project or cc.

[howardwu]

Still no luck. For future readers, I tried installing macOS 10.14 headers (open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg), installing Xcode updates, and updating CMake and LLVM. Additionally, I downloaded a pre-built binary of Clang 8.0.0 (https://releases.llvm.org/download.html), set respective environment variables, and attempted packaging with Clang 8.

For those with immediate need, I was able to package in a Ubuntu 18.04 Vagrantbox by installing a pre-built binary of Clang 8.0.0 (https://releases.llvm.org/download.html) into /usr/local/clang_8.0.0 and adding Clang with export PATH=/usr/local/clang_8.0.0/bin:$PATH and export LD_LIBRARY_PATH=/usr/local/clang_8.0.0/lib:$LD_LIBRARY_PATH. In addition, I installed the usual dependencies: build-essential, Git, CMake, Rust/Cargo, Node, wasm-pack, etc.

[laanwj]

Also probably seems like an issue that will break almost any CC-built projects building for WASM, not just us. Maybe either file on the appropriate wasm project or cc.

Right, cc-rs defaults to clang for wasm targets. Likely on OSX this will pick the system clang with the system headers, which is not going to work for this. It might help to override the compiler (either by overriding the path, or using CC environment variable) to a toolchain for wasm.

[TheBlueMatt]

Note that on Linux I had to rustup target add wasm32-wasi and then cargo build --target wasm32-wasi worked, though wasm-pack didn't seem to want to.