macOS library validation fails for FFI

[johnpryan]

It looks like #38288 has started happening again.

1. Clone https://github.com/dart-lang/samples
2. open the ffi/hello_world sample
3. run cd c && make dylib
4. run cd .. && dart hello.dart
5. observe "Failed to load dynamic library" exception

dart hello.dart 
Unhandled exception:
Invalid argument(s): Failed to load dynamic library (dlopen(hello_world.dylib, 1): no suitable image found.  Did find:
	hello_world.dylib: code signature in (hello_world.dylib) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.)
#0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:10:55)
#1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:17:12)
#2      main (file:///Users/ryjohn/code/github/dart-lang/samples/ffi/hello_world/hello.dart:17:36)
#3      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:305:19)
#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)

cc: @mit-mit @mjohnsullivan

[johnpryan]

I've tried using Apple's codesign tool to ad-hoc sign the library, but it's still not working:

codesign -f -s - hello_world.dylib

dart hello.dart 
Unhandled exception:
Invalid argument(s): Failed to load dynamic library (dlopen(hello_world.dylib, 1): no suitable image found.  Did find:
	hello_world.dylib: code signature in (hello_world.dylib) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?))
#0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:10:55)
#1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:17:12)
#2      main (file:///Users/ryjohn/code/github/dart-lang/samples/ffi/hello_world/hello.dart:17:36)
#3      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:305:19)
#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)

[mit-mit]

cc @dcharkes I think this isn't #38288 , but rather #38314 ?

[dcharkes]

Yeah, this is a duplicate of #38314, please add more details there.

[chris-schra]

I have a slightly different error:

Failed to load dynamic library (dlopen(./hello_library/libhello.dylib, 1): no suitable image found. Did find:
file system relative paths not allowed in hardened programs)

Update:
Fixed with executing
codesign --remove-signature /usr/local/bin/dart

[gerardsimons]

I get this on Mac M1, is this related somehow or something new altogether?

Invalid argument(s): Failed to load dynamic library (dlopen(./hello_library/libhello.dylib, 1): no suitable image found.  Did find:
	./hello_library/libhello.dylib: mach-o, but wrong architecture

[dcharkes]

I get this on Mac M1, is this related somehow or something new altogether?

Mac M1 is an arm64 chip, while all previous MacOS computers were x64. So, the dylib has to be compiled for arm64. Where did you get the dylib from? Did you compile it yourself?

Please open a new issue if you have further problems, is this is a separate issue from validation.

[gerardsimons]

Thanks @dcharkes

I actually followed the OP's ffi sample instructions to get a simple reproducible error

BTW it generates arm64 as you can see below... Let me know if I should raise this as a new issue

$> file libhello.1.0.0.dylib 
libhello.1.0.0.dylib: Mach-O 64-bit dynamically linked shared library arm64

[dcharkes]

@gerardsimons you might be running in x64 emulation mode, in which case you'd actually want to have a dylib compiled for x64 instead. https://en.wikipedia.org/wiki/Rosetta_(software)

[gerardsimons]

Actually it's true that I was running in emulation mode as I thought that was the issue, but I just turned it off and that doesn't work either (see screenshot)

However: running my terminal in emulation mode and recompiling makes an x64 library and then the hello.dart works (see next screenshot), so that's basically the old way of doing it on x64 Mac

[gerardsimons]

Ah actually once you have the library built in x64 you can run dart either way (x64 or arm64). I am guessing dart just forces Rosetta anyway

[dcharkes]

We don't have an arm64 built dart yet as far as I know. Once we have it, I'll verify that FFI works on it (#44772).

[gerardsimons]

Cool beans, happy to test something out if needed ... keep up the good job!

[lesnitsky]

for future readers on apple silicon:

• you need to install x86_64 version of homebrew https://stackoverflow.com/a/65398385/3891825 and use it to install llvm
• add llvm to path export PATH="/usr/local/homebrew/opt/llvm/bin:$PATH" (/usr/local/homebrew location means x86_65, /opt/homebrew/opt – arm)

I was able to build leveldb as a shared lib and use it via dart:ffi with setup above (leveldb was built in arch -x86_64 mode)

[benfgit]

Addition to @lesnitsky's comment, here is the complete steps on my workaround to build hello_world example in Apple Silicon:

Install x86 brew

arch -x86_64 zsh
cd /usr/local && mkdir homebrew
curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip 1 -C homebrew

Create brewARM and brew86 aliases in shell

Create corresponding aliases with this two commands. These can also be added to ~/.zshrc for permanent use.

alias brew86="arch -x86_64 /usr/local/homebrew/bin/brew"
alias brewARM="/opt/homebrew/bin/brew"

From now on, use brew86 to install x86 packages.

Install x86 cmake using brew86

brewARM uninstall cmake
brew86 install cmake

Add the cmake executable to the path, in my case it's /usr/local/homebrew/Cellar/cmake/3.21.0/bin.

Now you can use x86 architecture cmake to compile hello library and run it as described here.