Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot load unsigned dylibs on macOS with dart:ffi or native extensions #38314

Closed
mit-mit opened this issue Sep 11, 2019 · 25 comments
Closed

Cannot load unsigned dylibs on macOS with dart:ffi or native extensions #38314

mit-mit opened this issue Sep 11, 2019 · 25 comments

Comments

@mit-mit
Copy link
Member

@mit-mit mit-mit commented Sep 11, 2019

Now that the Dart SDK is signed (i.e. dart is signed), we don't seem to be able to load unsigned dylibs anymore:

Unhandled exception:
Invalid argument(s): Failed to load dynamic library (dlopen(osdialog.dylib, 1): no suitable image found.  Did find:
	osdialog.dylib: code signature in (osdialog.dylib) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.)
#0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:10:55)
#1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:17:12)
#2      main (file:///Users/mit/dev/playground/dart/ffi/osdialog/os.dart:11:36)
#3      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:305:19)
#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)
| ~/dev/playground/dart/ffi/osdialog @ mit-macbookpro3 (mit)
@aab29
Copy link

@aab29 aab29 commented Sep 22, 2019

We're running into this issue with the game engine we're developing. 👾 After upgrading to 2.5, we can no longer run the tests for our engine because they need to load dylibs. We've tried signing the dylibs, but we can't figure out a way to sign them such that dart can interface with them. ✍️ 🙅‍♂

@mit-mit
Copy link
Member Author

@mit-mit mit-mit commented Sep 23, 2019

@aab29 are you using ffi? Which operating system?

If on macOS, did you try signing the library?

@mit-mit mit-mit closed this Sep 23, 2019
@mit-mit mit-mit reopened this Sep 23, 2019
@mit-mit
Copy link
Member Author

@mit-mit mit-mit commented Sep 23, 2019

If macOS, can you try the following workaround:

  1. Make sure you have the codesign tool in your terminal
  2. Locate the main dart binary
  3. Run codesign --remove-signature <path to dart binary>
  4. Run tests again

You might want to take a backup copy of the dart command first, or re-install dart when done.

@aab29
Copy link

@aab29 aab29 commented Oct 11, 2019

Hi @mit-mit ! Thanks for your speedy, helpful response! I'm sorry my reply is so delayed. We've been busy over here. 🤓 ⌨️

No--we are not using ffi. The error message we were getting was similar to the one I found here, but I'm afraid I missed that detail before I commented. We're still using the old native extensions system to interface with our dylibs. 💾 Let me know if you'd like me to create a separate issue or anything.

We're currently running Dart 2.5.2, and we're hitting the same error messages we hit under Dart 2.5.0. 🎯 We are indeed running MacOS. For now we're running Mojave (10.14.6), but we're hoping to upgrade to Catalina soon. 🔜 🆙

The workaround you suggested (removing the signature on the main dart binary) works fine! Thank you! 🙌 For the moment we're happy with this, but obviously it's not ideal in the long run.

We don't have much experience dealing with code signing, so apologies if we're not doing something obvious. 🤔 From what we can tell, it seems like we should now be codesigning our own dynamic libraries, but if we sign them with our own credentials, it's not possible to interface with the dart binary (signed with a different team identifier). ✍️ I suppose we could build and sign our own dart binary, but that is still much more involved than we'd like for this use case. 💁‍♂

Please let me know if there's any other information we can provide that might be helpful! ℹ️

@keertip
Copy link
Contributor

@keertip keertip commented Nov 1, 2019

@mit-mit , folks are going to run into this with doing the ffi tutorial. Has there been any progress? Ran into this with -dev.8.2 installed using brew.

@dcharkes dcharkes changed the title Cannot load unsigned dylibs on macOS with dart:ffi Cannot load unsigned dylibs on macOS with dart:ffi or native extensions Nov 4, 2019
@dcharkes
Copy link
Contributor

@dcharkes dcharkes commented Nov 4, 2019

Conceptually speaking it makes sense that a signed Dart binary can only load signed (third party) shared libraries.

For now, unsigned binaries can load unsigned (and probably signed) third party libraries. However, future MacOS versions will likely disallow that.

I think the path forward would be to include in our tutorial how to sign your own custom made library on MacOS Catalina.

cc @mkustermann

@rmazumde
Copy link

@rmazumde rmazumde commented Nov 27, 2019

Now that the Dart SDK is signed (i.e. dart is signed), we don't seem to be able to load unsigned dylibs anymore:

Unhandled exception:
Invalid argument(s): Failed to load dynamic library (dlopen(osdialog.dylib, 1): no suitable image found.  Did find:
	osdialog.dylib: code signature in (osdialog.dylib) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.)
#0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:10:55)
#1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:17:12)
#2      main (file:///Users/mit/dev/playground/dart/ffi/osdialog/os.dart:11:36)
#3      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:305:19)
#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)
| ~/dev/playground/dart/ffi/osdialog @ mit-macbookpro3 (mit)

I am still getting same error while using flutter in macos and to call my c++ library
Building macOS application...
flutter: ══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
flutter: The following ArgumentError was thrown building MyHomePage(dirty, state: _MyHomePageState#86c2f):
flutter: Invalid argument(s): Failed to load dynamic library (dlopen(/usr/lib/structs.dylib, 1): image not
flutter: found)
flutter:
flutter: The relevant error-causing widget was:
flutter: MyHomePage
flutter: file:///Users/rmazumder/VisualCode-Workspace/flutter-desktop-embedding-master/example/lib/main.dart:39:13
flutter:
flutter: When the exception was thrown, this was the stack:

My Flutter and Dart version :

rmazumder$ flutter --version
Flutter 1.12.15-pre.12 • channel master • https://github.com/flutter/flutter.git
Framework • revision b2a36ffcd2 (22 hours ago) • 2019-11-26 11:45:51 -0500
Engine • revision e136d637a8
Tools • Dart 2.7.0

@mkustermann
Copy link
Member

@mkustermann mkustermann commented Nov 28, 2019

@rmazumde

Generally speaking there are two options: Use an unsigned Dart SDK during development or sign the shared libraries.

If you are using the standalone dart we encourage you to obtain the Dart SDK from Dart team's distribution channels (not using the dart-sdk used internally by flutter tooling).

We do have unsigned Dart SDKs available which you could use

% gsutil ls gs://dart-archive/channels/dev/raw/2.7.0-dev.1.0/sdk/     // <= Notice "raw"
gs://dart-archive/channels/dev/raw/2.7.0-dev.1.0/sdk/dartsdk-macos-ia32-release.zip
gs://dart-archive/channels/dev/raw/2.7.0-dev.1.0/sdk/dartsdk-macos-x64-release.zip
...

% gsutil ls gs://dart-archive/channels/dev/signed/2.7.0-dev.1.0/sdk/    // <= Notice "signed"
gs://dart-archive/channels/dev/signed/2.7.0-dev.1.0/sdk/dartsdk-macos-x64-release.zip

Those are downloadable via the http://storage.googleapis.com/dart-archive/... URL prefix.

@javaddict
Copy link

@javaddict javaddict commented Dec 31, 2019

Just find out something today when I am trying the ffi feature using this hello world example.
If I dart2native it, producing hello.exe, it runs with the same error:

Invalid argument(s): Failed to load dynamic library (dlopen(./hello_library/libhello.dylib, 1): no suitable image found.  Did find:
	/.../samples/ffi/hello_world/././hello_library/libhello.dylib: code signature in (/.../samples/ffi/hello_world/././hello_library/libhello.dylib) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

Here libhello.dylib is unsigned.
codesign -dv hello.exe shows that hello.exe is signed.
If I remove the signature of hello.exe, it should work, right? But no. It failed at codesign --remove-signature hello.exe. This is because hello.exe is not a well-signed binary. What dart2native does is simply concatenating the signed dartaotruntime with the snapshot of hello.dart and saving it to hello.exe.
So, backup dartaotruntime, remove the signature of it, rerun dart2native hello.dart, then it works again.

@filleduchaos
Copy link

@filleduchaos filleduchaos commented Jan 1, 2020

@mkustermann Signing the shared libraries just causes dlopen to complain that the team IDs don't match.

Library Validation: mapping process and mapped file (non-platform) have different Team IDs)

(and a similar error if you sign without a Team ID).

Given that FFI is now a stable feature of Dart, I think the fact that there are unsigned SDKs available (and why they're needed) should be as front-and-center as possible.

@gaaclarke
Copy link
Contributor

@gaaclarke gaaclarke commented Jan 15, 2020

@mkustermann

Generally speaking there are two options: Use an unsigned Dart SDK during development or sign the shared libraries.

Using a signed shared library isn't an option unless you provide the key that the Dart SDK was signed with. Using an adhoc signature causes:

Invalid argument(s): Failed to load dynamic library (dlopen(./c/build/libhello.dylib, 1): no suitable image found.  Did find:
	./c/build/libhello.dylib: code signature in (./c/build/libhello.dylib) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?))

Signing with your own cert causes the error @filleduchaos mentioned mapping process and mapped file (non-platform) have different Team IDs

It looks like there is something we have to do on the Dart side to validate the signatures: https://forums.developer.apple.com/thread/27799

@filleduchaos
Copy link

@filleduchaos filleduchaos commented Jan 15, 2020

@gaaclarke if you harden a macOS app's runtime then you can specify in its entitlements that it should be able to load unsigned dylibs at runtime (see the Apple docs. However I'm far from certain if using the hardened runtime is feasible for the Dart SDK.

In the meantime maybe the Dart homebrew tap could add an --unsigned option to install the unsigned SDK, which could then be documented at https://dart.dev/get-dart like the --devel option?

@dcharkes
Copy link
Contributor

@dcharkes dcharkes commented Jan 15, 2020

We discussed this a couple of days ago:

  1. For development, we could create an unsigned SDK. That would work for local development, but not when distributing your application.

  2. For deployment, we could give dart2native a key option to let devs sign the final executable with their own key. Both the final executable and the shared libraries would be signed with the same developer id. (Note that this would force deployment to always be AOT compilation, and never JIT. JIT would require devs sign the dart executable themselves.)

Conceptually any application that has a plugin architecture with shared objects would run into this.

@gaaclarke if you harden a macOS app's runtime then you can specify in its entitlements that it should be able to load unsigned dylibs at runtime (see the Apple docs. However I'm far from certain if using the hardened runtime is feasible for the Dart SDK.

That seems like a solution for plugin architectures. We could investigate if a jitting VM would qualify for that. @mit-mit.

@filleduchaos
Copy link

@filleduchaos filleduchaos commented Jan 15, 2020

@dcharkes allowing the execution of JIT-compiled code is another specifiable entitlement - I don't know that much about Dart SDK internals, but I think if it's signed with the hardened runtime enabled then it would need com.apple.security.cs.allow-jit for JITting, com.apple.security.cs.disable-library-validation to allow loading arbitrary (un)signed dylibs and probably com.apple.security.cs.debugger to facilitate debugging?

I have a bit of free time this weekend so I'll try to sign an unsigned SDK with --options runtime and see if the executables work without crashing with those entitlements enabled

@filleduchaos
Copy link

@filleduchaos filleduchaos commented Jan 23, 2020

I ran a whole bunch of experiments and I have found the issue, I think.

The distributed Dart SDKs are already signed with the hardened runtime, which is required for notarization (see: flutter/flutter#36714 (comment)) but don't have all the entitlements necessary to let them operate as freely as the unsigned SDKs. To elaborate, this is the signing information for a signed dart executable (note the runtime flag):

$ codesign -dv --verbose=1 /tmp/dart-sdk/bin/dart

Executable=/tmp/dart-sdk/bin/dart
Identifier=dart
Format=Mach-O thin (x86_64)
CodeDirectory v=20500 size=250480 flags=0x10000(runtime) hashes=7819+5 location=embedded
Signature size=9043
Timestamp=20 Nov 2019 at 4:07:54 PM
Info.plist=not bound
TeamIdentifier=EQHXZ8M8AV
Runtime Version=10.13.0
Sealed Resources=none
Internal requirements count=1 size=164

And these are the entitlements it was signed with:

$ codesign -d --entitlements :- /tmp/dart-sdk/bin/dart

Executable=/tmp/dart-sdk/bin/dart
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>com.apple.security.cs.allow-jit</key>
        <true/>
        <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
        <true/>
    </dict>
</plist>

As previously discussed, since dart uses the hardened runtime it needs to be signed with com.apple.security.cs.disable-library-validation to allow loading unvalidated dylibs; this plist explicitly excludes that entitlement, so dlopen rejects everything that's not a platform lib or signed with the same Team ID. I don't know if this entitlements plist is source-controlled anywhere though - it doesn't seem to be in this repository as far as I can tell.

cc @dcharkes @mit-mit

@mit-mit
Copy link
Member Author

@mit-mit mit-mit commented Jan 23, 2020

Excellent research! cc @athomas can you check with out signing contacts if that is a possible change we can make?

@athomas
Copy link
Member

@athomas athomas commented Jan 24, 2020

We're in control of the Entitlements.plist used to sign the Dart SDK executables (assuming potential security implications have been considered). Since we're using internal signing infrastructure, this file is not in the open source repo so I bumped an earlier internal email thread on this topic with the necessary information.

@dcharkes
Copy link
Contributor

@dcharkes dcharkes commented Feb 11, 2020

After changing the Entitlements.plist:

dart JIT:

dacoharkes-macbookpro:bin dacoharkes$ pwd
/Users/dacoharkes/Downloads/dart-sdk-resigned/bin
dacoharkes-macbookpro:bin dacoharkes$ ./dart ~/dart-sdk/sdk/samples/ffi/samples_test.dart 
0
1
...

dart2aot snapshot:

Generated: /Users/dacoharkes/Downloads/dart-sdk-resigned/bin/sample
dacoharkes-macbookpro:bin dacoharkes$ ./dart2native -k aot ~/dart-sdk/sdk/samples/ffi/samples_test.dart -o sample.aot
dacoharkes-macbookpro:bin dacoharkes$ ./dartaotruntime sample.aot 
0
1
...

dart2aot exe

dacoharkes-macbookpro:bin dacoharkes$ ./dart2native -k exe ~/dart-sdk/sdk/samples/ffi/samples_test.dart -o sample
Generated: /Users/dacoharkes/Downloads/dart-sdk-resigned/bin/sample.aot
dacoharkes-macbookpro:bin dacoharkes$ ./sample
0
...

Note that the shared libraries need to be in the same folder (or subfolder):

Invalid argument(s): Failed to load dynamic library (dlopen(libffi_test_dynamic_library.dylib, 1): no suitable image found.  Did find:
	file system relative paths not allowed in hardened programs)

@athomas
Copy link
Member

@athomas athomas commented Feb 11, 2020

2.8.0-dev.9.0 (probably going out later today) will be signed with the new entitlements.

@filleduchaos
Copy link

@filleduchaos filleduchaos commented Feb 12, 2020

@dcharkes that seems like a reasonable enough compromise for development, though it probably should work with an absolute path? I'll check that out when I pull the new version.

@dcharkes
Copy link
Contributor

@dcharkes dcharkes commented Feb 12, 2020

though it probably should work with an absolute path?

It doesn't matter whether you're using an absolute or relative path in Dart afaik. MacOS doesn't allow you to open a shared library from a folder that's not the same or a sub folder of the dart executable (or the dartprecompiledruntime executable).

@dcharkes
Copy link
Contributor

@dcharkes dcharkes commented Feb 28, 2020

This has landed a couple of weeks ago, and all new dev and stable released will be signed this way.

@jcush
Copy link

@jcush jcush commented Dec 27, 2020

Hi @mit-mit,

You asked the OP above if they use FFI - I've been having the same loadError when trying to run pod commands and it errors on FFI.

Obviously FFI is not your library, so I don't expect you to fully support it, but the fact you mentioned it makes me wonder if you know of an issue between FFI and mac builds? It seems like nobody has the answer!

Thanks

@johnpryan
Copy link
Contributor

@johnpryan johnpryan commented Jan 6, 2021

@jcush Are you are running a pod install on an Apple Silicon processor? Are you asking for insight into CocoaPods/CocoaPods#9907? At first glance, these look like they might be different problems, even though they show similar errors.

@jcush
Copy link

@jcush jcush commented Jan 7, 2021

Hi @johnpryan,

Thanks for your response - I'm not on a Silicon processor, no, I'm running a 2019 16" MBP. I've actually checked out CocoaPods/CocoaPods#9907 already and that's about Silicon.

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests