System.Private.CoreLib.dll cannot be opened because the developer cannot be verified

[k15tfu]

Hi!

When I unpack an official dotnet-sdk-5.0.100-preview.5.20279.10-osx-x64 archive I cannot build a simple app without tons of *.dll cannot be opened because the developer cannot be verified warnings for PE files on macOS 10.15. I found some code that uses pread in case of PROT_EXEC when IsRunningOnMojaveHardenedRuntime which looks like a workaround for the same problem (dotnet/coreclr#26193), but it's unclear for me how it's supposed to work with .dll(s) now.

Can you shed some light on this? cc @janvorli @jkotas

Deleting com.apple.quarantine xattr helps to get rid of this.

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[janvorli]

I found some code that uses pread in case of PROT_EXEC when IsRunningOnMojaveHardenedRuntime which looks like a workaround for the same problem

That is not a workaround for a similar problem. That code is to enable us to map pages from managed R2R compiled assemblies as executable. On Mojave hardened runtime, it is not allowed to create executable mappings directly from a file. So we have to create anonymous mapping that's executable and read the executable section data into it using pread.

[k15tfu]

@janvorli My bad, I meant addresses the same problem. In other words, when I do mmap PROT_EXEC for a file with com.apple.quarantine xattr I see similar warning.
But how it should work with .dll files like System.Private.CoreLib.dll? Not sure if they are mapped as executables, but still it should work without warnings after your change. Any thoughts why we get this?

[janvorli]

@k15tfu can you please share steps that you do to repro the problem? I have just downloaded that tarball to my Mac Mini with macOS 10.15.5 and then created a hello world app using

/path/to/the/new/dotnet new console
/path/to/the/new/dotnet build
/path/to/the/new/dotnet run

And it worked fine.

[k15tfu]

@janvorli I found something: for some reason this mmap (https://github.com/dotnet/coreclr/blob/018cfd06dceb19b6eb1e9217a500fb1071946fcd/src/pal/src/misc/utils.cpp#L343) doesn't fail, so IsRunningOnMojaveHardenedRuntime returns 0 on my macOS 10.15.5, and that's why I see these warnings (if interested, they look like in microsoft/azure-pipelines-agent#2457). Mono project uses MAP_JIT depending on darwin version (mono/mono#9210), and therefore it also explains why I faced with this problem after switching to dotnet in my app. Bingo!

Steps to reproduce:

1. Download .NET Core Runtime 3.1.5 binaries for macOS from Safari
2. Check that dotnet-runtime-3.1.5-osx-x64.tar.gz has com.apple.quarantine

$ xattr dotnet-runtime-3.1.5-osx-x64.tar.gz
com.apple.lastuseddate#PS
com.apple.macl
com.apple.metadata:kMDItemDownloadedDate
com.apple.metadata:kMDItemWhereFroms
com.apple.quarantine

3. Unpack & try to run simple hello world app. Now you should start getting these warnings but for dotnet and other *.dylib libraries. Not sure if this is a problem because they can be signed, so I just clears this xattr for Mach-O files

$ cd your-path
$ xattr -d com.apple.quarantine dotnet `find . -name 'lib*.dylib'`

4. Finally, do ./dotnet exec ../myApp/bin/Debug/netcoreapp3.1/myApp.dll. Under the hood, it will open your-path/shared/Microsoft.NETCore.App/3.1.5/System.Private.CoreLib.dll file, and will try to mmap it with prot=PROT_READ|PROT_EXE flags=MAP_PRIVATE|MAP_FIXED (https://github.com/dotnet/coreclr/blob/018cfd06dceb19b6eb1e9217a500fb1071946fcd/src/pal/src/map/map.cpp#L2181). Now I see the warning about System.Private.CoreLib.dll, and if I cancel it, mmap will return MAP_FAILED.

[janvorli]

Ah, so downloading it via Safari causes the trouble. Download via wget from the command line that I was using works fine, it doesn't add the com.apple.quarantine.
So after all, it is not related to the hardened runtime support, but rather to the fact that download via Safari marks all downloads with this com.apple.quarantine attr just to stay on the safe side. It is fine until the code tries to create an executable mapping from one of those files, which is suspect from the OS point of view.

[k15tfu]

@janvorli Yes, exactly. BTW, why do you use com.apple.security.cs.allow-unsigned-executable-memory? And when this entitlement is used, looks like IsRunningOnMojaveHardenedRuntime will always be 0, so probably it should be named more carefully.

I'm asking because in other case, if IsRunningOnMojaveHardenedRuntime is 1, the problem shouldn't exist.

[janvorli]

@k15tfu I have just noticed you have asked a question long time ago here. I am actually not sure if we really need com.apple.security.cs.allow-unsigned-executable-memory when we are also adding com.apple.security.cs.allow-jit. I've create a new issue #45677 for looking into that and I am closing this one.