Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hardened runtime causes app to crash on open. (Signed and notarized) #4040

Open
UdaraJay opened this issue Jul 10, 2019 · 126 comments

Comments

@UdaraJay
Copy link

@UdaraJay UdaraJay commented Jul 10, 2019

  • Versions:
    electron: 5.0.6
    electron-builder: 21.0.11
    electron-notarize: 0.1.1
    electron-webpack: 2.7.4
    Working on: MacOS Catalina 10.15 Beta 3 (19A501i)
    Xcode: Xcode 11 beta 3

  • What I'm trying to do:
    Sign and notarize an electron, web-packed, react desktop application for distribution outside the mac store.

  • Problem and exact case of error:

  1. Build the app unsigned, unnotarized, no hardened runtime: runs
  2. Build the app signed, no hardened runtime: runs
  3. Build the app signed, hardened runtime: (build and sign successful) error below when opening app
  4. Build the app signed, hardened runtime, notarize: (build, sign and notatrize successful) error below when opening app
  • In both the error cases I've run:
    1/ Verify code signing
test.app: valid on disk
test.app: satisfies its Designated Requirement

2/ Verify code notarization

test.app: accepted
source=Notarized Developer ID
origin=Developer ID Application: XXX, Inc. (XXXXXXXXXX)
  • Entitlements for mac
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
  </dict>

  • Target: darwin

  • Error dump

Process:               Test [7467]
Path:                  /Users/USER/Documents/*/Test.app/Contents/MacOS/Test
Identifier:            ai.XXXX.desktop
Version:               0.0.4 (0.0.4)
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           Test [7467]
User ID:               501

Date/Time:             2019-07-10 16:44:52.073 -0400
OS Version:            Mac OS X 10.15 (19A501i)
Report Version:        12
Bridge OS Version:     4.0 (17P50496d)
Anonymous UUID:        259AA2B3-5AA8-D576-9F9B-EF113BC33FAD

Sleep/Wake UUID:       BFEDF03B-3097-40F1-9CC0-19B87A69E03E

Time Awake Since Boot: 27000 seconds
Time Since Wake:       11000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (Code Signature Invalid)
Exception Codes:       0x0000000000000032, 0x00001a9c2b202040
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    Namespace CODESIGNING, Code 0x2

...

Logical CPU:     6
Error Code:      0x00000015 (invalid protections for user instruction write)
Trap Number:     14
@itsthisjustin

This comment has been minimized.

Copy link

@itsthisjustin itsthisjustin commented Jul 11, 2019

Confirmed same issue happening to us. I'm not sure what to even do to work around this.

@itsthisjustin

This comment has been minimized.

Copy link

@itsthisjustin itsthisjustin commented Jul 11, 2019

is this the fix? #3926

@itsthisjustin

This comment has been minimized.

Copy link

@itsthisjustin itsthisjustin commented Jul 11, 2019

Further info:

You need at least the following entitlements:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>
com.apple.security.cs.allow-jit</key>
    <true/>
    <key>
com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>
com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
  </dict>
</plist>

Further info from https://medium.com/@TwitterArchiveEraser/notarize-electron-apps-7a5f988406db

Make sure to try out starting the app after this process, if some entitlements are missing, the app might be broken after notarization.

@UdaraJay

This comment has been minimized.

Copy link
Author

@UdaraJay UdaraJay commented Jul 11, 2019

@itsthisjustin tried all sorts of entitlements, including the ones above, not the fix. Also tried gatekeeper access to false and afterSign hook to notarize before making the DMG; still not the fix. The only variable that fixes this is turning hardened runtime off; but that means you can't notarize.

  1. I've tried this entire process on a non-webpacked electron app and that seems to work fine; feels like it could be connected.

  2. I've also tried setting the sandboxed entitlement; while the app opens in this scenario and the Code Signature Invalid error goes away, but it leads to a different crash error. Expected since the app I'm working on is not ready to be sandboxed yet.

@itsthisjustin

This comment has been minimized.

Copy link

@itsthisjustin itsthisjustin commented Jul 11, 2019

@UdaraJay entitlements got rid of the sign error but yeah my UI crashes on start now

@UdaraJay

This comment has been minimized.

Copy link
Author

@UdaraJay UdaraJay commented Jul 11, 2019

Hey, @itsthisjustin - wondering what MacOS you use? and Xcode version? Curious if it's an xcode issue.

@itsthisjustin

This comment has been minimized.

Copy link

@itsthisjustin itsthisjustin commented Jul 11, 2019

@UdaraJay I installed the latest Xcode beta today actually in order to get notarization working. So 11 beta 3. I'm on OSX 10.14.5. I have this running in Travis CI as well.

Update: Tested on xcode 10.2 and same issue

@matteotomasoni

This comment has been minimized.

Copy link

@matteotomasoni matteotomasoni commented Jul 11, 2019

Yes, the issue I'm having in #3989 seems exactly the same, except I'm still using electron 4.2.6 and I'm building on an older Mac (OSX 10.13.6, according to Apple's own documentation this is the minimum version able to notarize apps - and I verified that using Xcode to build a simple native app it works perfectly well). Also, my app doesn't use webpack but only simple old-school html/css/js.

Both signature and notarization seems ok on my app, and even the error report seems the same:

Process:               TestApp [4615]
Path:                  /Applications/TestApp.app/Contents/MacOS/TestApp
Identifier:            com.mycompany.test-app
Version:               1.4.4 (1.4.4)
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           TestApp [4615]
User ID:               501

Date/Time:             2019-07-10 18:02:04.255 +0200
OS Version:            Mac OS X 10.14.5 (18F132)
Report Version:        12
Anonymous UUID:        516B346A-7835-6066-F0D6-F79409DF2C59

[...]

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (Code Signature Invalid)
Exception Codes:       0x0000000000000032, 0x00002f36dcf86660
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    Namespace CODESIGNING, Code 0x2

kernel messages:

VM Regions Near 0x2f36dcf86660:
    Memory Tag 255         00002f36dcf83000-00002f36dcf84000 [    4K] ---/rwx SM=NUL  
--> Memory Tag 255         00002f36dcf84000-00002f36dcfff000 [  492K] r-x/rwx SM=COW  
    Memory Tag 255         00002f36dcfff000-00002f36e4f69000 [127.4M] ---/rwx SM=NUL  

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   ???                           	0x00002f36dcf86660 0 + 51912682006112
1   com.github.Electron.framework 	0x00000001016b9a41 0x1002f3000 + 20736577
2   com.github.Electron.framework 	0x00000001016b9778 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) + 152
3   com.github.Electron.framework 	0x0000000101a7d4fc v8::Script::Run(v8::Local<v8::Context>) + 492
4   com.github.Electron.framework 	0x00000001050c63af 0x1002f3000 + 81605551
5   com.github.Electron.framework 	0x00000001050c5f0f node::LoadEnvironment(node::Environment*) + 191
6   com.github.Electron.framework 	0x0000000102067968 0x1002f3000 + 30886248
7   com.github.Electron.framework 	0x0000000101fed395 0x1002f3000 + 30385045
8   com.github.Electron.framework 	0x0000000100d0042a 0x1002f3000 + 10540074
9   com.github.Electron.framework 	0x0000000100d05afa 0x1002f3000 + 10562298
10  com.github.Electron.framework 	0x0000000100cffb38 0x1002f3000 + 10537784
11  com.github.Electron.framework 	0x0000000101f582f8 0x1002f3000 + 29774584
12  com.github.Electron.framework 	0x000000010334dbbd 0x1002f3000 + 50703293
13  com.github.Electron.framework 	0x0000000101f573a4 0x1002f3000 + 29770660
14  com.github.Electron.framework 	0x00000001002f5d54 AtomMain + 84
15  com.mycompany.test-app           	0x00000001002e9f10 0x1002e9000 + 3856
16  libdyld.dylib                 	0x00007fff603583d5 start + 1

[...]

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x00002f36dcf86660  rcx: 0x0000000000000000  rdx: 0x00001a9d74082201
  rdi: 0x00001a9d052026f1  rsi: 0x00001a9da082e1f9  rbp: 0x00007ffeef915f90  rsp: 0x00007ffeef915ee8
   r8: 0x0000000000000000   r9: 0x0000000000000000  r10: 0x0000000000000000  r11: 0x0000000106b6bd68
  r12: 0x00007fa71d03b878  r13: 0x0000000106b65000  r14: 0x00001a9da082e1f9  r15: 0x00001a9d74082201
  rip: 0x00002f36dcf86660  rfl: 0x0000000000010246  cr2: 0x00002f36dcf86660
  
Logical CPU:     0
Error Code:      0x00000015
Trap Number:     14

[...]

I tried adding all these entitlements, but with no success:

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
	<dict>
		<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
		<true/>
		<key>com.apple.security.cs.allow-jit</key>
		<true/>
		<key>com.apple.security.cs.allow-dyld-environment-variables</key>
		<true/>
		<key>com.apple.security.cs.disable-library-validation</key>
		<true/>
		<key>com.apple.security.cs.disable-executable-page-protection</key>
		<true/>
	</dict>
</plist>

I also verified that the correct entitlements are embedded in the application.

Last week I opened a DTS at Apple but even with their help we've not found a solution yet.

@itsthisjustin

This comment has been minimized.

Copy link

@itsthisjustin itsthisjustin commented Jul 11, 2019

@UdaraJay Alright, I've finally got it working. The trick was two fold.

First:

Set both entitlements and entitlementsInherit in your mac build settings. Here's mine:

"mac": {
      "hardenedRuntime": true,
      "gatekeeperAssess": false,
      "artifactName": "${productName}-${version}-${arch}.${ext}",
      "entitlements": "mac_config/entitlements.mac.plist",
      "entitlementsInherit": "mac_config/entitlements.mac.plist",
      "target": [
        "dmg",
        "zip"
      ],

Then, in my entitlements file, I stripped it down to only what I need.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.security.files.user-selected.read-write</key>
	<true/>
	<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
	<true/>
	<key>com.apple.security.device.audio-input</key>
	<true/>
	<key>com.apple.security.files.user-selected.read-only</key>
	<true/>
</dict>
</plist>

com.apple.security.cs.allow-unsigned-executable-memory is really the key one here. Others mentioned adding another one, but I was back to the signing crash when I did that.

@matteotomasoni

This comment has been minimized.

Copy link

@matteotomasoni matteotomasoni commented Jul 11, 2019

I've tried your same entitlements, but unfortunately my application still crashes with the same symptoms

@itsthisjustin

This comment has been minimized.

Copy link

@itsthisjustin itsthisjustin commented Jul 11, 2019

@matteotomasoni is there any other entitlements you might need? Ive basically found this:

If you don't do the entitlementsInherit thing, your app won't load. If you mess up the entitlements with too many or too few, it'll crash with the signing error.

For instance if I add the sandbox entitlement I crash with the sign error

@matteotomasoni

This comment has been minimized.

Copy link

@matteotomasoni matteotomasoni commented Jul 12, 2019

@itsthisjustin is there a reason for the com.apple.security.device.audio-input entitlement? Does your app really use the audio input?
My app doesn't do anything fancy at all, but I use a couple of libraries that are like a black box to me.
If the problem is on entitlements, then is there a way (or better, a tool) to know exactly what entitlements an application needs?

@itsthisjustin

This comment has been minimized.

Copy link

@itsthisjustin itsthisjustin commented Jul 12, 2019

@matteotomasoni ha yeah we are building https://yac.chat. So definitely needs audio.

I had the same fear as you and I think ultimately just got lucky. That’s also why the inherit property is so important. I’m not sure who at Apple thinks it’s normal to just crash an app if the entitlements are wrong though.

@UdaraJay

This comment has been minimized.

Copy link
Author

@UdaraJay UdaraJay commented Jul 12, 2019

@itsthisjustin glad you figured it out! I've tried those entitlements and other combination (for both the entitlements and the inheritedEntitlements) with no luck. But I didn't realize having too many entitlements could cause signing errors too, that's important to note!

And yea! Truly, I wish there was a way to access more specific errors on code-signing, or at least fail more elegantly.

@staifan

This comment has been minimized.

Copy link

@staifan staifan commented Jul 17, 2019

Hello,
Do you find a solution?
I encounter exactly the same problem.
I've been trying for more than a week for just about every possible configuration ...
I even tried to do some tests with the basic application of electron (https://electronjs.org/docs/tutorial/first-app#electron-development-in-a-nutshell) and unfortunately, I get always the same result:
All this is correct, I get each time the apple mail that says: "Your Mac software has been notarized. You can now export this software and distribute it directly to users".
The application installs correctly, and at launch I get the message inexorably:

Date/Time:             2019-07-17 10:54:31.390 +0200
OS Version:            Mac OS X 10.14.5 (18F203)
Report Version:        12
Anonymous UUID:        1984F4B3-87AC-D6FD-4C5D-6A3242112EE9


Time Awake Since Boot: 1600 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (Code Signature Invalid)
Exception Codes:       0x0000000000000032, 0x00002c32c2302040
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    Namespace CODESIGNING, Code 0x2

For now I have no idea ... So if someone at a track, I'm interested
Thank you

@starkos

This comment has been minimized.

Copy link

@starkos starkos commented Jul 17, 2019

This suggestion from @itsthisjustin got me past the startup crash. I'm still having other issues (unable to access web content for starters) but at least things are running.

@HeberLemus

This comment has been minimized.

Copy link

@HeberLemus HeberLemus commented Jul 20, 2019

i too am having a similar problem. cant get past startup crash.
i tried using the same entitlement settings with no dice
i also tried other entitlement lines and combinations of them with no luck either.
the error log starts as /Volume/VOLUME/ because i was running it off a flash drive, both the .app file itself and the .dmg file
feeling terrible that im so close to getting the app to run yet being denied so hard

Process: Appname [874]
Path: /Volumes/VOLUME/*/Appname.app/Contents/MacOS/Appname
Identifier: com.appname.desktopapp
Version: 1.0.0 (1.0.0)
Code Type: X86-64 (Native)
Parent Process: ??? [1]
Responsible: Appname [874]
User ID: 501

Date/Time: 2019-07-19 15:27:26.821 -0700
OS Version: Mac OS X 10.14.5 (18F2058)
Report Version: 12
Anonymous UUID: A4DA30A6-09EA-9677-95CD-EA316769DD4D

Sleep/Wake UUID: CC4B1217-0165-46A8-846E-BFA4D38C58E6

Time Awake Since Boot: 16000 seconds

System Integrity Protection: enabled

Crashed Thread: 0 Dispatch queue: com.apple.main-thread

Exception Type: EXC_BAD_ACCESS (Code Signature Invalid)
Exception Codes: 0x0000000000000032, 0x0000108a00082040
Exception Note: EXC_CORPSE_NOTIFY

Termination Reason: Namespace CODESIGNING, Code 0x2

kernel messages:

VM Regions Near 0x108a00082040:
Memory Tag 255 0000108a00081000-0000108a00082000 [ 4K] ---/rwx SM=NUL
--> Memory Tag 255 0000108a00082000-0000108a000ff000 [ 500K] r-x/rwx SM=COW
Memory Tag 255 0000108a000ff000-0000108a07fbe000 [126.7M] ---/rwx SM=NUL

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 ??? 0x0000108a00082040 0 + 18184892063808
1 com.github.Electron.framework 0x0000000103ffdad8 0x10273b000 + 25963224
2 com.github.Electron.framework 0x0000000103ffc6ce 0x10273b000 + 25958094

@UdaraJay

This comment has been minimized.

Copy link
Author

@UdaraJay UdaraJay commented Jul 20, 2019

@starkos @HeberLemus if you don't need any specific entitlements, try removing the entitlements files and let electon-builder set the default one. That's the only way I got signing and notarizing to work on my end. (still not perfect; but it at-least gets the application to run)

@HeberLemus

This comment has been minimized.

Copy link

@HeberLemus HeberLemus commented Jul 21, 2019

@UdaraJay tried that just now, removed the properties "entitlements" and "entitlementsInherit" from the package json under the mac key, still the exact same error

also, running "codesign -d --ent :- "Appname.app/Contents/MacOS/Appname" doesnt give back anything if its supposed to set default entitlements
what are the defaults?

update: nevermind, the defaults are being set.

in the end, my build is loading with the default entitlements. the other issue i had was that i had set mac: {type: "development"}

@UdaraJay

This comment has been minimized.

Copy link
Author

@UdaraJay UdaraJay commented Jul 22, 2019

Perfect! @HeberLemus did it start working?

@HeberLemus

This comment has been minimized.

Copy link

@HeberLemus HeberLemus commented Jul 22, 2019

@UdaraJay yes, no build issues, no startup issues and no later issues thus far. guess my problem really was that one property just like for this

update: actually there is one problem. while the file works fine when downloaded to a flashdrive and then mounted onto the mac, the .pkg isnt getting signed so when attempting to open said file it comes back as unidentified developer when attempting to download from the net

going to try and see if the AfterAllArtifactBuild hook may let me sign and notarize the .pkg beofre its done

update two: so it would seem that signing and getting it notarized works, but stapling does not
Error:

electron-notarize notarization was successful +0ms
electron-notarize attempting to staple app: /pathToFile/Appname-1.0.0.pkg +0ms
⨯ Failed to staple your application with code: 66

Processing: /pathToFile/Appname-1.0.0.pkg
Properties are {
NSURLIsDirectoryKey = 0;
NSURLIsPackageKey = 0;
NSURLIsSymbolicLinkKey = 0;
NSURLLocalizedTypeDescriptionKey = "Installer package";
NSURLTypeIdentifierKey = "com.apple.installer-package-archive";
"_NSURLIsApplicationKey" = 0;
}
Could not find an appropriate "code signature" in the Appname-1.0.0.pkg installer package.
Cannot download ticket. CDHash must be set.

my method was
const cp = require('child_process');
const path = require('path');
const {notarize} = require('electron-notarize');

exports.default = async function (context) {
if (process.platform !== 'darwin') return; //dont bother if a windows build
const { artifactPaths, outDir } = context;
for (let i = 0; i< artifactPaths.length; i++) {
const APP_PATH = artifactPaths[i];
const ENTITLEMENTS_PATH = path.join(outDir, '../build/mac/entitlements.mac.plist');
const cmd =codesign -s "${process.env.MAC_DEVELOPER_ID}" -f --entitlements "${ENTITLEMENTS_PATH}" "${APP_PATH}";
await cp.execSync(cmd, {stdio: "inherit"});
await notarize({
appBundleId:com.appname.desktopapp,
appPath: APP_PATH,
appleId: process.env.APPLEID,
appleIdPassword: process.env.APPLEIDPASS,
});
await cp.execSync(spctl -a -t exec -vv "${APP_PATH}", {stdio: "inherit"});
}
console.log('Build Complete');
};

if someone can suggest a cleaner way to do this i greatly appreciate it.

update 3: despite Apples documentation that even if a file doesnt have its notarization ticket stapled to it, that it should be able to get the notarization clearance from the net as stated here

After notarization completes successfully, the next time any user attempts to run your app on macOS 10.14 or later, Gatekeeper finds the ticket online...

the .pkg still refuses to open. any ideas?

@UdaraJay

This comment has been minimized.

Copy link
Author

@UdaraJay UdaraJay commented Jul 23, 2019

@HeberLemus so I had this problem too. If signing and notarizing works anything like dmgs, then you can leave the pkg unsigned and unnotarized and the OS should see the signed and notarized application inside. I couldn't get my pkg to notarize; but I'm using a DMG for distribution currently.

@HeberLemus

This comment has been minimized.

Copy link

@HeberLemus HeberLemus commented Jul 24, 2019

@UdaraJay i also swapped back to dmg, as i asked a coworker to try and use the .pkg and his system refused to open saying unknown developer, much in the same way as if its not notarized. if i downloaded the pkg to a flash and then open the pkg from the flash on the mac after transferring it to a mac, then it would open. but if downloaded from the internet it refused

ive run into a different problem now. anytime i attempt to set entitlements other than let the defaults be set the app refuses to run. but i need the camera entitlement for my app to access webcam and audio entitlement to access microphone. any recommendations?

@UdaraJay

This comment has been minimized.

Copy link
Author

@UdaraJay UdaraJay commented Jul 27, 2019

@HeberLemus this may be a real issue, I can't seem to get signing to work using any custom entitlements either. One thing you could try would be to look at what the exact default entitlements are and then try using those + the camera entitlements?

@patrickmichalina

This comment has been minimized.

Copy link

@patrickmichalina patrickmichalina commented Jul 29, 2019

WOW! Thanks for this thread. Totally saved me. Couldn't figure out why navigator.mediaDevices.getUserMedia() wasn't working after bundling for production. I used a custom entitlements file and it works just great! Thanks!

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>com.apple.security.cs.allow-jit</key><true/>
  <key>com.apple.security.cs.allow-unsigned-executable-memory</key><true/>
  <key>com.apple.security.cs.allow-dyld-environment-variables</key><true/>
  <key>com.apple.security.device.audio-input</key><true/>
  <key>com.apple.security.device.camera</key><true/>
</dict>
</plist>
@HeberLemus

This comment has been minimized.

Copy link

@HeberLemus HeberLemus commented Jul 29, 2019

@patrickmichalina did you assign that to both entitlements and entitlementsInherit keys?

@patrickmichalina

This comment has been minimized.

Copy link

@patrickmichalina patrickmichalina commented Jul 29, 2019

I used both but should it be different?

 "mac": {
  "entitlements": "entitlements.mac.plist",
  "entitlementsInherit": "entitlements.mac.plist"
}
@alibosworth

This comment has been minimized.

Copy link

@alibosworth alibosworth commented Oct 21, 2019

@semireg I'm having trouble finding the dynamically generated sub-application that triggers the "___ cannot be opened because the developer cannot be verified" warning... Where did you find it?

@semireg

This comment has been minimized.

Copy link

@semireg semireg commented Oct 22, 2019

Try a find /var/folders | grep -i BUNDLEID to get an idea of where to look. It’s probably buried 4 directories deep. You may also want to check console.app for absolute paths that trigger these warnings. Use a unique portion of your BUNDLEID as a search filter.

@klen

This comment has been minimized.

Copy link

@klen klen commented Oct 23, 2019

@semireg man you saved me! Your investigation totally makes sense. I was struggling with notarization for a few days and it's completely wrong. Today I've passed the review.

I'm able to confirm you don't need notarization for MAS. Ensure that if you have native modules you have to unpack them from asar. If you sign your application the "developer cant be verified" problem on Catalina related only to dynamically unpacking modules.

@alibosworth

This comment has been minimized.

Copy link

@alibosworth alibosworth commented Oct 23, 2019

Yes thanks again @semireg, like @klen above we also passed review today thanks to your leads.

The problem for us was electron-spellchecker and including it in asarUnpack was part of the fix, although we also had to include node_modules/@felixrieseberg/spellchecker there as well.

Tips for anyone else on this same issue:

  • We built using Electron 6.0.12 and electron-builder 21.20.0 (see note below about this version)
  • Our MAS build was not notarized, hardenedRuntime: false, and had a provisioningProfile defined
  • Because we were using an electron-builder JSON config defined via ELECTRON_BUILDER_CONFIG env var, the asarUnpack in our package.json was ignored so we had to define it in the config-specific JSON file. Check the generated builder-effective-config.yaml to see what options EB is really using. (It may be a bug that the configs aren't merged)
  • As discussed above, the steps to correlate Catalina's "cannot be opened because the developer cannot be verified" warnings:
    1. (on catalina machine where error seen) find /private/var/folders -name [cryptic filename from warning]
    2. dump strings with strings [full path to resulting file]
    3. note a string that seems unique to the module, in our case one was "getAvailableDict"
    4. (on build machine) search all modules in app source dir mdfind "[unique string]" -onlyin ./app/node_modules
  • It is also useful to "Show Package Contents" on your built app, to see what Resources/app.asar.unpacked contains. You can also copy out and extract app.asar to see what it contains
  • Electron-builder 21.2.0 has a bug with Electron 6 which was fixed/merged a month ago but didn't get into a release until yesterday (22.1.0) which was after our successful build. The bug required that this small fix be applied locally
@semireg

This comment has been minimized.

Copy link

@semireg semireg commented Oct 23, 2019

I'm relieved we have work-arounds in place - and nervous that the main devs like @develar and @stefanjudis have been missing in this process.

I suggest everyone donate money to this project so we can get the most experienced developers involved in these recent issues.

https://www.electron.build/donate

Make it rain! 💰💰💰

@danielweck

This comment has been minimized.

Copy link
Contributor

@danielweck danielweck commented Oct 23, 2019

Note that publishing Electron apps to the Windows 10S store also requires keeping native NodeJS plugins outside of app.asar, due to the security policy (i.e. signed executables / dynamic libraries).
I have been using this electron-builder directive to ensure that all native modules (regardless of their "owner" NPM package) are deployed into the app.asar.unpacked folder as standalone files (which would otherwise be extracted into temporary folders at runtime, thus the security breach):

    "asarUnpack": [
      "**/*.node"
    ],
@johannesjo

This comment has been minimized.

Copy link

@johannesjo johannesjo commented Oct 27, 2019

I finally got it working. My debugging efforts were seriously confused by this bug: electron/electron#9985

So take care to comment out app.makeSingleInstance() in case your trying to debug this.

@JohnTendik

This comment has been minimized.

Copy link
Contributor

@JohnTendik JohnTendik commented Oct 28, 2019

@johannesjo amazing! thank you, the singleApp fix allowed me to build a mas-dev application and have it launch! This was the problem haha but now the app just shows blank white screen.. Time to debug this problem now 😂

@JohnTendik

This comment has been minimized.

Copy link
Contributor

@JohnTendik JohnTendik commented Oct 28, 2019

Awesome! Im happy to report we've solved all the build problems that we've been experiencing. Thank you @johannesjo, this was the issue driving us up a wall 😂

"mac": {
      "category": "public.app-category.productivity",
      "icon": "./assets/icons/icon.icns",
      "asarUnpack": [
        "**/*.node"
      ],
      "entitlements": "mac.plist",
      "entitlementsInherit": "mac.plist",
      "aftersign": "notarize.js",
      "gatekeeperAssess": false
    },
    "mas": {
      "provisioningProfile": "./embedded.prod.provisionprofile",
      "entitlements": "./entitlements.mas.plist",
      "entitlementsInherit": "./entitlements.mas.inherit.plist",
      "hardenedRuntime": false,
      "provisionProfile": "YOURPROVISIONPROFILE"
    },

entitlements for mac:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.network.client</key>
    <true/>
    <key>com.apple.security.files.user-selected.read-write</key>
    <true/>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
  </dict>
</plist>

mac entitlementsInherit is the same as regular mac entitlements

MAS entitlements:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
    <key>com.apple.security.network.client</key>
    <true/>
    <key>com.apple.security.files.user-selected.read-only</key>
    <true/>
    <key>com.apple.security.files.downloads.read-write</key>
    <true/>
    <key>com.apple.security.files.user-selected.read-write</key>
    <true/>
    <key>com.apple.security.files.all</key>
    <true/>
    <key>com.apple.security.application-groups</key>
    <string>xxxENTERYOUROWNTHINGHERE</string>
  </dict>
</plist>

MAS entitlementsInherit:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
    <key>com.apple.security.inherit</key>
    <true/>
  </dict>
</plist>
@DominikLevitsky

This comment has been minimized.

Copy link

@DominikLevitsky DominikLevitsky commented Oct 31, 2019

We are currently struggling with this issue. Electron 5.0.11 and 6.1.2, electron-builder 22.1.0, macOS Catalina.

package.json:

...
"mac": {
    "hardenedRuntime": true,
    "gatekeeperAssess": false,
    "entitlements": "mac.plist",
    "entitlementsInherit": "mac.plist"
},
"afterSign": "resources/notarize.js",
...

mac.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
	<dict>
		<key>com.apple.security.files.user-selected.read-write</key>
		<true/>
		<key>com.apple.security.cs.allow-jit</key>
		<true/>
		<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
		<true/>
		<key>com.apple.security.cs.allow-dyld-environment-variables</key>
		<true/>
		<key>com.apple.security.network.client</key>
		<true/>
	</dict>
</plist>

Build is OK, signing is OK, notarizing is OK. Launching the app: crashes with EXC_BAD_ACCESS (Code Signature Invalid). A notarized app without the hardenedRuntime works perfectly fine. What are we doing wrong? Where can be the problem? What direction to look at?

@JohnTendik

This comment has been minimized.

Copy link
Contributor

@JohnTendik JohnTendik commented Oct 31, 2019

@DominikLevitsky what are you signing you mac app for? Is this standalone version or app store version? Also which certificate are you signing with? Youll need different cert for standalone vs app store builds.

Also for us we have not yet tested with Catalina so try the asarUnpack thing people mentioned above or the way you see it on my config.

@DominikLevitsky

This comment has been minimized.

Copy link

@DominikLevitsky DominikLevitsky commented Oct 31, 2019

@JohnTendik We are signing for mac, standalone, .dmg. We are signing with a certificate that worked flawlessly before Catalina. Also, as I mentioned this same config, with hardenedRuntime: false, works and launches absolutely fine. So I guess the certificate should be fine?

"asarUnpack": [ "**/*.node" ] Does not help.

@alibosworth

This comment has been minimized.

Copy link

@alibosworth alibosworth commented Oct 31, 2019

Run the crashing app through https://brockerhoff.net/RB/AppCheckerLite/

We had success in pairing down the entitlements (not defining any custom entitlements for MAC build, but we still have some for MAS build)

@dongjian

This comment has been minimized.

Copy link

@dongjian dongjian commented Nov 8, 2019

Further info:

You need at least the following entitlements:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>
com.apple.security.cs.allow-jit</key>
    <true/>
    <key>
com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>
com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
  </dict>
</plist>

Further info from https://medium.com/@TwitterArchiveEraser/notarize-electron-apps-7a5f988406db

Make sure to try out starting the app after this process, if some entitlements are missing, the app might be broken after notarization.

really thx !!!! solved my problem!!!!

@akashnimare

This comment has been minimized.

Copy link
Contributor

@akashnimare akashnimare commented Nov 9, 2019

Does anyone know how to fix the following error -

Error: Failed to upload app to Apples notarization servers

2019-11-09 18:03:49.456 altool[32910:902264] 
*** Error: Unable to validate your application. 
This Apple ID has been locked for security reasons. 
Visit iForgot to reset your account (https://iforgot.apple.com).

I'm trying to notarize my app and distribute on GitHub. I have a valid developer certificate which used to work well but everything is broken on Catalina.

entitlements.mac.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
  </dict>
</plist>


electron-builder: 20.43.0
electron-notarize: 0.1.1

Package.json

  "build": {
    "afterSign": "scripts/notarize.js",
    "asar": true,
    "asarUnpack": [
      "**/*.node"
    ],
    "files": [
      "**/*",
      "!docs${/*}",
      "!node_modules/@paulcbetts/cld/deps/cld${/*}"
    ],
    "mac": {
      "category": "public.app-category.productivity",
      "darkModeSupport": true,
      "artifactName": "${productName}-${version}-${arch}.${ext}",
      "hardenedRuntime": true,
      "entitlements": "build/entitlements.mac.plist",
      "entitlementsInherit": "build/entitlements.mac.plist",
      "gatekeeperAssess": false
    },

notarize.js

require('dotenv').config();
const { notarize } = require('electron-notarize');

exports.default = async function notarizing(context) {
  const { electronPlatformName, appOutDir } = context;  
  if (electronPlatformName !== 'darwin') {
    return;
  }

  const appName = context.packager.appInfo.productFilename;

  return await notarize({
    appBundleId: 'my_app_id',
    appPath: `${appOutDir}/${appName}.app`,
    appleId: process.env.APPLE_ID,
    appleIdPassword: process.env.APPLE_ID_PASS,
  });
};
@JohnTendik

This comment has been minimized.

Copy link
Contributor

@JohnTendik JohnTendik commented Nov 9, 2019

@akashnimare that error says your apple developer account has been locked. Are you sure you're using the correct password? It should be an app-specific password you create in the account settings page, after turning on 2-factor auth.

Also, it was mentioned above that for mac notarization you need some more entitlements. Try adding these

<key>com.apple.security.cs.allow-jit</key>
<true/>
<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
<true/>
<key>com.apple.security.cs.allow-dyld-environment-variables</key>
<true/>
@akashnimare

This comment has been minimized.

Copy link
Contributor

@akashnimare akashnimare commented Nov 11, 2019

@JohnTendik thanks a lot for the help. Let me try it. BTW, I need to get the password from here, right? I have 2FA on.

image

I have tried your fix but the npm run dist command hangs here for hours -

Failed to upload app to Apples notarization servers
  • signing         file=dist/mac/Zulip.app identityName=Developer ID Application: Namme, Inc. (XXXXXXXX) identityHash=XXXXXXXXXXXXXXX provisioningProfile=none


    electron-builder     at processImmediate (timers.js:658:5) +0ms
Error: Failed to upload app to Apples notarization servers

2019-11-11 18:25:18.731 altool[91150:1226453] *** Error: Unable to validate your application. This Apple ID has been locked for security reasons. Visit iForgot to reset your account (https://iforgot.apple.com).

    at /Users/myname/dev/work/zulip-desktop/node_modules/electron-notarize/src/index.ts:76:13
    at step (/Users/myname/dev/project/zulip-desktop/node_modules/electron-notarize/lib/index.js:31:23)
    at Object.next (/Users/myname/dev/project/zulip-desktop/node_modules/electron-notarize/lib/index.js:12:53)
    at fulfilled (/Users/myname/dev/project/zulip-desktop/node_modules/electron-notarize/lib/index.js:3:58)
    at process._tickCallback (internal/process/next_tick.js:68:7)
From previous event:
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
From previous event:
    at MacPackager.doPack (/Users/myname/dev/project/zulip-desktop/node_modules/app-builder-lib/src/platformPackager.ts:167:165)
    at /Users/myname/dev/project/zulip-desktop/node_modules/app-builder-lib/src/macPackager.ts:90:63
    at Generator.next (<anonymous>)
From previous event:
    at MacPackager.pack (/Users/myname/dev/project/zulip-desktop/node_modules/app-builder-lib/src/macPackager.ts:82:95)
    at /Users/myname/dev/project/zulip-desktop/node_modules/app-builder-lib/src/packager.ts:430:24
    at Generator.next (<anonymous>)
    at xfs.stat (/Users/myname/dev/project/zulip-desktop/node_modules/fs-extra/lib/mkdirs/mkdirs.js:56:16)
    at callback (/Users/myname/dev/project/zulip-desktop/node_modules/fs-extra/node_modules/graceful-fs/polyfills.js:289:20)
    at FSReqWrap.oncomplete (fs.js:154:5)
From previous event:
    at Packager.doBuild (/Users/myname/dev/project/zulip-desktop/node_modules/app-builder-lib/src/packager.ts:396:24)
    at /Users/myname/dev/project/zulip-desktop/node_modules/app-builder-lib/src/packager.ts:366:57
    at Generator.next (<anonymous>)
    at /Users/myname/dev/project/zulip-desktop/node_modules/fs-extra/node_modules/graceful-fs/graceful-fs.js:111:16
    at /Users/myname/dev/project/zulip-desktop/node_modules/graceful-fs/graceful-fs.js:43:10
    at /Users/myname/dev/project/zulip-desktop/node_modules/fs-extra/node_modules/graceful-fs/graceful-fs.js:45:10
    at FSReqWrap.args [as oncomplete] (fs.js:140:20)
From previous event:
    at Packager._build (/Users/myname/dev/project/zulip-desktop/node_modules/app-builder-lib/src/packager.ts:335:133)
    at /Users/myname/dev/project/zulip-desktop/node_modules/app-builder-lib/src/packager.ts:331:23
    at Generator.next (<anonymous>)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
From previous event:
    at Packager.build (/Users/myname/dev/project/zulip-desktop/node_modules/app-builder-lib/src/packager.ts:288:14)
    at build (/Users/myname/dev/project/zulip-desktop/node_modules/app-builder-lib/src/index.ts:59:28)
    at build (/Users/myname/dev/project/zulip-desktop/node_modules/electron-builder/src/builder.ts:228:10)
    at then (/Users/myname/dev/project/zulip-desktop/node_modules/electron-builder/src/cli/cli.ts:49:19)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! zulip@4.0.1 dist: `tsc && electron-builder`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the zulip@4.0.1 dist script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/myname/.npm/_logs/2019-11-11T12_55_18_768Z-debug.log

Even though I'm using the right ID and pass, Apple is not letting me notarize the app.

@akashnimare

This comment has been minimized.

Copy link
Contributor

@akashnimare akashnimare commented Nov 11, 2019

You need to add you app email and password as environment variables appleId and appleIdPassword

@itsthisjustin is this the same as my appleID/applePassword or something different?

@itsthisjustin

This comment has been minimized.

Copy link

@itsthisjustin itsthisjustin commented Nov 11, 2019

You need to add you app email and password as environment variables appleId and appleIdPassword

@itsthisjustin is this the same as my appleID/applePassword or something different?

Yes but it’s an app password that you have to generate in the account.

@akashnimare

This comment has been minimized.

Copy link
Contributor

@akashnimare akashnimare commented Nov 11, 2019

@itsthisjustin this one, right -

@JohnTendik

This comment has been minimized.

Copy link
Contributor

@JohnTendik JohnTendik commented Nov 11, 2019

@akashnimare Yes, correct. "App-specific Passwords" you want to generate one and use that as the applepassword and ur email as the appleId

@akashnimare

This comment has been minimized.

Copy link
Contributor

@akashnimare akashnimare commented Nov 11, 2019

Okay, thanks. I have tried it but I'm still getting the error -


 Error: Failed to upload app to Apples notarization servers
                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                               2019-11-11 20:05:20.107 altool[1017:1275185] *** Error: Unable to validate your application. 
This Apple ID has been locked for security reasons. 
Visit iForgot to reset your account (https://iforgot.apple.com).

My .env file looks like this -

APPLE_ID: 'abc@gmail.com',
APPLE_ID_PASS: 'xxxx-xxx-xxxx-xxxx'

electron-builder: v22.0.1
electronn-notarize: v0.2.0
electron: v3.1.10
node: v10.16.3

Other details here.

Is there any hidden thing I'm missing here. I have spent weeks trying to notarize the app but no luck :/ can you help @JohnTendik, @itsthisjustin?

Here is the app I'm trying to build - https://github.com/zulip/zulip-desktop

@JohnTendik

This comment has been minimized.

Copy link
Contributor

@JohnTendik JohnTendik commented Nov 11, 2019

@akashnimare my guess is your .env file is not working. Try adding debug=electron-osx-sign and see if the log changes. How are you running your build / notarize ?

Try hard coding the apple id and password inside then notarize js file and see if that fixes it. If so, then you know your env file is not working.

@akashnimare

This comment has been minimized.

Copy link
Contributor

@akashnimare akashnimare commented Nov 11, 2019

@JohnTendik okay, on your suggestion I hardcoded the appleID and pass into the notarize.js script and I didn't receive any error. I'm guessing the code-signing + notarization worked well. I just have two questions now -

Update - I have received the email from apple so I think the app was successfully notarized. Just need to figure out why .env is not working. I'm using electron-builder here with npm run dist to build + notarize the app.

@JohnTendik

This comment has been minimized.

Copy link
Contributor

@JohnTendik JohnTendik commented Nov 11, 2019

you should receive an email saying the notarization was successful. You can also test it, try to install the app and launch it, if you don't get the malware message than you have notarized successfully.

@alibosworth

This comment has been minimized.

Copy link

@alibosworth alibosworth commented Nov 11, 2019

@akashnimare

Is there any way to know that my app is notarized?

It should be indicated in this tool https://brockerhoff.net/RB/AppCheckerLite/

What could be wrong with the .env script?

An .env file is only useful if you have something configured to read those values and load them as actual Environment Variables. That topic is a general one and has nothing to do with electron-builder or this issue. Electron-builder DOES have its own built-in support for loading EVs from a file however you must name the file electron-builder.env Please see the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.