Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notarization fails on MacOS 10.15.3 #4656

Closed
Ankitr19 opened this issue Feb 7, 2020 · 59 comments
Closed

Notarization fails on MacOS 10.15.3 #4656

Ankitr19 opened this issue Feb 7, 2020 · 59 comments
Labels

Comments

@Ankitr19
Copy link

Ankitr19 commented Feb 7, 2020

We are trying to notarize our electron app, but the notarization is not working . It fails with the following issue :-

"issues": [ { "severity": "error", "code": null, "path": "/Contents/Resources/app/node_modules/fsevents/build/Release/.node", "message": "The binary is not signed.", "docUrl": null, "architecture": "x86_64" }, { "severity": "error", "code": null, "path": "/Contents/Resources/app/node_modules/fsevents/build/Release/.node", "message": "The signature does not include a secure timestamp.", "docUrl": null, "architecture": "x86_64" } ]

How to resolve this error, and also if we have a method for deep code signing in electron-builder?

@twigs67
Copy link

twigs67 commented Feb 9, 2020

I'm having the same issue with both fsevents and Puppeteer. @Ankitr19 did you figure this out?

Also, are you using electron-notarize?

@dboakes
Copy link

dboakes commented Feb 10, 2020

Started having the same issue within the last week. For me it's python binaries within app.asar.unpacked. Presumably Apple have changed how they handle notarization?

@timfish
Copy link
Contributor

timfish commented Feb 10, 2020

Notarisation has started failing for us for the same reason.

{
  "logFormatVersion": 1,
  "jobId": "xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxxxx",
  "status": "Invalid",
  "statusSummary": "Archive contains critical validation errors",
  "statusCode": 4000,
  "archiveFilename": "OurApp.zip",
  "uploadDate": "2020-02-10T12:43:07Z",
  "sha256": "9517c159237a6c18c16d439894eb16abec73757c5e194833bdec7a35bad661a2",
  "ticketContents": null,
  "issues": [
    {
      "severity": "error",
      "code": null,
      "path": "OurApp.zip/OurApp.app/Contents/Resources/app.asar.unpacked/node_modules/fsevents/build/Release/.node",
      "message": "The binary is not signed.",
      "docUrl": null,
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "OurApp.zip/OurApp.app/Contents/Resources/app.asar.unpacked/node_modules/fsevents/build/Release/.node",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": null,
      "architecture": "x86_64"
    }
  ]
}

@twigs67
Copy link

twigs67 commented Feb 10, 2020

Ok, I think I narrowed it down. This tweet/video helped me.

So it looks like we have to sign each of the individual packages. Now the question is, how do we do that? Any guesses @timfish @dboakes @Ankitr19 ?

@timfish
Copy link
Contributor

timfish commented Feb 10, 2020

It looks like Apple tightened up notarisation on Feb 3rd and warnings became errors. Because warnings are not displayed by electron-notarize we're only finding out about them now.

For electron-builder macOS configuration there is a binaries option but looking at it's usage, these need to be an array of full paths to each binary. electron-builder is already detecting binaries for relocation to app.asar.unpacked so it should ideally add these to the list of files to automatically sign.

@twigs67
Copy link

twigs67 commented Feb 10, 2020

@timfish I was about to mention electron-builder's option to add additional binaries. I'm not having any luck with it though right now, but I don't think I have the correct paths.

I'm using, myApp.zip/myApp.app/Contents/Resources/node_modules/puppeteer/.local-chromium/mac-706915/chrome-mac/Chromium.app/Contents/MacOS/Chromium

@timfish
Copy link
Contributor

timfish commented Feb 10, 2020

I think the signing step occurs before packaging so the path would be more like ./node_modules/puppeteer/.local-chromium/mac-706915/chrome-mac/Chromium.app/Contents/MacOS/Chromium.

I've also seen some logic in electron-builder that finds *.app within and signs them too. I'm pretty sure you want to be signing Chromium.app rather than the internal binary.

I'm in the process of running some CI builds with DEBUG=electron-builder to get some helpful debug output.

@twigs67
Copy link

twigs67 commented Feb 10, 2020

I'm trying out your suggestions now. I also found this plist param that seems to allow you to skip code signing for "arbitrary" plugins & frameworks. Not entirely sure what that means, but I tried it and it didn't work.

@timfish
Copy link
Contributor

timfish commented Feb 10, 2020

It seems like that might be a runtime option but you never know, maybe the notarisation service pays attention to it?

Edit
We already have com.apple.security.cs.disable-library-validation: true so I doubt that will fix it.

@twigs67
Copy link

twigs67 commented Feb 10, 2020

@timfish I'm not having any luck. I've tried a number of different ways. Are you having any luck?

I also noticed that sometimes I get this error and sometimes I don't.

    {
      "severity": "error",
      "code": null,
      "path": "myApp.zip/myApp.app/Contents/Resources/node_modules/fsevents/build/Release/.node",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": null,
      "architecture": "x86_64"
    }

I don't even have fsevents in my package.json. I have it in my yarn.lock file for some reason. I tried deleting it, but it still pops up. So obviously when I try adding it to the list of binaries, it says it can't find it.

@timfish
Copy link
Contributor

timfish commented Feb 10, 2020

Nope, no luck working around this for now.

fsevents is likely a sub-dependency of one of your dependencies. yarn why fsevents should tell you why its included.

@dboakes
Copy link

dboakes commented Feb 11, 2020

I tried messing around with afterPack yesterday, ran a function to manually codesign the binaries that are coming causing the issue, but like you said, says the path doesn't exist.

I'm all out of ideas as well, but desperate for a fix!

@kzimny
Copy link

kzimny commented Feb 11, 2020

Can you post your electron-builder config part from package.json? Do you build for MAS or without any target specified?

@dboakes
Copy link

dboakes commented Feb 11, 2020

"build": {
    "afterSign": "deploy/notarize.js",
    "appId": "com.electron.APP_NAME",
    "buildDependenciesFromSource": true,
    "generateUpdatesFilesForAllChannels": true,
    "icon": "build/assets/icon.png",
    "productName": "APP_NAME",
    "directories": {
      "buildResources": "build",
      "output": "dist"
    },
    "mac": {
      "hardenedRuntime": true,
      "gatekeeperAssess": false,
      "entitlements": "build/entitlements.mac.plist",
      "entitlementsInherit": "build/entitlements.mac.plist",
      "category": "public.app-category.sports",
      "target": "zip"
    },
    "asarUnpack": [
      "build/pydist"
    ],
    "publish": [
      {
        "provider": "github",
        "owner": "OWNER_NAME",
        "repo": "APP_NAME",
        "token": "TOKEN",
        "private": true,
        "releaseType": "release"
      }
    ]
  }

@kzimny
Copy link

kzimny commented Feb 11, 2020

Try to unpack all .node executable files and put them in a folder called app.asar.unpacked. You do this by adding:

"asarUnpack": [
  "**/*.node"
]

Then the files will be signed and Apple will be able to read the signature.

@timfish
Copy link
Contributor

timfish commented Feb 11, 2020

@kzimny we are using that option and it doesn't appear to help. Our config is:

productName: **snip**
appId: **snip**
copyright: Copyright © **snip**
directories:
  buildResources: '../build'
  output: '../releases'
  app: './'
files:
  - '**/*'
  - '!**/*.{map,dll,so,dylib,cat,inf,sys,exe}'
  - '!**/node_modules/**/*.{cc,c,h,obj,pdb,map,lib,tlog,m4,sh,S}'
  - '!**/node_modules/**/doc/**'
  - '!**/node_modules/**/configure'
afterPack: ../build/afterpack.js
afterSign: ../build/notarize.js
asarUnpack:
  - '**/*.node'
win:
  **snip**
linux:
  **snip**
mac:
  hardenedRuntime: true
  darkModeSupport: true
  extraFiles:
    - from: ../build/${os}/${arch}
      to: ./resources/
      filter:
        - '**/*'
  target:
    - target: dmg
      arch:
        - x64

and we're using the latest electron-builder and electron-notarize.

Since Apple tightened up their notarisation on 3rd of Feb, we've started getting this error. I've run some builds with DEBUG=electron-builder where I can see the final sign step but it doesn't look like it's signing any other binaries.

@kzimny
Copy link

kzimny commented Feb 11, 2020

Just for a test, try to remove the target and all excluded files:

files:
  - '**/*'

@dboakes
Copy link

dboakes commented Feb 11, 2020

Made no difference for me sadly. For me, the binaries that are showing as unsigned, are within the python binary I am already unpacking.

@timfish
Copy link
Contributor

timfish commented Feb 11, 2020

Just for a test, try to remove the target and all excluded files

Build is running in CI. Will let you know how it goes. I could dump the debug output here but it would take me a while to sanitise it!

@timfish
Copy link
Contributor

timfish commented Feb 11, 2020

No, notarisation still fails with the same error with the simplified config.

Oh, one point worthy of note is that we don't get any notarisation errors about the other 5 native modules we're using (keytar, usb-detection, level, ref and ffi) which are all direct dependencies of our app. The error only lists fsevents which is a dependency of chokidar on macOS. So maybe the other native modules are getting signed? There is nothing in the debug output to suggest they are getting signed but Apple isn't complaining about them.

I've got to leave my desk for much of the rest of the day but later I'll try a build with chokidar excluded and see if that fixes it.

@twigs67
Copy link

twigs67 commented Feb 11, 2020

but like you said, says the path doesn't exist.

@dboakes I debugged the CI build yesterday and fsevents doesn't become available until some point after the code signing event and I'm not sure how to remove chokidar from the build as @timfish suggested.

I tried manually code signing each binary in travis-ci, but I was unable to add the keychain to "list-keychains".

Has anyone tried to codesign manually?

@twigs67
Copy link

twigs67 commented Feb 11, 2020

I was able to successfully notarize the app, but I've made so many changes I'm not sure what did it. However, I made the following changes prior to:

  • Removed build.extraResources: [ 'node_modules'] from package.json
  • Added asarUnpack
  • Updated osx_image in .travis.yml to xcode11.3 (part of the new requirements state you have to have SDK version 10.9 or above, I was on 10

package.json

"build": {
    "asar": true,
    "mac": {
      "target": [
        "dmg",
        "zip"
      ],
      "hardenedRuntime": true,
      "entitlements": "./build/entitlements.mac.plist",
      "entitlementsInherit": "./build/entitlements.mac.plist",
      "gatekeeperAssess": true,
      "cscLink": "build/DevApp_cert.p12",
      "cscInstallerLink": "build/devInstaller_cert.p12"
    },
    "afterSign": "./build/afterSignHook.js",
    "dmg": {
        "sign": false
    }
  },

entitlements.mac.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
  </dict>
</plist>

I also just ran a build without asarUnpack (because that causes path issues in my app) and it worked.

Update: my app is now not running correctly, because it cannot find Puppeteer. It's included in the package, but it can't reach it. It only seems to work if I add it to extraResources, but then it gives me the issue we've all been dealing with.

@kzimny
Copy link

kzimny commented Feb 11, 2020

Great. Thank you for the information.

@Christilut
Copy link

Same problem here. I tried asarUnpack but seemed to make no difference in the errors that Apple reports in their log. Also tried with disable-library-validation.

It's mostly tripping over extraResources that I added, all the .so and .dylib files.

@timfish
Copy link
Contributor

timfish commented Feb 12, 2020

We're using extraFiles over extraResources but we're not seeing any errors about anything we copy there.

@twigs67
Copy link

twigs67 commented Feb 12, 2020

@timfish I tried using extraFiles instead of extraResources, but my app still can't locate Puppeteer. Can you think of any way around this?

Is there a way to add these directories to the system path so that it will look there when being imported? I see the files option has to and from, but I'm not sure how I'm supposed to use that.

@timfish
Copy link
Contributor

timfish commented Feb 12, 2020

@twigs67 I've not solved these issues yet for our app yet so I'm not really in a position to advise!

Today I'm going to put together a simple reproduction of this issue. This is not being helped by the fact that the npm registry is returning 404 for 7zip-bin which is a dependency for electron-builder!

@Christilut
Copy link

What does work is codesign these extra resources manually, something like this: codesign -s "Company Name" --options=runtime my-file
The Apple server no longer complains about them then.

I'm still figuring out if I should write a script that codesigns everything or some other method. But this is a temporary workaround at the very least.

@timfish
Copy link
Contributor

timfish commented Feb 12, 2020

Our signing is all on CI and uses CSC_LINK and CSC_KEY_PASSWORD so it's going to be a pain to get manual codesign working with that.

I'd rather get this fixed in electron-builder so everyone can benefit from it.

@ching2018
Copy link

我通过找到一种将其删除fsevents为依赖项的方法来解决此问题。

I use puppeteer whith Chromium.The fsevents is not in devDependencies.

@twigs67
Copy link

twigs67 commented Feb 17, 2020

Ya, I found a workaround for this as I wasn't able to get the Chromium binary to sign or Apple to take it. @HoldSkill are you able to notarize your app with Chromium?

@Ankitr19
Copy link
Author

We resolved the issue, by moving fsevents as a Dev dependency.

@parachvte
Copy link

parachvte commented Feb 25, 2020

I've checked the debug output with DEBUG=electron-osx-sign and it lists all the binaries it's signing in the app. Everything looks good and there is no erroneous node_modules/fsevents/build/Release/.node entry listed there.

@timfish I did set DEBUG=electron-osx-sign as you mentioned, seems like the electron-osx-sign ignores executable files that are hidden (start with dot), while two other files (fsevents.node and fse.node) are not ignored:

...
electron-osx-sign Signing... /Users/path-to-my-app.app/Contents/Resources/app/node_modules/fsevents/bin/darwin-x64-57/fsevents.node +302ms
electron-osx-sign Signing... /Users/path-to-my-app.app/Contents/Resources/app/node_modules/fsevents/build/Release/fse.node +297ms
...

And I checked the fsevents folder in node_modules/fsevents/build/Release with ls -al, both fse.node and .node executables are listed there.

@parachvte
Copy link

parachvte commented Feb 25, 2020

After read through the source code of electron-osx-sign, I think the problem is that in https://github.com/electron/electron-osx-sign/blob/master/util.js#L227, it rejectes hidden file when walking through Contents folder.

case '': // Binary
  if (path.basename(filePath)[0] !== '.') {
    return getFilePathIfBinaryAsync(filePath)
  } // Else reject hidden file
  break

@develar @sethlu Have any clue that why hidden files are ignored, and why they are still included in _CodeSignature/CodeResources? There is a related PR with this issue open since 2018.

@timfish
Copy link
Contributor

timfish commented Feb 25, 2020

@parachvte It looks like electron-builder vendors electron-osx-sign here.

@parachvte
Copy link

parachvte commented Feb 25, 2020

@timfish
Yes @develar copied electron-osx-sign here in order to solve MAS build on July 17 2019. Later on the two branches get maintained independently. I don't know how it goes but the source code I posted above remains the same in both repos.

Edit: Finally we resolve this issue by moving fsevents dependencies to devDependencies.
Common related packages like:

  • chokidar
  • babel-cli
  • nodemon
  • stylint
  • webpack (<-- we misplaced some webpack deps in dependencies

parachvte added a commit to parachvte/Luban that referenced this issue Feb 25, 2020
@sethlu
Copy link
Member

sethlu commented Mar 5, 2020

I think there are kind of 2 questions brought up here?

  1. Why are hidden files ignored?
  2. Why is the PR feat: Codesign all binary-like files electron/osx-sign#169 there?

The historical/original intention for hidden files is that files that .gitignore or .DS_Store can be skipped because they aren't really executable binaries. Also, funnily they don't have an extension name because the dot-DS_Store is its name (as far as path.basename() concerns). The actual binary executables also don't usually have an extension. So the decision was made to skip all the hidden-file-looking files.

However, this turned out to be overlook as reported in electron/osx-sign#168 not because native node modules are ignored and causing code-signing issues, but because the compiled native code modules can simply be named as .node and placed in a dist directory (which was unheard of, at that time, to me). And the current version of electron-osx-sign@0.4.15 still skips the .node file, but you can force-code-sign that file by passing an argument to electron-osx-sign.

Therefore, electron/osx-sign#169 is created so that we check all files if they are binary-looking and we code-sign them all. However, I didn't have much time to test it, so it's left as an open PR. Theoretically, it should work. I'll be curious to hear if this PR works for people so it can be merged to master without causing troubles for existing workflows that people set up.

The _CodeSignature/CodeResources are artifacts from code-signing. And you may find more information about them here: https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/AboutCS/AboutCS.html

Hope this helps!

@eschirtz
Copy link

To add my two cents..

I'm also experiencing similar issues with deep code signing, this time with @tensorflow/tfjs-node.

{
severity: "error",
code: null,
path: "Samply_Staging.zip/Samply (Staging).app/Contents/Resources/app.asar.unpacked/node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow.so",
message: "The binary is not signed.",
docUrl: null,
architecture: "x86_64"
},
{
severity: "error",
code: null,
path: "Samply_Staging.zip/Samply (Staging).app/Contents/Resources/app.asar.unpacked/node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow.so",
message: "The signature does not include a secure timestamp.",
docUrl: null,
architecture: "x86_64"
},
{
severity: "error",
code: null,
path: "Samply_Staging.zip/Samply (Staging).app/Contents/Resources/app.asar.unpacked/node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow_framework.so",
message: "The binary is not signed.",
docUrl: null,
architecture: "x86_64"
},
{
severity: "error",
code: null,
path: "Samply_Staging.zip/Samply (Staging).app/Contents/Resources/app.asar.unpacked/node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow_framework.so",
message: "The signature does not include a secure timestamp.",
docUrl: null,
architecture: "x86_64"
}

I've read through this thread a couple times now and can't quite find "the fix". Wondering if anyone has any input, or if any help is needed to get this resolved in electron-builder itself.

Thanks!

@eschirtz
Copy link

After inspecting _CodeSignature/CodeResources

I'm seeing entries for the two binaries that apple is complaining about, and I can't see why they are any different than any of the other files...

<key>Resources/app.asar.unpacked/node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow.so</key>
<dict>
	<key>hash</key>
	<data>
	9U5XsHIre9vGzF+oKWbZLjs2yM0=
	</data>
	<key>hash2</key>
	<data>
	Icrwobv2XnX5PewitMAGEA6XMbYB1C3IqQ+DbXQfzZk=
	</data>
</dict>
<key>Resources/app.asar.unpacked/node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow_framework.so</key>
<dict>
	<key>hash</key>
	<data>
	DvQTWVYAdgjJtJGJPNEyi6Xl9u0=
	</data>
	<key>hash2</key>
	<data>
	6CixRVOadOYI5f37uAXKK4RHAhED63ZVWCU6vgglxJk=
	</data>
</dict>

Again, any advice would be awesome!

@sethlu
Copy link
Member

sethlu commented Apr 3, 2020

@eschirtz Sorry about the delay. Can you open an issue at electron-osx-sign and I can take a closer look at the issue? My take on the problem you're having right now is that the tensorflow shared objects (.so) are not codesigned. If you're using electron-osx-sign, then you can explicitly provide a list of paths to these binary files to include them during codesigning.

@davidmurdoch
Copy link

Finally we resolve this issue by moving fsevents dependencies to devDependencies.

#4656 (comment)

@parachvte I'm currently trying the "devDependencies" workaround for fsevents (I think ours is coming from chokidar, which should technically be in "dependencies" for us). I don't understand how this could work though. Any insight as to why this would work at all?

@parachvte
Copy link

parachvte commented Apr 8, 2020

@davidmurdoch I would suggest that you check for an upgrade of fsevents or chokidar, I think that newer version of it/them already resolved the issue. By upgrading all dependencies (that requires old version of fsevents) to the newer version will solve your problem.

The second option is to sign code by calling electron-osx-sign command programmatically, specify paths to be signed manually as @sethlu suggested.

Also you could consider testing for electron/osx-sign#169 and get it merge into electron-builder main stream so everyone can benefit.

@eschirtz
Copy link

eschirtz commented Apr 9, 2020

@sethlu

That actually did work (specifying the binaries). Would you still like me to open an issue in that repo, or is this the expected functionality?

Thank you!

@sethlu
Copy link
Member

sethlu commented Apr 12, 2020

@eschirtz I'm glad it worked! This is the expected behavior at the moment before electron/osx-sign#169 is merged in. If you're interested in giving this PR a try perhaps we can make it available to everyone who may get similar issues of binary files missing codesigning.

@portah
Copy link

portah commented May 3, 2020

@sethlu Your patch worked. All binaries included in my bundle were signed.

@sethlu
Copy link
Member

sethlu commented May 4, 2020

@portah Thanks for testing! electron/osx-sign#169 is just merged and will come with a next release in a few days.

@stale
Copy link

stale bot commented Jul 14, 2020

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@gregoirepuget
Copy link

Solution to use puppeteer and notorize app :
https://www.npmjs.com/package/puppeteer-chromium-resolver

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests