Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build with codesign takes a long time when 'nodegit' is in my dependencies #1147

Closed
matthewtoast opened this issue Jan 19, 2017 · 9 comments
Closed
Labels

Comments

@matthewtoast
Copy link

  • Version: 11.3.0
  • Target: Mac

Problem:

I am trying to build a project that has nodegit@0.16.0 as a dependency. When I run:

CSC_LINK={mycertfilepath} CSC_KEY_PASSWORD={mypass} build --mac

The build takes a long time (i.e. never finishes). On my MacBook Pro (Retina, Mid 2012) running MacOS Sierra, the "Rebuilding" and "Packaging" steps take ~5 minutes, which is a bit slow. For the "Signing" step, however, I've waited 30+ minutes and never seen it finish.

With nodegit deleted from "dependencies", the build finishes in seconds.

Details:

A minimal package.json that reproduces the issue looks like this:

{
  "name": "repro-nodegit-electron-build-demo",
  "version": "0.0.0",
  "private": true,
  "description": "Minimal test of building Electron with nodegit and codesigning",
  "main": "index.js",
  "author": "Matthew",
  "engines": {
    "node": "6.6.0",
    "npm": "3.10.3",
    "electron": "1.4.5"
  },
  "build": {
    "appId": "com.electron.nodegitelectronbuilddemo",
    "copyright": "none",
    "productName": "nodegitelectronbuilddemo",
    "forceCodeSigning": true,
    "electronVersion": "1.4.5",
    "mac": {
      "category": "public.app-category.developer-tools"
    }
  },
  "scripts": {
    "build": "build --mac"
  },
  "dependencies": {
    "nodegit": "0.16.0"
  },
  "devDependencies": {
    "electron-builder": "11.3.0"
  }
}

(To repro the signing slowness, you would need to get a developer certificate, etc.)

Hypotheses:

Based on the output of $ ps ax | grep codesign while running the build, my guess is that there are simply so many files to sign that it is just taking a long time. I see it signing a lot of files that seem ancillary.

With that in mind, these are my hypotheses:

  • I am supposed to somehow pre-bundle my code before using electron-builder. (I.e. concatenate my JavaScript files, etc., so that there are fewer files to sign?)

  • I am supposed to configure the "files" build option so that code-signing doesn't have to trawl through a bunch of ancillary files. (It looks like nodegit and its dependencies are rather huge, and there are some test/VCS artifacts also being signed.)

  • I have misconfigured the installation of nodegit somehow, resulting in a big complicated bundle.

  • I am supposed to pre-compile nodegit down to a single bundle before using it here.

  • Waiting over 30 minutes in my scenario is normal. This is simply a fact of life I will have to accept if I want to include nodegit in my project.

  • I am supposed to use the two-package.json setup.

Am I onto something with any of these? Any tips or suggestions?

Apologies if I've missed some important note in the documentation that pertains to this.

@matthewtoast matthewtoast changed the title Build takes a long time when 'nodegit' is in my dependencies Build with codesign takes a long time when 'nodegit' is in my dependencies Jan 19, 2017
@matthewtoast
Copy link
Author

matthewtoast commented Jan 21, 2017

When I ps ax | grep codesign as the codesign step is working, I see output like:

codesign --sign {name_of_my_developer_id_certificate} --force --keychain {target_file_in_temp_folder} node_modules/vendor/libgit2/tests/resources/merge-resolve/.gitted/objects/54/269b3f6ec3d7d4ede24dd350dd5d605495c3ae

So it looks like codesign is going deep into hidden folders, including VCS databases, and signing all of those entries too. That's a lot to sign. Which explains the wait.


My workaround is to add some "files" entries to the build config in my package.json, so that there are fewer files for codesign to crunch:

"build": {
  "files": [
    // ...snip...
    "!node_modules/nodegit",
    "node_modules/nodegit/package.json",
    "node_modules/nodegit/dist/**",
    "node_modules/nodegit/build/Release/nodegit.node",
    // ...snip...
  ]
}

First I exclude the whole nodegit package from my app bundle. Then, I re-include only the parts of nodegit necessary for it to work.

I did this through trial and error, so my file patterns are probably excluding some critical file. But this at least gets the build working, and I don't initially see any problems with require('nodegit') when I launch my packaged app.

@matthewtoast
Copy link
Author

I think it's unlikely there will be a lot of commentary on this one, so I'm going to close it.

@develar
Copy link
Member

develar commented Jan 26, 2017

@matthewtoast No, it will be investigated and fixed when I will have time :)

@develar
Copy link
Member

develar commented Jan 31, 2017

Solution — do not use npm. Use yarn and execute yarn clean. All junk directories like vendor/libgit2/tests will be removed.

@fab1an
Copy link

fab1an commented Sep 8, 2017

I have a similar problem, but I don't want to use yarn (I used it for a year, but switch back to npm after various troubles with it).

For me codesign, the final package _CodeSignature contains entries like:

                <key>Resources/app.asar.unpacked/node_modules/lzz-gyp/package.json</key>
		<dict>
			<key>hash</key>
			<data>
			V46JaYPK9FUORjoONzTgww3ovBo=
			</data>
			<key>hash2</key>
			<data>
			c2Zqm6KAgz7BZzpLOD8K9yrP1ocqjCjA0V/jJv58gc0=
			</data>
		</dict>
  • Is this needed?
  • Are signatures for .cpp or .dylib needed?

@develar
Copy link
Member

develar commented Sep 13, 2017

Yes, such files must be signed. Because it is library.

@develar
Copy link
Member

develar commented Sep 13, 2017

You can try to add ignores to files option if some files are not required for you.

@andrewluetgers
Copy link

andrewluetgers commented Oct 24, 2019

Curious if this still applies in 2019? Apparently https://yarnpkg.com/lang/en/docs/cli/autoclean/ is the current command and there are separate packages that can accomplish this for npm users such as https://github.com/tj/node-prune

storm1er added a commit to Ryzen-Controller-Team/Ryzen-Controller that referenced this issue Jan 14, 2020
BREAKING CHANGE: ⚠️ You must uninstall the previous version to avoid any conflicts.

For users:
- New installer/uninstaller with customizable installation path
- Better night mode
- Release tab has been deleted
- Apply on [...] are now in the preset tabs with more options
- Notifications are now handled within the app/system accordingly to the app's minimized state

For developers:
- Now using React and TypeScript
- Switched from npm to yarn: electron-userland/electron-builder#1147 (comment)
- Updated Continuous Delivery to GitLab v12.7.0
storm1er added a commit to Ryzen-Controller-Team/Ryzen-Controller that referenced this issue Mar 14, 2020
BREAKING CHANGE: ⚠ Please avoid using v1 and v2 simultaneously.

This version is stable enough to be released but is still under heavy development,
**You may encounter bugs, if you encounter any, you can uninstall this version and go back to v1.**

For users:
- Users can now share presets and vote for them.
- New installer/uninstaller with customizable installation path
- #66 Add multi-language: English, Français, 简体中文, Deutsch, Türkçe.
- Presets triggers are now in preset tabs with more options
- Added splash screen
- Better night mode
- Release tab now open browser
- Notifications are now handled within the app/system accordingly to the app's minimized state

For developers:
- Now using React and TypeScript
- Switched from npm to yarn: electron-userland/electron-builder#1147 (comment)
- Updated Continuous Delivery

Contributors: @rikoopa @Jamiexhz @Silvaburn#3669
@ppoliani
Copy link

Has this been sorted? I couldn't make it the codesign complete

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants