support Electron: respect process.env.npm_config_arch & process.env.npm_config_platform

[mifi]

Because in v4 it only downloads the binary for the current operating system arch, how can I now build an electron app that includes the correct ffmpeg binary according to which target platform I am building for?

For example if I want to run electron-builder --mac --win --linux

Before v4, I would just include the correct binary from the right arch directory when building like this:

electron-builder package.json "build":

    "mac": {
...
      "extraResources": [
        "node_modules/ffmpeg-static/bin/darwin/**",
        "node_modules/ffprobe-static/bin/darwin/**"
      ],
    },
    "win": {
...
      "extraResources": [
        "node_modules/ffmpeg-static/bin/win32/x64/**",
        "node_modules/ffprobe-static/bin/win32/x64/**"
      ],
    },

See
https://github.com/mifi/lossless-cut/blob/74439d716db5ff550a1851fa5b58e30ba188c477/package.json#L108

But now this doesn't work. Any tips of how to cleanly solve this? Or isn't electron really supported by this module anymore?

[derhuerst]

Bundling the binaries with the npm package gradually became too clunky, as outlined in #20 and #33. This breaks Electron for now, sorry. (Keep in mind that the previous hacks were helpful but not very future-proof.)

---

First, let me put the problem in the perspective of ffmpeg-static:

The new download-based approach is a problem whenever you want to package ffmpeg-static into a platform-independent or per-platform "distributable"; Electron apps are likely the most common case, but .zip archives are another.

I don't see this a problem of `ffmpeg-static though. Developers need to figure this out in their build scripts anyways whenever they use

• a native dependency (which needs to be compiled for the/each platform/arch), or
• a dependency with a platform-/arch-specific install script (quite a few npm packages), or
• other platform-/arch-specific assets, e.g. images.

Therefore, developers should configure their build tooling to set up the right environment for these dependencies to build (themselves) correctly.

---

I don't have any experience with building Electron apps. It seems like it has a mechanism for this:

Digging into the electron-builder (from electron-builder's install-app-deps command via installOrRebuild via installDependencies to getGypEnv), I stumbled upon the script for installing Electron via npm and the Using Native Node Modules docs.

They which mention the npm_config_platform, npm_config_arch & npm_config_target_arch env vars. It seems like we should respect them here in ffmpeg-static as well.

@mifi Would you mind to add the env vars to install.js in your fork of ffmpeg-static and try it with your electron-builder setup?

[pietrop]

👋 this comment pietrop#1 (comment) and blog post explains how I dealt with making static ffmpeg work with electron build process. Sharing it in case it helps.

[derhuerst]

👋 this comment pietrop#1 (comment) and blog post explains how I dealt with making static ffmpeg work with electron build process. Sharing it in case it helps.

Maybe I misunderstood it, but It's not as zero-config as I'd like it to be:

• you have to manually add a disallow-list to your electron config
• you have to know about every large file in the ffmpeg-static package.

[pietrop]

Yeah, fair, at the moment is not zero-config but it works.

The main problem is if you want to build for mac, linux, and windows on OSX (I do that on travis CI) then you'd still need all 3 binaries to be able to do the build, but you'd just want to package only the one you need for each distribution to keep the package size contained etc..

Hope this use case helps.

I do agree that having binaries inside a repo is not ideal, and perhaps there could be a way to pull those down eg from github releases section and add them to the module this way as part of a pre installation step. But not sure if that's a possibility?

[mifi]

It seems that install.js isn't being run when I run a complete build with electron-builder (even for a different OS than my current). I think electron-builder will run node-gyp for all the native dependencies, maybe ffmpeg-static needs to hook into that somehow and do the downloading of the correct binary instead of building

[derhuerst]

I do agree that having binaries inside a repo is not ideal, and perhaps there could be a way to pull those down eg from github releases section and add them to the module this way as part of a pre installation step. But not sure if that's a possibility?

That is the case right now: The install script defined in package.json downloads the binary for the current platform, according to os.platform() & os.arch().

It seems that install.js isn't being run when I run a complete build with electron-builder (even for a different OS than my current).

Again, I don't know Electron well, but I think running the install script is the job of any tool that installs (and builds) dependencies.

But downloading the intended binaries for a different platform than the current one seems to be the crucial point here.

You could help me by

1. setting up a clean electron-builder-based project,
2. adding any npm dependency (e.g. lodash),
3. add an install script to it inside node_modules, that
4. prints npm_config_platform,npm_config_arch & npm_config_target_arch env vars, and
5. run electron-builder to find out if it sets these env vars.

[mifi]

That’s Essentially what i already did. I tried to modify install.js in ffmpeg-static in node_modules and add console.log, then run electron-builder, but it was never being executed. The install script only gets executed when running npm install / yarn. Electron-builder never runs this afaik. I think wwhat electron builder does is to take modules from node_modules as is and pacakge them up. But first it will run node-gyp to rebuild any binary dependencies inside node_modules to the target arch. If ffmpeg-static was using node-gyp to compile ffmpeg then i think it would just work. So i think if there is some way we can hook into node gyp and make it run the install.js script, that might be a way to do it. But i know next to nothing about node gyp.

[derhuerst]

The install script only gets executed when running npm install / yarn. Electron-builder never runs this afaik. [...] If ffmpeg-static was using node-gyp to compile ffmpeg then i think it would just work.

Sorry, I'm not going to do that:

• The preinstall/install/postinstall scripts are the right place for npm packages to declare that they need to do work to "become functional". If I'm not missing something, electron-builder should add support for getting the right "variant" of node_modules.
• Abusing mechanisms intended for different things adds a long-term maintenance burden. This mechanism might change in the future.

[derhuerst]

Because in v4 it only downloads the binary for the current operating system arch, how can I now build an electron app that includes the correct ffmpeg binary according to which target platform I am building for?

Keep in mind that the electron-builder docs explicitly mention:

Don’t expect that you can build app for all platforms on one platform.
If your app has native dependency, it can be compiled only on the target platform unless prebuild is not used.

prebuild is a solution, but most node modules don’t provide prebuilt binaries.

electron-builder mentions prebuild & prebuild-install as working though. prebuild-install latter respects process.env.npm_config_arch & process.env.npm_config_platform.

PR welcome!

[mifi]

Yea, I just tested adding the following scripts in node_modules/ffmpeg-static/package.json and then running electron-builder, but none of the scripts gets run. So it seems electron just uses whatever node_modules is already there.

  "scripts": {
    "prebuild": "echo prebuild && exit 1",
    "build": "echo build && exit 1",
    "prerebuild": "echo prerebuild && exit 1",  
    "rebuild": "echo rebuild && exit 1",  
    "preinstall": "echo preinstall && exit 1",
    "install": "echo install && exit 1",
    "postinstall": "echo postinstall && exit 1",
  }

I think you are right that prebuild is the right way to do it. I will instead just rely on GitHub actions for my project which builds on separate OS images for win, mac and linux. I believe all major build systems support this. Only drawback with this is that there is no easy way to build non-open source electron apps using ffmpeg-static without paying monthly for a build service.

[derhuerst]

I think you are right that prebuild is the right way to do it.

No, i'm not trying to say that!! I don't want to use prebuild because there's no need.

All I want to do is change install.js to respect process.env.npm_config_arch & process.env.npm_config_platform.

[derhuerst]

Only drawback with this is that there is no easy way to build non-open source electron apps using ffmpeg-static without paying monthly for a build service.

As I said, this is not only the case with ffmpeg-static, but – AFAICT – any other native dependency that you use, e.g. some crypto packages, buffer-util/utf8-validate for WebSocket performance, node-sass, etc.

[pietrop]

Only drawback with this is that there is no easy way to build non-open source electron apps using ffmpeg-static without paying monthly for a build service.

What about Travis CI? I’ve used that in a couple of projects (that use ffmpeg).
If you choose a Mac OS X image you can do mac, win and Linux and publish to github releases of your github repo

[mifi]

No, i'm not trying to say that!! I don't want to use prebuild because there's no need.

All I want to do is change install.js to respect process.env.npm_config_arch & process.env.npm_config_platform.

I was thinking that ffmpeg-static could publish prebuild binaries using prebuild, kind of like how other packages are doing, like drivelist for example. But I've never used prebuild so I don't know if that is possible with downloaded binaries like ffmpeg-static does.

All I want to do is change install.js to respect process.env.npm_config_arch & process.env.npm_config_platform.

I'm not sure if I understand how that will help when we don't know how we can make install.js run during electron-builder's build phase.

What about Travis CI? I’ve used that in a couple of projects (that use ffmpeg).
If you choose a Mac OS X image you can do mac, win and Linux and publish to github releases of your github repo

Yea Travis CI or github actions or similar will work, but afaik it's only free for open source projects. Which is fine in my case for LosslessCut but for proprietary software / private repos they typically requires you to pay a subscription.

[derhuerst]

I was thinking that ffmpeg-static could publish prebuild binaries using prebuild, kind of like how other packages are doing, like drivelist for example. But I've never used prebuild so I don't know if that is possible with downloaded binaries like ffmpeg-static does.

ffmpeg-static and drivelist are different: ffmpeg-static merely downloads existing ffmpeg binaries; It does not do native calls to ffmpeg or build it. drivelist has its own native code, and needs to be built, and prebuild makes a lot of sense for that.

One might be able to trick prebuild into using ready-made ffmpeg binaries (instead of building & uploading native code, as intended); But as I said, IMO it's definitely not worth the complexity & maintenance burden. The current install script works fine; What we're discussing here is a "problem" in the Electron tooling.

As I said several times: Let's respect process.env.npm_config_arch & process.env.npm_config_platform, but then the Electron tooling needs to be adapted to install dependencies with the right env vars.

[RavisMsk]

I stumbled on the same problem and researched electron-builder source code to handle this problem. It actually can be handled without fixing electron-builder, but adding process.env.npm_config* as prioritised source is required.
Electron-builder runs dependencies rebuild/installation during build, but performs actual install only if deps are not installed.
I made a fork #47 , which uses npm env-variables as prioritised source of platform/arch.
And modified packaging commands to remove node_modules of app that's using ffmpeg-static:

"package-mac": "rm -rf ./app/node_modules && yarn build && electron-builder build --mac",
"package-linux": "rm -rf ./app/node_modules && yarn build && electron-builder build --linux",
"package-win": "rm -rf ./app/node_modules && yarn build && electron-builder build --win --x64",

While building-packaing two-layered app (with 2 package.json files as in this boilerplate) with this setup electron-builder prints:

...
  • installing production dependencies  platform=win32 arch=x64 appDir=/Users/username/projects/test/app
...

And newly installed node_modules contain correct binary (built for windows on mac):

file ./app/node_modules/ffmpeg-static/ffmpeg
./app/node_modules/ffmpeg-static/ffmpeg: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows

I think electron-builder build -mwl won't work with this setup (I'm packaging one-by-one), but maybe same effect could be achieved by using beforeBuild hook to remove node_modules.

[RavisMsk]

Had time to test the resulting package on windows just now. For some reason ffmpeg binary doesn't have exe extension and can't be used. Adding the extension manually resolves the problem.

This commit seems to resolve the issue without manual interventions.

[derhuerst]

Thanks @RavisMsk for investigating this! It's nice to hear that your findings are in line with what I found out earlier.

[derhuerst]

ffmpeg-static@4.2.2 includes #47, which should allow you to cross-platform-package Electron apps with the mentioned rm -rf node_modules workaround. I have created #48 to make this experience better.

I will close this ticket. Please re-open if there's a bug with @RavisMsk's logic that I have just published.

[pietrop]

In my current setup I use electron builder on TravisCI to build for Mac, Linux and Windows on a TravisCI OSX image. It then publishes to GitHub release.

The recent change would not work in this “edge” case, right? Coz it would just use the OS of the platform / npm etc..? Or am I missing something?

As mentioned in previous comment I already have a workaround to package the right ffmpeg bin a part of the electron build process
#35 (comment)

But wanted to check if this latest change, changes anything in this context. If it doesn’t no worries.

[derhuerst]

@pietrop Cross-platform-packaging Electron apps should just work with ffmpeg-static@4.2.2:

And modified packaging commands to remove node_modules of app that's using ffmpeg-static:

"package-mac": "rm -rf ./app/node_modules && yarn build && electron-builder build --mac",
"package-linux": "rm -rf ./app/node_modules && yarn build && electron-builder build --linux",
"package-win": "rm -rf ./app/node_modules && yarn build && electron-builder build --win --x64",

In the macOS Travis build, do rm -rf node_modules each time before you call electron-builder.

[pietrop]

Awesome, thanks will try it out, at the moment I do all 3 in one command but I guess I could split it

[delewis13]

@RavisMsk i'm a little confused how your commands listed are working O.o Given you delete all your node_modules, how does yarn build [which in turn looking @ https://github.com/electron-react-boilerplate/electron-react-boilerplate/blob/main/package.json runs webpack], actually work, given you just deleted webpack and all ur other deps?

I'm trying to get this to work on my end, but if I rm -rf ./app/node_modules && electron-builder build --win --x64, the build fails as electron-builder node_module was deleted... So I need to yarn install everything which gets me back to square one, given that the necessary environment variables aren't defined at that point