Cross platform / architecture install option

[wereHamster]

• I'd be willing to implement this feature (contributing guide)

Describe the user story

npm install offers option to select a different platform/architecture. This is useful for example in case packages have a post-install script that fetches arch-specific binaries (such as sharp).

See https://sharp.pixelplumbing.com/install#cross-platform for an example.

Describe the solution you'd like

pnpm install --platform=linux --architecture=x64

Describe the drawbacks of your solution

I don't think there are any drawbacks. NPM has implemented this, but may have taken some shortcuts that I am not aware of and pnpm project isn't willing to take.

Describe alternatives you've considered

Spinning up a machine (or docker container) with that architecture and run pnpm install in that. But that's a rather heavyweight solution to this problem.

[zkochan]

Yarn allows to use an array of supported architectures: https://yarnpkg.com/configuration/yarnrc/#supportedArchitectures

Yarn's approach looks better to me.

[wereHamster]

How does yarn's supportedArchitectures interact with packages which use a post-install script to download arch-specific binaries (such as sharp does).

[keithlayne]

☝️ That's my question too - I suspect that it only really works for os/arch-specific npms, but I haven't had a chance to try. I'd love it if postinstall scripts went away, but that's not the current reality. I have a repo with packages that have different targets...maybe I'm wrong and the package.json just needs cpu/arch set. Seems like a complex thing.

[zkochan]

The package manager don't have control over what happens in the postinstall script.

All that these flags can do it (the Yarn one at least) is deciding which optional dependencies are fetched.

If these flags are used by postinstall script then you just need to run this:

pnpm install --config.platform=linux --config.architecture=x64

[apaniel]

@zkochan if would implement the same supported architectures feature as a list like yarn has, where could I start looking inside pnpm?

[zkochan]

Start with the package-is-installable project.

[vzrenggamani]

Any update regarding this issue ? Just got dumped here with same problem :)

[stabai]

Just found this as a result of trying to get esbuild to work on both my mac dev machine and my docker dev container without having to reinstall everything. Being able to install things for both linux and darwin at once would be amazing. The esbuild error message specifically mentions Yarn's built-in support for installing a package on multiple
platforms simultaneously and its supportedArchitectures config as solutions.

[sfc-gh-ptabor]

Proposed fix to rules_js to pass arch and platform to pnpm by default from bazel builds.

I hope we can consider passing: --config.platform=linux --config.architecture=x64 a recommended pattern for cross-compilation when needed:

aspect-build/rules_js#1226

[spaceemotion]

Just came across the same issue trying to run https://github.com/Brooooooklyn/snappy on lambda. I can't get the correct dependencies installed on Github Actions. Seems like the --config.platform trick should work (as it apparently does with sharp), but it doesn't.

[sfc-gh-ptabor]

@spaceemotion
It depends whether the lifecycle hooks access the standard npm flags.

E.g. does it as follows:
https://github.com/lovell/sharp/blob/9c217ab580123ee14ad65d5043d74d8ea7c245e5/lib/platform.js#L11

  const arch = env.npm_config_arch || process.arch;
  const platform = env.npm_config_platform || process.platform;

This seems to be a common set of variables as: https://yarnpkg.com/package?name=prebuild-install says:

When prebuild-install is run via an npm script, options --build-from-source, --debug, --download, --target, --runtime, --arch --platform and --libc may be passed through via arguments given to the npm command.

Alternatively you can set environment variables npm_config_build_from_source=true, npm_config_platform, npm_config_arch, npm_config_target npm_config_runtime and npm_config_libc.

I don't know snappy... nor napi but from quick look the --platform seems to remain unconfigured:
https://github.com/Brooooooklyn/snappy/blob/be0f4aa70ddb1b812a22fce154c33291fb76d1cf/package.json#L56C4-L56C12

[spaceemotion]

@sfc-gh-ptabor Thanks for the clarifications! It actually ended up being that I had --no-optional enabled. As the binaries seemed to be optional things, pnpm didn't include them.

[sojimaxi]

There is still a serious problem with this: (Windows)

I already have the windows-64 installed in node_modules but every time I want to add a new package or run install I needs to download all these unnecessary options which will never be used.

Is there not global config we can set to install from because using --config.platform=win --config.architecture=x64 didn't work for me.

Here is the full download.

Could there be a better solution?

Related to vercel/turborepo#2517