Pre-builds for more platforms / architectures

[geekgonecrazy]

Is your feature request related to a problem? Please describe.
Looks like currently there are pre-builds happening for linux via:
https://github.com/matrix-org/matrix-rust-sdk-bindings/blob/main/crates/matrix-sdk-crypto-nodejs/release/Dockerfile.linux#L34

But for others it utilizes a post install hook:
https://github.com/matrix-org/matrix-rust-sdk-bindings/blob/main/crates/matrix-sdk-crypto-nodejs/check-exists.js#L7

Which then means you need to have rust, cargo and numerous other dependencies installed.

Describe the solution you'd like

Looks like the current CI builds for Mac and linux. I'd propose utilizing the CI instead to do the build of the artifacts.

The hard part is need to then release to npm from the CI. Or.. upload as artifacts and make the post install fetch from those build artifacts if not present.

Honestly not sure on the best solution.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context

In my particular case. The check-exists fails regardless because we utilize yarn workspaces. Something about having yarn workspace plugin configured causes the yarn build:release from inside the check-exists.js to fail with an error like:

# This file contains the result of Yarn building a package (@turt2live/matrix-sdk-crypto-nodejs@npm:0.1.0-beta.10)
# Script name: postinstall

�[31m�[1mInternal Error�[22m�[39m: @turt2live/matrix-sdk-crypto-nodejs@workspace:.: This package doesn't seem to be present in your lockfile; run "yarn install" to update the lockfile
    at tF.getCandidates (/Users/aaron/go/src/github.com/RocketChat/Rocket.Chat/.yarn/releases/yarn-3.2.0.cjs:437:4480)
    at wd.getCandidates (/Users/aaron/go/src/github.com/RocketChat/Rocket.Chat/.yarn/releases/yarn-3.2.0.cjs:395:1281)
    at /Users/aaron/go/src/github.com/RocketChat/Rocket.Chat/.yarn/releases/yarn-3.2.0.cjs:441:7765
    at Pg (/Users/aaron/go/src/github.com/RocketChat/Rocket.Chat/.yarn/releases/yarn-3.2.0.cjs:394:11098)
    at le (/Users/aaron/go/src/github.com/RocketChat/Rocket.Chat/.yarn/releases/yarn-3.2.0.cjs:441:7745)

you can reproduce by cloning: https://github.com/RocketChat/Rocket.Chat on a Mac and doing yarn add matrix-appservice-bridge or directly this package yarn add @turt2live/matrix-sdk-crypto-nodejs.

I believe its related to yarn workspaces as if I copy the .yarn folder and .yarnrc.yml to a new folder and try the same error happens.

Current work around is go into node_modules/@turt2live/matrix-sdk-crypto-nodejs and run npm run build:debug or npm run build:release

[geekgonecrazy]

https://github.com/lovell/sharp/blob/main/install/libvips.js sharp is an interesting one where the script actually fetches from GitHub Releases. Its also rust with some libc requirements

Here is where they actually upload the artifact when tagged: https://github.com/lovell/sharp/blob/main/.github/workflows/ci.yml#L97

[geekgonecrazy]

On m1 Mac we needed to do some additional steps:

brew install lib-olm
export OLM_LINK_VARIANT=dylib

cd node_modules/@turt2live/matrix-sdk-crypto-nodejs
yarn
yarn build:darwin:x64
mv lib/index.node lib/index.node-bak

With out doing this.. this happens:

/Users/douglasgubert/dev/rocket.chat/Rocket.Chat-2/apps/meteor/node_modules/@turt2live/matrix-sdk-crypto-nodejs/check-exists.js:15
        throw e;
        ^

Error: dlopen(/Users/douglasgubert/dev/rocket.chat/Rocket.Chat-2/apps/meteor/node_modules/@turt2live/matrix-sdk-crypto-nodejs/lib/index.node, 1): no suitable image found.  Did find:
	/Users/douglasgubert/dev/rocket.chat/Rocket.Chat-2/apps/meteor/node_modules/@turt2live/matrix-sdk-crypto-nodejs/lib/index.node: mach-o, but wrong architecture
	/Users/douglasgubert/dev/rocket.chat/Rocket.Chat-2/apps/meteor/node_modules/@turt2live/matrix-sdk-crypto-nodejs/lib/index.node: mach-o, but wrong architecture
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1144:18)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:93:18)
    at Object.<anonymous> (/Users/douglasgubert/dev/rocket.chat/Rocket.Chat-2/apps/meteor/node_modules/@turt2live/matrix-sdk-crypto-nodejs/lib/napi.js:21:11)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:12) {
  code: 'ERR_DLOPEN_FAILED'
}

Guessing something to do with node being x86_64 vs node module detecting and compiling the arm64

Oddly though even with: yarn build:darwin:x64 we had to mv the index.node so this condition is hit:

https://github.com/matrix-org/matrix-rust-sdk-bindings/blob/main/crates/matrix-sdk-crypto-nodejs/src/ts/napi.ts#L23

which I guess is generated at build time seemed to analyze appropriate architecture and load in the right one

[geekgonecrazy]

Looks like #5 and #6 are basically same root cause as this

[CallMeLaNN]

I just use old Mac and running yarn add matrix-appservice-bridge failed due to the same error.

I tried on x86 and M1 mac. both are the same. Tried on old yarn v1 it is ok.

The exact issue is this package with yarn v2 and above.

# This file contains the result of Yarn building a package (@turt2live/matrix-sdk-crypto-nodejs@npm:0.1.0-beta.10)
# Script name: postinstall

Internal Error: @turt2live/matrix-sdk-crypto-nodejs@workspace:.: This package doesn't seem to be present in your lockfile; run "yarn install" to update the lockfile
    at AF.getCandidates (/Users/aaa/Documents/code/bbb/.yarn/releases/yarn-3.1.0.cjs:437:4421)
    at pd.getCandidates (/Users/aaa/Documents/code/bbb/.yarn/releases/yarn-3.1.0.cjs:432:5275)
    at /Users/aaa/Documents/code/bbb/.yarn/releases/yarn-3.1.0.cjs:441:7674
    at du (/Users/aaa/Documents/code/bbb/.yarn/releases/yarn-3.1.0.cjs:382:4003)
    at R (/Users/aaa/Documents/code/bbb/.yarn/releases/yarn-3.1.0.cjs:441:7654)