Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto-install Rust compiler toolchain locally for CI and JS devs of other projects #1373

Merged
merged 2 commits into from
Feb 5, 2024

Conversation

peaBerberian
Copy link
Collaborator

@peaBerberian peaBerberian commented Feb 2, 2024

Since we brought the WebAssembly MPD parser to fix performance issues we had with some contents at canal+, we added another language (Rust) to the project which is not compatible with most of the JS-world tools we and other JS developers usually deal with.

Now that we added the MULTI_THREAD experimental feature and embeds, the WebAssembly generation from that other language is even a required step in our main build script (npm run build).

Consequently, when another developer or a CI needs to build an RxPlayer (e.g. to test a development before a release is made), they have to install:

  1. NodeJS + npm
  2. our JavaScript dependencies
  3. A Rust toolchain able to output WebAssembly. We generally go through rustup here basically an helper tool to install Rust toolchains
  4. binaryen, which is a collection of tools to transform that WebAssembly file.

(1) and (2) are very common use cases that both JS people and CI workflows do very easily: they mostly already have nodejs + npm installed locally (or it's very simple to set-up) and the second is only an npm install call away.

(3) and (4) however, though not hard (you have to install rustup and binaryen which is pretty simple on all platforms we relied on), are more unusual dependencies that JS developers might be reticent to install and that are more work to set-up on a CI (it's fine on ours, but we also sometimes want other projects, with their own CI, to test our developments for their use cases).

Consequently I'm doing here a proof-of-concept where those tools are auto-installed locally if not found when doing an npm run build call, letting both people and CI produce a build easily without having to deal with the dual-language complexity.


For rustup it's pretty straightforward as their install script already take care of the platform detection, WebAssembly target installation and so on. All I had to do is to make sure binaries were installed locally (./tmp) and did not pollute the developers' environment (e.g. no PATH modification).

For binaryen however, it's less simple as they don't propose such auto-install system.
However, I noticed that they propose prebuilt binaries for the most usual platforms on their github's release page:
https://github.com/WebAssembly/binaryen/releases

The idea here (not yet finished), is to automatically detect the platform of the user and fetch the right binaryen binary locally to then rely on both it and the local rustup to produce builds in case the command isn't available globally.

Also, because it will be faster and less error-prone for frequent developers to install both of those globally once and for all, we print a notice explaining what we do when that auto-install script is run.

Since we brought the WebAssembly MPD parser to fix performance issues we
had with some contents at canal+, we added another language (Rust) to
the project which is not compatible with most of the JS-world tools we and
other JS developers usually deal with.

Now that we added the `MULTI_THREAD` experimental feature and embeds,
the WebAssembly generation from that other language is even a required
step in our main build script (`npm run build`).

Consequently, when another developer or a CI needs to build an RxPlayer
(e.g. to test a development before a release is made), they have to
install:
  1. NodeJS + npm
  2. our JavaScript dependencies
  3. A Rust toolchain able to output WebAssembly. We generally go
     through `rustup` here basically an helper tool to install Rust
     toolchains
  4. `binaryen`, which is a collection of tools to transform that
     WebAssembly file.

(1) and (2) are very common use cases that both JS people and CI workflows
do very easily: they mostly already have nodejs + npm installed locally
(or it's very simple to set-up) and the second is only an `npm install`
call away.

(3) and (4) however, though not hard (you have to install rustup and
binaryen which is pretty simple on all platforms we relied on), are more
unusual dependencies that JS developers might be reticent to install and
that are more work to set-up on a CI.

Consequently I'm doing here a proof-of-concept where those tools are
auto-installed locally if not found when doing an `npm run build` call,
letting both people and CI produce a build easily without having to deal
with the dual-language complexity.

---

For rustup it's pretty straightforward as their install script already
take care of the platform detection, target installation and so on. All
I had to do is to make sure binaries were installed locally (`./tmp`) and
did not pollute the developers environment (e.g. no `PATH`
modification).

For binaryen however, it's less simple as they don't propose such
auto-install system.
however, I noticed that they propose prebuilt binaries for the most usual
platforms on their github's release page:
https://github.com/WebAssembly/binaryen/releases

The idea here (not yet finished), is to automatically detect the
platform of the user and fetch the right binaryen binary locally to then
rely on both it and the local rustup to produce builds in case the
command isn't available globally.

Also, because it will be faster and less error-prone for frequent
developers to install both of those globally once and for all, we print a
notice explaining what we do when that auto-install script is run.
Since we brought the WebAssembly MPD parser to fix performance issues we
had with some contents at canal+, we added another language (Rust) to
the project which is not compatible with most of the JS-world tools we and
other JS developers usually deal with.

Now that we added the `MULTI_THREAD` experimental feature and embeds,
the WebAssembly generation from that other language is even a required
step in our main build script (`npm run build`).

Consequently, when another developer or a CI needs to build an RxPlayer
(e.g. to test a development before a release is made), they have to
install:
  1. NodeJS + npm
  2. our JavaScript dependencies
  3. A Rust toolchain able to output WebAssembly. We generally go
     through `rustup` here basically an helper tool to install Rust
     toolchains
  4. `binaryen`, which is a collection of tools to transform that
     WebAssembly file.

(1) and (2) are very common use cases that both JS people and CI workflows
do very easily: they mostly already have nodejs + npm installed locally
(or it's very simple to set-up) and the second is only an `npm install`
call away.

(3) and (4) however, though not hard (you have to install rustup and
binaryen which is pretty simple on all platforms we relied on), are more
unusual dependencies that JS developers might be reticent to install and
that are more work to set-up on a CI.

Consequently I'm doing here a proof-of-concept where those tools are
auto-installed locally if not found when doing an `npm run build` call,
letting both people and CI produce a build easily without having to deal
with the dual-language complexity.

---

For rustup it's pretty straightforward as their install script already
take care of the platform detection, target installation and so on. All
I had to do is to make sure binaries were installed locally (`./tmp`) and
did not pollute the developers environment (e.g. no `PATH`
modification).

For binaryen however, it's less simple as they don't propose such
auto-install system.
however, I noticed that they propose prebuilt binaries for the most usual
platforms on their github's release page:
https://github.com/WebAssembly/binaryen/releases

The idea here (not yet finished), is to automatically detect the
platform of the user and fetch the right binaryen binary locally to then
rely on both it and the local rustup to produce builds in case the
command isn't available globally.

Also, because it will be faster and less error-prone for frequent
developers to install both of those globally once and for all, we print a
notice explaining what we do when that auto-install script is run.
@peaBerberian peaBerberian added Priority: 0 (Very high) This issue or PR has a very high priority. Efforts should be concentrated on it first. and removed work-in-progress This Pull Request or issue is not finished yet labels Feb 5, 2024
@peaBerberian peaBerberian changed the title [WIP] Auto-install Rust compiler toolchain locally for CI and JS devs of other projects Auto-install Rust compiler toolchain locally for CI and JS devs of other projects Feb 5, 2024
@peaBerberian peaBerberian mentioned this pull request Feb 5, 2024
Copy link
Collaborator

@Florent-Bouisset Florent-Bouisset left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@peaBerberian peaBerberian merged commit fee73d0 into stable Feb 5, 2024
5 checks passed
@peaBerberian peaBerberian added this to the 4.0.0-rc.2 milestone Feb 5, 2024
@peaBerberian peaBerberian deleted the misc/auto-install-script branch February 7, 2024 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority: 0 (Very high) This issue or PR has a very high priority. Efforts should be concentrated on it first.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants