Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Yarn-pnp, esy, and npm-link #3276

Open
jordwalke opened this issue Jan 13, 2019 · 2 comments

Comments

@jordwalke
Copy link

commented Jan 13, 2019

The following workflows and package managers don't work when tools build into node_modules directories, or rely on dependencies existing in node_modules.

Because of this, developers building native apps with esy cannot use bs-platform at all. They have very frequently requested the ability to use esy to build bs apps, but there's not much esy can do at this point, and the solution to esy is the same as the solution to yarn pnp and npm link, so might as well solve all three at once.

  1. Yarn Plug'n'play. (Stores dependencies in global cache).
  2. esy used to manage JS dependencies (uses yarn plugin'n'play standard).
  3. Npm link workflows. Two packages can link to the same dependency, with different build flags so putting build artifacts into that dependency's root will have artifacts conflict with each other).

Here's what I believe needs to happen to make bs-platform work with all three of these. The solutions are largely the same for all use cases:

Stop Building Into Dependencies' node_modules

Solves Problems with: Yarn pnp, Esy pnp.

The idea here is that all libraries should be built into the top level project's root directory. Separate Tracking Issue

Don't assume dependencies even exist in node_modules.

Solves Problems with: Yarn pnp, Esy pnp.

Since esy and yarn both never touch the disk if it can read the dependencies from an immutable cache, we can't assume the packages are located at node_modules. Instead yarn pnp generates a .pnp.js and esy generates a ./_esy/default/pnp.js file. That tells you the real locations of dependencies in the immutable cache. Esy uses the exact same pnp standard, but we can make some changes if it makes it easier for you.

Optimization: Yarn pnp requires starting up a JS VM to determine the location of dependencies. I believe we can make esy's pnp.js file carefully constructed so that bsb could just read the locations directly from the text file by parsing simple strings instead of starting a JS VM. It would also work if you started it up with a JS VM but we can overlay an additional protocol on top of yarn pnp for the sake of additional performance.

Avoid postinstall for bs-platform's installation.

Solves Problems with: Yarn pnp.
Esy and npm can still tolerate a postinstall step for bs-platform, but yarn pnp cannot. It's better to avoid it if possible so that even yarn pnp can work well. If not, then at least esy pnp and npm link workflows would still work.

Model compiler explicitly:

This one is only relevant for esy. But if there is an esy.json file alongside package.json for bs-platform, which expresses the dependency on the ocaml compiler it uses, and its version, then esy could be used to install native ocaml ppx's for BuckleScript projects.

@texastoland

This comment has been minimized.

Copy link

commented May 9, 2019

Keeping notes as I look. AFAIK the main change needs to happen in Bsb_pkg. Every other reference to node_modules is either a template or a test. Also relevant is @bsansouci's stale (what happened?) #2514.

@texastoland

This comment has been minimized.

Copy link

commented May 9, 2019

I believe we can make esy's pnp.js file carefully constructed so that bsb could just read the locations directly from the text file by parsing simple strings instead of starting a JS VM.

The manifest file appears resolved by yarnpkg/yarn#6388 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.