Yarn v2 / PnP Support

[rwjblue]

Yarn 1.12 added support for a new "Plug 'n Play" feature by way of an --pnp option. This issue is meant to track adding support for ember new foo --yarn to support using --pnp without issue.

Further Reading:

• Yarn Plug'n'Play: Getting rid of node_modules yarnpkg/rfcs#101
• https://twitter.com/yarnpkg/status/1057632564217704453
• https://gist.github.com/arcanis/02b49752c385908479734d8027d7a6c7

Current known issues:

• Need to remove the node_modules appears empty, you may need to run 'yarn install' warning when .pnp.js is present
• PackageInfoCache has significant issues (currently emits tons of warnings, since all of the structures that it expects to exist are missing), and may not be relevant at all in a PnP app. Needs further research...
• Update ember-try to handle this nicely Yarn PnP Support? ember-try#214
• Updated version of resolve including the changes from Implements useProcessResolution browserify/resolve#170 Adds support for "paths" being a function browserify/resolve#172, Implements skipNodeModules browserify/resolve#173. and Implements a "normalize-options" pseudo-hook browserify/resolve#174 (to ensure resolve.sync usage is aware of PnP)
• Update ember-cli-dependency-checker Yarn PnP Support quaertym/ember-cli-dependency-checker#92
• Update ember-cli-version-checker to address Yarn PnP Support ember-cli-version-checker#78

[drummy2]

I need this so bad. Still looks quite far away :(

[musaffa]

@rwjblue Hi, it has been a while. Are you still pursuing this feature? What are the main blockers?

[bartocc]

I am experimenting with the PnP API to make ember-cli compatible with Yarn PnP.

I might open a draft PR at some point to show the progress and get ideas from the community

[sesam]

How is Yarn PnP compatibility going?

[bartocc]

@sesam for the time being, I use https://github.com/yarnpkg/berry/tree/master/packages/plugin-node-modules

@ef4, do you believe embroider will change things regarding yarn PNP ?

[gilest]

Not sure if I should post this here or make another issue but ember install is incompatible too.

Seems like v2 of the yarn CLI uses -i or --interactive rather than --non-interactive.

ember install ember-file-upload

🚧  Installing packages... This might take a couple of minutes.
Command failed: yarn add --dev ember-file-upload --non-interactive

Unknown Syntax Error: Command not found; did you mean one of:

  0. yarn add [--json] [-E,--exact] [-T,--tilde] [-C,--caret] [-D,--dev] [-P,--peer] [-O,--optional] [--prefer-dev] [-i,--interactive] [--cached] ...
  1. yarn add [--json] [-E,--exact] [-T,--tilde] [-C,--caret] [-D,--dev] [-P,--peer] [-O,--optional] [--prefer-dev] [-i,--interactive] [--cached] ...

While running add --dev ember-file-upload --non-interactive

[ef4]

I spent some time trying to test embroider under PNP and it seems to be blocked on issues within ember-cli itself.

For example, ember-cli/lib/broccoli/ember-app.js is trying to load jquery from bower, because it fails to detect the @ember/jquery and ember-source addons. @embroider/compat dutifully tries to follow along and find jquery from bower, which doesn't exist so it blows up.

I also hit stefanpenner/resolve-package-path#24

The process of debugging this stuff is fairly painful. yarn 2 seems... not really quite done yet. Just getting it to execute ember-cli inside the node debugger was an adventure, requiring hacks. And I still have not managed to get yarn linking of the embroider monorepo's packages into a test app working properly.

[chrisvdp]

@ef4 I have had similar experiences trying to use yarn 2. For what it's worth, PnP was introduced in yarn classic. You might have a better time testing with that https://classic.yarnpkg.com/en/docs/pnp

[bartocc]

@ef4 I'd like to know what hack you had to use to execute ember-cli inside the node debugger if you have the time to explain. I have tried to use the yarn run --inspect-brk option listed at https://yarnpkg.com/cli/run, but without luck.

@chrisvdp I had missed that PnP was actually introduced under yarn 1. Interesting!

[ef4]

@bartocc I had to make a shim program that I could launch with node like:

// launch.js
require('ember-cli/bin/ember')

Then something like:

node --inspect-brk --require /Path/To/Your/.pnp.js ./launch.js

[bartocc]

thx @ef4

[chrisvdp]

FWIW, yarn 2.1 is out now and seems better.

[drummy2]

Anymore news? npm installs are so bad for all of the reasons listed here https://yarnpkg.com/features/pnp
Allowing PnP with Ember would help massively for the CI jobs.

[Alonski]

@rwjblue @ef4 @NullVoxPopuli Pinging to see if we can push this forward again :)

[Turbo87]

FWIW it seems that https://pnpm.io is a much easier way to achieve the same goals and aside from ember-cli not supporting it officially yet it works quite well :)

[Alonski]

@Turbo87 Do you have any tips/guide on moving from Yarn Workspaces to pnpm?

[Turbo87]

can't say anything about workspaces since I rarely use those, but AFAIK they are supported by pnpm in some way too

[shamrt]

Not sure if I should post this here or make another issue but ember install is incompatible too.

Seems like v2 of the yarn CLI uses -i or --interactive rather than --non-interactive.

ember install ember-file-upload

🚧  Installing packages... This might take a couple of minutes.
Command failed: yarn add --dev ember-file-upload --non-interactive

Unknown Syntax Error: Command not found; did you mean one of:

  0. yarn add [--json] [-E,--exact] [-T,--tilde] [-C,--caret] [-D,--dev] [-P,--peer] [-O,--optional] [--prefer-dev] [-i,--interactive] [--cached] ...
  1. yarn add [--json] [-E,--exact] [-T,--tilde] [-C,--caret] [-D,--dev] [-P,--peer] [-O,--optional] [--prefer-dev] [-i,--interactive] [--cached] ...

While running add --dev ember-file-upload --non-interactive

For anyone experiencing this bugaboo, I've created a simple Yarn 2 plugin to mitigate the issue: https://github.com/rewardops/plugin-ignore-add-options

[bartocc]

Not sure if I should post this here or make another issue but ember install is incompatible too.
Seems like v2 of the yarn CLI uses -i or --interactive rather than --non-interactive.

ember install ember-file-upload

🚧  Installing packages... This might take a couple of minutes.
Command failed: yarn add --dev ember-file-upload --non-interactive

Unknown Syntax Error: Command not found; did you mean one of:

  0. yarn add [--json] [-E,--exact] [-T,--tilde] [-C,--caret] [-D,--dev] [-P,--peer] [-O,--optional] [--prefer-dev] [-i,--interactive] [--cached] ...
  1. yarn add [--json] [-E,--exact] [-T,--tilde] [-C,--caret] [-D,--dev] [-P,--peer] [-O,--optional] [--prefer-dev] [-i,--interactive] [--cached] ...

While running add --dev ember-file-upload --non-interactive

For anyone experiencing this bugaboo, I've created a simple Yarn 2 plugin to mitigate the issue: https://github.com/rewardops/plugin-ignore-add-options

Thx for this plugin 👍

@shamrt I'm curious, are you using nodeLinker: node-modules in you ember project?

[shamrt]

@shamrt I'm curious, are you using nodeLinker: node-modules in you ember project?

Yep, we are 🙂

[bartocc]

@shamrt I'm curious, are you using nodeLinker: node-modules in you ember project?

Yep, we are 🙂

Thanks for the feedback 👍

@rwjblue do you know if there are any plans to make ember-cli compatible with yarn's Zero-Installs? I've seen that in ember-cli v4.0.0-beta.1, pnpm support was added via #9563, so I thought maybe some new work on yarn is in the process as well ;-)

[shamrt]

@rwjblue do you know if there are any plans to make ember-cli compatible with yarn's Zero-Installs? I've seen that in ember-cli v4.0.0-beta.1, pnpm support was added via #9563, so I thought maybe some new work on yarn is in the process as well ;-)

I should mention that we're using Zero-installs as well!

[LucasHillDex]

@rwjblue Could you comment on whether ember-cli plans to support PnP? Our team is reviewing options for where to go after yarn v1.

@shamrt If you are using nodeLinker: node-modules you probably aren't fully using zero-installs I'd think? You can commit the .yarn/cache folder to get part way there, but unless you switch to PnP you still have to run a yarn install to explode the cache into the node_modules folder. Unless there's something I'm missing.

[ef4]

Closing this as stale because it's working in yarn 3.

[aberres]

@ef4 Is there a writeup somewhere on what needs to be done to switch an existing app to PnP?

My naive start with setting nodeLinker: pnp led to

$ ember build 
Required packages are missing, run `yarn install` from this directory to install them.

[ef4]

This issue was created at the time when Yarn 2 forced PnP, so supporting PnP was a blocker for upgrading Yarn. Yarn maintainers eventually realized that wasn't practical and made PnP optional in later 2.x and all of 3.x Yarn. I'm pretty sure the people reporting they're using Yarn 3 are doing it without PnP. I doubt PnP works out of the box.

I don't think anybody would be against seeing PnP blockers fixed, but I also don't expect anybody is working on it because the people who would be working on it (1) have adopted pnpm instead and (2) are putting their time into getting us all off Ember-specific build tooling and onto general-Javascript-ecosystem-supported build-tooling (where the issue of "does it work with Yarn PnP" is not one we have to solve by ourselves).

[aberres]

@ef4 Thanks for the extensive answer. This clarifies things.