Implement Yarn 2 support #1630

constgen · 2020-10-03T22:25:06Z

Resolves #1277

Bump minimal Jest version to support Plug and Play
Add PnP middleware
Make sure all resolves are PnP compatible
Use PnP middleware in all presets
Adopt all ESLint presets as there are known problems with ESLint sharable configs and PnP

constgen · 2020-10-03T22:46:05Z

I can see in the build logs that the job published packages to NPM. But why? It even didn't pass the code formatting check. And this is only Pull Request.

edmorley

Hi! Thank you for the PR.

I much prefer Yarn over NPM and have wanted PnP support for a while.

That said:

My understanding is that webpack 5 supports Yarn PnP out of the box, so much of this may be redundant in Neutrino 10+
Jest needs updating to a minor newer version (but bumping the peer dep is a breaking change), so we'd have to have a docs element to this anyway
This implementation adds the pnp-webpack-plugin dependency and config complexity to NPM users not just yarn

As such, I'm slightly reluctant to add a new middleware for Neutrino 9 only, which would be removed in Neutrino 10 - and think it might make sense for this to be a new section in the docs for Neutrino 9, that contains a snippet for people to add to their .neutrino.js along with some suggestions as to Jest upgrades etc.

Then in Neutrino 10 with webpack 5, this could just be supported natively with little implementation needed on Neutrino's side.

Thoughts?

packages/jest/package.json

edmorley · 2020-10-12T08:13:25Z

I can see in the build logs that the job published packages to NPM. But why?

It's not a real NPM server - see:
https://github.com/neutrinojs/neutrino/blob/master/scripts/test-create-project-ci.sh

constgen · 2020-10-12T10:47:07Z

Here are my thoughts.
I have my personal set of Neutrino middlewares and presets for React, ESLint and other. I reuse couple of Neutrino core middlewares, but most are custom. I faced really a lot of problems with Yarn 2. Especially with ESlint. I resolved them. Some solutions were really hacky. So I decided to integrate my experience to the Neutrino core.

Lets discuss your points

PnP will be supported in Webpack 5. This is true. Until this moment configuration need to use PnPPlugin. Some projects need this already today. Internally the plugin checks the PnP environment (Yarn documented how to do this) and enables necessary functionality on demand. Speaking simply it has no effect on NPM and PNPM projects. I also thought about conditionally enabling it but realized that it is easier to inspect Webpack Config when it is always included. This is how CRA and Storybook do - they always include PnPPlugin to configuration.
I will document the integration of Jest and PnP. But please recommend me where to write "PnP" section
I thought that Neutrino goal is to hide the config complexity from the consumer. That's why I decided that react and vue presets should support this out of the box without thinking how to enable this. And according to the first point it should not affect the build process on NPM. Again, CRA does this. Also it is little easier to upgrade to Webpack 5 later. You will not need to edit .neutrinorc.js

merceyz · 2020-11-03T14:37:09Z

packages/eslint/alias-plugins.js

+    if (pnpApi) {
+      resolvedPluginPath = pnpApi.resolveRequest(pluginName, relativeFilename);
+    } else {
+      resolvedPluginPath = require.resolve(pluginName, {
+        paths: [relativeFilename],
+      });
+    }


You should just use require.resolve you almost never need to use the pnpapi directly, this is one of those cases

I know. But unfortunately __filename gives a virtual File System path in Yarn 2 which can't work correctly in paths options of require.resolve. Module will be not found with an exception. Struggled with this 2 days.

Do you have a reproduction for that? We have tests making sure it works. I could see that happening if you have enableGlobalCache: true but that will be fixed in yarnpkg/berry#2068

In any place of your code in Yarn 2 environment try this console.log(require.resolve('any-of-your-dependency', { paths: [__filename] })). It will not find your module even if it is relative from the current file. The same code works without any problems in regular NPM environment

It will and it does. As I said we have tests that cover this, but just for fun I did try exactly what you said and it works, if you have a reproduction I'd love to take a look at it

Some tests that check this:
https://github.com/yarnpkg/berry/blob/f8344493b5cf3948e71211697b50a268b756d8fc/packages/acceptance-tests/pkg-tests-specs/sources/pnp.test.js#L1714
https://github.com/yarnpkg/berry/blob/f8344493b5cf3948e71211697b50a268b756d8fc/packages/acceptance-tests/pkg-tests-specs/sources/pnp.test.js#L609-L612
https://github.com/yarnpkg/berry/blob/f8344493b5cf3948e71211697b50a268b756d8fc/packages/acceptance-tests/pkg-tests-specs/sources/pnp.test.js#L647-L651

I realized that the mentioned test case works in the Yarn 2 project itself. But in the linked module it fails with not found dependency. As you understand I use linked packages for local environments. Such modules are defined as

{ "devDependencies": { "@neutrinojs/eslint": "portal:C:/Projects/Projects_GitHub/neutrino-dev-fork/packages/eslint", } }

yarn install is called to setup all links in .yarn/cache and .pnp.js
And after that I can run local script "lint": "eslint ./src --ext .js,.jsx", to run the code and test the case

Yeah, that makes more sense. I'm fairly certain the issue you're running into was fixed in yarnpkg/berry#2068, could you try with yarn set version from sources or by downloading the bundle here https://github.com/yarnpkg/berry/actions/runs/346381133?

Before: Yarn version v2.2.2
After: Yarn version 2.3.3-git.20201112.f5ca1c57
Result is the same in the linked package: works when require.resolve('babel-eslint') and fails when require.resolve('babel-eslint', { paths: [__filename] })

Oops! Something went wrong! :( ESLint: 7.9.0 Error: Cannot read config file: C:\Users\const\Desktop\yarn-test\.eslintrc.js Error: Cannot find module 'babel-eslint'

Could you try using __dirname instead?
Again, if you could provide a reproduction I'll be able to look into it

Update:
After removing .yarn/cache and reinstalling dependencies require.resolve('babel-eslint', { paths: [__filename] }) started to work. It was the version 2.3.3-git.20201112.f5ca1c57.
Than I upgraded Yarn to the latest 2.4.0 and reinstalled dependencies again. And it still works. So I removed usage of PnP API as you asked

constgen · 2020-11-03T16:56:59Z

Please, look at the new section in the documentation https://github.com/constgen/neutrino-dev/tree/pnp/packages/eslint#utility-functions

constgen · 2020-11-16T19:42:29Z

I think everything related to Yarn 2 is resolved.
The only left are tests. And I think there is a separate problem. I feel like tests can affect each other and they are not isolated. Seems like they may have a shared neutrino config and all Webpack plugins are applied across test. Is it possible? Because currently just commenting one test can make other tests red or green.

constgen · 2020-12-01T21:23:46Z

@edmorley in Webpack 5 only pnp-webpack-plugin will be not necessary. All other fixes to ESLint, Jest and etc. will be highly required. Their issues still remain. So with pnp-webpack-plugin or without it we need this PR any way. But without pnp-webpack-plugin it will be hard to automate testing in Yarn 2. In my opinion it affects only several presets and it will be really easy to remove it in Neutrino 10 without affecting anything.

constgen · 2021-03-08T21:15:10Z

@edmorley @eliperelman can you look at final version of this PR?

edmorley · 2021-03-08T21:21:58Z

@constgen Thank you for working on this :-) Before it can be merged (and so before spending time on review), we need to have CI passing, which is blocked on Travis CI credits running out, due to Travis CI's new policy (that is, there are no more free credits). The best way to resolve this would be to migrate to another provider -- I would recommend Circle CI. I don't have time to work on such a migration, but would be happy to review a PR that did so.

constgen · 2021-03-09T13:10:57Z

Is GitHub Actions is an option? I tried it recently for one of my projects as a replacement for Travis. Have to say that it also has free plan limits but it worked well for me

edmorley · 2021-03-09T13:13:41Z

Yeah that works too :-)

edmorley · 2024-02-03T19:51:29Z

Closing since unfortunately this project is no longer maintained. See:
#1707
neutrinojs/webpack-chain#358

constgen added 2 commits October 3, 2020 23:32

Use only PnP compatible versions of Jest

b752d2f

Add PnP middleware

f1fe44d

Use PnP middleware in presets

4052136

edmorley suggested changes Oct 12, 2020

View reviewed changes

packages/jest/package.json Outdated Show resolved Hide resolved

edmorley marked this pull request as draft October 12, 2020 09:10

edmorley added the feature label Oct 12, 2020

constgen added 4 commits October 13, 2020 20:08

Add compatibility of PnP in Jest preset aliases

4096203

Merge branch 'master' into pnp

bb9c7e1

Revert Jest peer dependency

9aa579d

Add utility to alias eslint plugins

6b79b04

merceyz reviewed Nov 3, 2020

View reviewed changes

constgen added 2 commits November 3, 2020 18:31

Add utility to alias eslint import resolvers

a0aa2da

Support Yarn 2 in ESLint configuration of all presets

a922481

Make Webpack optional dependency for ESLint preset

816197d

constgen requested a review from edmorley November 6, 2020 12:22

constgen marked this pull request as ready for review December 1, 2020 21:17

constgen changed the title ~~[WIP] Implement Yarn 2 support~~ Implement Yarn 2 support Dec 1, 2020

constgen added 5 commits January 2, 2021 13:07

Remove usage of PnP API

d031f29

Merge branch 'master' into pnp

c4f46a2

Create new instancies of PnP plugin every time

128029d

Correctly test Vue ESlint config dynamic values

86fe991

Fix Prettier violations

53e6b6b

edmorley removed their request for review April 26, 2022 08:36

edmorley closed this Feb 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Yarn 2 support #1630

Implement Yarn 2 support #1630

constgen commented Oct 3, 2020 •

edited

constgen commented Oct 3, 2020

edmorley left a comment

edmorley commented Oct 12, 2020

constgen commented Oct 12, 2020 •

edited

merceyz Nov 3, 2020

constgen Nov 3, 2020 •

edited

merceyz Nov 3, 2020

constgen Nov 3, 2020

merceyz Nov 3, 2020 •

edited

constgen Nov 6, 2020 •

edited

merceyz Nov 6, 2020

constgen Nov 12, 2020

merceyz Dec 1, 2020

constgen Jan 2, 2021

constgen commented Nov 3, 2020

constgen commented Nov 16, 2020

constgen commented Dec 1, 2020

constgen commented Mar 8, 2021

edmorley commented Mar 8, 2021

constgen commented Mar 9, 2021 •

edited

edmorley commented Mar 9, 2021

edmorley commented Feb 3, 2024

Implement Yarn 2 support #1630

Implement Yarn 2 support #1630

Conversation

constgen commented Oct 3, 2020 • edited

constgen commented Oct 3, 2020

edmorley left a comment

Choose a reason for hiding this comment

edmorley commented Oct 12, 2020

constgen commented Oct 12, 2020 • edited

merceyz Nov 3, 2020

Choose a reason for hiding this comment

constgen Nov 3, 2020 • edited

Choose a reason for hiding this comment

merceyz Nov 3, 2020

Choose a reason for hiding this comment

constgen Nov 3, 2020

Choose a reason for hiding this comment

merceyz Nov 3, 2020 • edited

Choose a reason for hiding this comment

constgen Nov 6, 2020 • edited

Choose a reason for hiding this comment

merceyz Nov 6, 2020

Choose a reason for hiding this comment

constgen Nov 12, 2020

Choose a reason for hiding this comment

merceyz Dec 1, 2020

Choose a reason for hiding this comment

constgen Jan 2, 2021

Choose a reason for hiding this comment

constgen commented Nov 3, 2020

constgen commented Nov 16, 2020

constgen commented Dec 1, 2020

constgen commented Mar 8, 2021

edmorley commented Mar 8, 2021

constgen commented Mar 9, 2021 • edited

edmorley commented Mar 9, 2021

edmorley commented Feb 3, 2024

constgen commented Oct 3, 2020 •

edited

constgen commented Oct 12, 2020 •

edited

constgen Nov 3, 2020 •

edited

merceyz Nov 3, 2020 •

edited

constgen Nov 6, 2020 •

edited

constgen commented Mar 9, 2021 •

edited