Yarn 2 package manager: some undeclared dependencies

[DemianX0]

Follow-up for 6.2.9: #14838
These warnings have been fixed pre-6.1.0 in eaa9410

Describe the bug
yarn install (Yarn 2 - berry) produces this output. These are warnings only, not blocking:

➤ YN0000: ┌ Resolution step
➤ YN0002: │ @storybook/api@npm:5.2.8 doesn't provide react-dom@* requested by @storybook/router@npm:5.2.8
➤ YN0002: │ @storybook/api@npm:5.2.8 doesn't provide react-dom@* requested by @storybook/theming@npm:5.2.8
➤ YN0002: │ @storybook/html@npm:5.2.8 [b2d93] doesn't provide react@* requested by @storybook/core@npm:5.2.8
➤ YN0002: │ @storybook/html@npm:5.2.8 [b2d93] doesn't provide react-dom@* requested by @storybook/core@npm:5.2.8
➤ YN0002: │ @storybook/client-api@npm:5.2.8 doesn't provide react@* requested by @storybook/router@npm:5.2.8
➤ YN0002: │ @storybook/client-api@npm:5.2.8 doesn't provide react-dom@* requested by @storybook/router@npm:5.2.8
➤ YN0002: │ @storybook/ui@npm:5.2.8 doesn't provide @emotion/core@^10.0.27 requested by emotion-theming@npm:10.0.27
➤ YN0002: │ @storybook/core@npm:5.2.8 [55985] doesn't provide @babel/core@^7.0.0-0 requested by @babel/plugin-proposal-c
lass-properties@npm:7.8.3
➤ YN0002: │ @storybook/core@npm:5.2.8 [55985] doesn't provide @babel/core@^7.0.0-0 requested by @babel/plugin-proposal-o
bject-rest-spread@npm:7.9.6
➤ YN0002: │ @storybook/core@npm:5.2.8 [55985] doesn't provide @babel/core@^7.0.0-0 requested by @babel/plugin-syntax-dyn
amic-import@npm:7.8.3
➤ YN0002: │ @storybook/core@npm:5.2.8 [55985] doesn't provide @babel/core@^7.0.0-0 requested by @babel/plugin-transform-
react-constant-elements@npm:7.9.0
➤ YN0002: │ @storybook/core@npm:5.2.8 [55985] doesn't provide @babel/core@^7.0.0-0 requested by @babel/preset-env@npm:7.
9.6
➤ YN0000: └ Completed in 15.13s
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed in 23.51s
➤ YN0000: ┌ Link step
➤ YN0000: └ Completed in 2.42s
➤ YN0000: Done in 3.07m

Fixed in 6.1.0 [EDITED]

The missing peerDeps were added in:
https://github.com/storybookjs/storybook/releases/tag/v6.1.0-beta.1
eaa9410#diff-3e4e211bc6c8ad1c4934045fbab3fac862e23c8e00915e433d09d7eed15a4851R49

Temporary fix for previous versions [EDITED]

Create .yarnrc.yml next to package.json:

packageExtensions:
  "@storybook/addon-actions@*":
    peerDependencies:
      react-dom: "*"
  "@storybook/addons@*":
    peerDependencies:
      react-dom: "*"
  "@storybook/api@*":
    peerDependencies:
      react: "*"
      react-dom: "*"
  "@storybook/client-api@*":
    peerDependencies:
      react: "*"
      react-dom: "*"
  "@storybook/core@*":
    peerDependencies:
      "@babel/core": "^7.0.0-0"
  "@storybook/html@*":
    peerDependencies:
      "@babel/core": "^7.0.0-0"
      react: "*"
      react-dom: "*"
  "@storybook/ui@*":
    peerDependencies:
      emotion/core: "^10.0.27"

To Reproduce
Steps to reproduce the behavior:

1. Install Yarn 2 berry in a project using storybook
2. Run: yarn

Expected behavior
No warning messages:

➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed in 1.33s
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed in 11.63s
➤ YN0000: ┌ Link step
➤ YN0000: └ Completed in 8.73s
➤ YN0000: Done in 21.91s

Screenshots

Code snippets
Relevant part of package.json

"devDependencies": {
	"@babel/core": "7.7.7",
	"@storybook/html": "5.2.8",
	"@types/jquery": "3.3.33",
	"@types/mustache": "4.0.1",
	"@types/node-fetch": "2.5.7",
	"babel-loader": "8.0.6",
	...
}

System:
Environment Info:
System:
OS: Windows 8.1 6.3.9600
Binaries:
Node: 12.14.1
Yarn: 2.0.0-rc.27
npm: 6.13.4
npmPackages:
@storybook/cli: ^5.3.19 => 5.3.19
@storybook/html: 5.2.8 => 5.2.8

Additional context

[shilman]

cc @gaetanmaisse

[gaetanmaisse]

I plan to do a big deps cleaning in 6.1 so I hope it will fix this issue.

[stale]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[DavidSabine]

+1 this issue.

It's unclear how to resolve these YN0002 warnings. And it's annoying that yarn keeps growling about these warnings.

1. First, these warnings are produced by various and deep modules.
2. Second, there doesn't appear to be any problem or consequence if package A doesn't provide package B. (e.g. gastby-plugin-sass doesn't provide webpack... but a zillion others DO request webpack... so webpack is in the tree... so why the warning?)
3. Is there a way to silence these warnings? Fine, maybe it's appropriate that yarn growl about these issues. But (sorry, not sorry) the maintenance of an infinite number of packages is not my concern. An option to show these warnings (opt in) or silence them (opt out) would improve the developer experience.

[gaetanmaisse]

To give some context about a big part of these warnings, they are related to the modularity of Storybook and the fact it has to support a dozen of different framework. One of these frameworks is React but the Storybook app itself is also using React, so to avoid any compatibility issue or dependencies duplication some Storybook packages need to have React as peer dependencies and not regular ones. You can found more details about all that in #13194.

Regarding:

Second, there doesn't appear to be any problem or consequence if package A doesn't provide package B. (e.g. gastby-plugin-sass doesn't provide webpack... but a zillion others DO request webpack... so webpack is in the tree... so why the warning?)

It certainly works in many cases but what if the version of package B requested by other packages isn't compatible with the one expected in package A? And what if the dep tree is a bit different because node_modules aren't hoisted the same way?
Dependency management isn't as simple as it looks and even if things tend to work almost all the time, the edge cases can be 🤯 to debug when it comes to finding out why a dep is sometimes here and sometimes there.

Also related to this subject there is a great article of Yarn lead about transitive peer dep, explaining other tricky situations: https://dev.to/arcanis/implicit-transitive-peer-dependencies-ed0

Is there a way to silence these warnings? Fine, maybe it's appropriate that yarn growl about these issues. But (sorry, not sorry) the maintenance of an infinite number of packages is not my concern. An option to show these warnings (opt in) or silence them (opt out) would improve the developer experience.

I'm not a Yarn 2 expert but I think logFilters could help us on that side: https://yarnpkg.com/configuration/yarnrc#logFilters

[DavidSabine]

I'm not a Yarn 2 expert but I think logFilters could help us on that side: https://yarnpkg.com/configuration/yarnrc#logFilters

This solves my concern. Awesome. Thanks.

[spiltcoffee]

@DavidSabine you might wanna look at packageExtensions, actually: https://yarnpkg.com/configuration/yarnrc#packageExtensions
Purpose built for resolving these kind of dependency issues in upstream packages. You can use the advice from the YN0002 messages and yarn explain peer-requirements to add entries to the packageExtensions entry.

[spiltcoffee]

@gaetanmaisse as for this issue, looks like 6.2.9 is still producing some YN0002 messages from Yarn 2.

package.json:

{
  "devDependencies": {
    "@babel/core": "^7.14.0",
    "@storybook/html": "^6.2.9"
  }
}

Yarn 2:

$ yarn
...
➤ YN0002: │ @storybook/core@npm:6.2.9 [3d935] doesn't provide webpack (p66fcf), requested by @storybook/core-client
➤ YN0002: │ @storybook/html@npm:6.2.9 [dc3fc] doesn't provide webpack (pc0241), requested by html-loader
...

$ yarn explain peer-requirements p66fcf
➤ YN0000: @storybook/core@npm:6.2.9 [3d935] doesn't provide webpack, breaking the following requirements:

➤ YN0000: @storybook/core-client@npm:6.2.9 [cebdf] → * ✘

$ yarn explain peer-requirements pc0241
➤ YN0000: @storybook/html@npm:6.2.9 [dc3fc] doesn't provide webpack, breaking the following requirements:

➤ YN0000: html-loader@npm:1.3.2 [3d935] → ^4.0.0 || ^5.0.0 ✘

Can probably fix these errors by specifying the relevant dependency in the peerDependencies key for each of @storybook/core and @storybook/html.

[spiltcoffee]

For completeness, here are all the warnings Yarn 2 is putting out for v6.2.9 app level packages.

Common

➤ YN0002: │ @storybook/core@npm:6.2.9 [d8225] doesn't provide webpack (p0a9d0), requested by @storybook/core-client

$ yarn explain peer-requirements p0a9d0
➤ YN0000: @storybook/core@npm:6.2.9 [d8225] doesn't provide webpack, breaking the following requirements:
➤ YN0000: @storybook/core-client@npm:6.2.9 [3841a] → * ✘

@storybook/aurelia

➤ YN0002: │ @storybook/aurelia@npm:6.2.9 [dc3fc] doesn't provide typescript (pafa52), requested by fork-ts-checker-webpack-plugin
➤ YN0002: │ @storybook/aurelia@npm:6.2.9 [dc3fc] doesn't provide typescript (p3852a), requested by ts-loader

$ yarn explain peer-requirements pafa52
➤ YN0000: @storybook/aurelia@npm:6.2.9 [dc3fc] doesn't provide typescript, breaking the following requirements:
➤ YN0000: fork-ts-checker-webpack-plugin@npm:4.1.6 [04c11] → >= 2.7 ✘

$ yarn explain peer-requirements p3852a
➤ YN0000: @storybook/aurelia@npm:6.2.9 [dc3fc] doesn't provide typescript, breaking the following requirements:
➤ YN0000: ts-loader@npm:6.2.2 [04c11] → * ✘

@storybook/html

➤ YN0002: │ @storybook/html@npm:6.2.9 [dc3fc] doesn't provide webpack (pc0241), requested by html-loader

$ yarn explain peer-requirements pc0241
➤ YN0000: @storybook/html@npm:6.2.9 [dc3fc] doesn't provide webpack, breaking the following requirements:
➤ YN0000: html-loader@npm:1.3.2 [3d935] → ^4.0.0 || ^5.0.0 ✘

@storybook/marionette

➤ YN0002: │ @storybook/marionette@npm:6.2.9 [dc3fc] doesn't provide webpack (p6672f), requested by html-loader

$ yarn explain peer-requirements p6672f
➤ YN0000: @storybook/marionette@npm:6.2.9 [dc3fc] doesn't provide webpack, breaking the following requirements:
➤ YN0000: html-loader@npm:1.3.2 [f29ef] → ^4.0.0 || ^5.0.0 ✘

@storybook/vue

➤ YN0002: │ @storybook/vue@npm:6.2.9 [dc3fc] doesn't provide typescript (pdfbe4), requested by ts-loader

$ yarn explain peer-requirements pdfbe4
➤ YN0000: @storybook/vue@npm:6.2.9 [dc3fc] doesn't provide typescript, breaking the following requirements:
➤ YN0000: ts-loader@npm:8.2.0 [178ae] → * ✘

@storybook/vue3

➤ YN0002: │ @storybook/vue3@npm:6.2.9 [dc3fc] doesn't provide typescript (pfb977), requested by ts-loader

$ yarn explain peer-requirements pfb977
➤ YN0000: @storybook/vue3@npm:6.2.9 [dc3fc] doesn't provide typescript, breaking the following requirements:
➤ YN0000: ts-loader@npm:8.2.0 [01bb4] → * ✘

[shilman]

@spiltcoffee @gaetanmaisse @merceyz can we solve these warnings by simply adding optional peer dependencies for these packages?

[spiltcoffee]

@shilman

Yep, that seems to be the advice from Yarn: https://yarnpkg.com/advanced/error-codes#yn0002---missing_peer_dependency

I've also successfully nullified the warnings in my own project for now by using the packageExtensions key in my project's .yarnrc.yml (a Yarn 2 project file that sits next to the package.json): https://yarnpkg.com/configuration/yarnrc#packageExtensions
The fact this works using packageExtensions should mean just adding the peerDependencies to those projects should resolve the issue.

If you want to test the dependencies are correct, here is a fairly straightforward^ way to test:

1. $ yarn set version berry - to set the version to Yarn 2 in the current folder
2. $ yarn init - create an empty project
3. $ yarn add -D @storybook/<app> - install the app you're testing
4. Check output for YN0002 warnings complaining about things that @storybook/<app> or @storybook/core "doesn't provide". You can use yarn explain peer-requirements <hash> to get more details about the YN0002 message.

^ definition of straightfoward may vary...

[DavidSabine]

@DavidSabine you might wanna look at packageExtensions, actually: https://yarnpkg.com/configuration/yarnrc#packageExtensions
Purpose built for resolving these kind of dependency issues in upstream packages. You can use the advice from the YN0002 messages and yarn explain peer-requirements to add entries to the packageExtensions entry.

Thank you for the advice.

I find Yarn's documentation about packageExtensions really cryptic. I wonder if you'd be able to set me in the right direction.

Here's an example of YN0002 messages in my build:

So, let's take hash p3a008 and ask yarn to explain the peer-requirements

yarn explain peer-requirements p3a008

Result:

It's not clear (to me) what this message means. Anyways, I've tried many things... this for example:

(Note the addition to .yarnrc.yml; and note the terminal output. Clearly my addition hasn't fixed anything.)

Would you be able to tell me what you'd add to packageExtensions config based on the p3a008 (or any other YN0002) message?

[spiltcoffee]

@DavidSabine Sure, can give it a shot.

With the YN0002 messages that state x doesn't provide y, requested by z, this is because package z lists package y in it's peerDependencies. To resolve this, package x needs to satisfy z's request for y, either in x's dependencies, devDependencies or peerDependencies.

So doing the yarn explain peer-requirements <hash> for each of these will list the requests not satisfied, and you can then address these using the packageExtensions. Using the first one listed as an example:

packageExtensions:
  # spiltcoffee: also, I did just notice that you've dropped the @scope from this package accidentally. Perhaps that's why you had trouble with this one...
  "@endemolshinegroup/cosmiconfig-typescript-loader@3.0.2":
    peerDependencies:
      typescript: ">=2.7"

(preferring to use peerDependencies here as a package that forgets to satisfy a downstream package's peerDependencies probably intends to mimic those peerDependencies.)

The other pattern that might occur is denoted with YN0060, which is where the wrong version of a dependency is provided. These ones can be more problematic, but you can still attempt to resolve them via the above process all the same and see what happens.

With YN0060s, it might happen that the right version cannot be patched in via the packageExtensions, in which case, a PR should be raised on that package to address the inconsistency.

[DavidSabine]

# spiltcoffee: also, I did just notice that you've dropped the @scope from this package accidentally. Perhaps that's why you had trouble with this one...

Actually, it didn't occur to me I'd need quotes. So I removed the @ symbol to avoid an apparent syntax error. (Again, Yarn's documentation is lacking... almost as much as my familiarity with yaml syntax.)

I'll read on...

[DavidSabine]

@DavidSabine Sure, can give it a shot.

With the YN0002 messages that state x doesn't provide y, requested by z ...

I understand now. And I've successfully implemented a few fixes with your help.

Thank you.

BUT OMG! This is a crazy mess. I can't see how this is sustainable — and I can totally see how this will be everyone's last priority. Here are just a few problems:

1. In my tiny website, and with modules that are all perfectly up to date, I have no less than 11 such YN0002 warnings.

2. So, let's say I add 11 entries to my packageExtensions section. Today. And the process takes me no less than 11 minutes to:

• yarn, then yarn explain bla-bla , then modify my yarnrc.yml
• repeat 11 times

3. Let's say tomorrow I run yarn upgrade-interactive and update all my modules, run my tests, deploy my site. BUT tomorrow's modules fix some of these peer requirements, and break others.

4. When this much easier and only slightly more risky.

logFilters:
  - code: YN0002
    level: discard

Perhaps managing peer requirements are not the responsibility of a consumer of a module, but the responsibility of the module publisher(s). Perhaps some enforcement could be applied by the package manager (npmjs.org?) whereby a module cannot be published if peer requirements are properly addressed in package.json.

I don't know the answer to this.

Yarn2 is far superior to v1 in this regard. So, I'm not complaining.

And I do appreciate all the advice in this thread.

[spiltcoffee]

If upgrading a module results in a dependency being met correctly (that is, a rule specified in the packageExtensions no longer applies), Yarn will tell you as such, so eliminating that should be fairly straightforward. But, generally, yes, it's a bit of a mess.

It's not particularly Yarn 2's fault, though - it's just pointing out more clearly that, hey, there are dependency issues here you might wanna do something about.

Best thing to do is to try and report these issues to the affected projects. For example, this issue we're commenting on, and another example is for react-dev-utils that I've raised here: facebook/create-react-app#10902
(But, ah, that repo looks kinda dead to me at the moment, so not sure that one will ever fade haha.)

Yarn does also have a plugin-compat project which aims to consolidate a lot of these packageExtensions with PRs or Issues raised so that you don't have to worry about it as much: https://github.com/yarnpkg/berry/tree/master/packages/plugin-compat
If a project appears particularly dead (like react-dev-utils does to me, for some reason), this can be a way of resolving the issue more globally across Yarn 2 so that you don't need to specify the extension yourself.

[shilman]

My understanding is that the entire JS ecosystem is in transition, and this is a symptom. The new scheme is effectively calling out technical debt that millions of packages have accumulated over the past decade. Some of those will fix the problem, some will get replaced by packages that fix the problem (and written in Typescript or whatever the language-du-jour is), and gradually we'll converge on a new ecosystem that plays by the rules. In the meantime we just need to do our best to fix things where possible, and put up with lots of warnings where it's not.

[DemianX0]

I've just checked back to see that this commit added the missing peerDependencies:
eaa9410#diff-3e4e211bc6c8ad1c4934045fbab3fac862e23c8e00915e433d09d7eed15a4851R49
Thanks to @gaetanmaisse for the commit.

Which was released in https://github.com/storybookjs/storybook/releases/tag/v6.1.0-beta.1
Therefore this issue can be closed.

In storybook 6.2.9 there are 2 more missing peerDependencies:

➤ YN0002: │ @storybook/html@npm:6.2.9 [b2d93] doesn't provide webpack@^4.0.0 || ^5.0.0 requested by html-loader@npm:1.3.2
➤ YN0002: │ @storybook/core@npm:6.2.9 [4fadb] doesn't provide webpack@* requested by @storybook/core-client@npm:6.2.9

These weren't part of this issue. For clarity I've opened the separate #14838 to track those.
Just like in this one, the missing peerDeps don't cause any issue, it's just a warning about missing declarations.

[spiltcoffee]

@DemianX0 will that issue address the other complaints from Yarn 2 about missing dependencies listed here? #11397 (comment)

[merceyz]

can we solve these warnings by simply adding optional peer dependencies for these packages?

@shilman - #11397 (comment)

Sorry for the late response but yes, that would silence them

[DemianX0]

@DemianX0 will that issue address the other complaints from Yarn 2 about missing dependencies listed here? #11397 (comment)

Thanks for mentioning, I've added those too to #14838.
Do you feel it was too early to close this issue? Feel free to reopen.