Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implements peerDependenciesMeta #224

Closed
wants to merge 1 commit into from

Conversation

arcanis
Copy link
Contributor

@arcanis arcanis commented Jul 30, 2019

This diff implements the peerDependenciesMeta field specified in the following rfc.

What is it?

This field provides a way to package authors to disable the peer dependency warnings for packages that may be needed for some integrations, but only affect a subset of the users and aren't worth printing a warning to all.

Why implement it this way?

The rfc can be read for more details on why this syntax and not another; it's worth noting that both Yarn and pnpm (cc @zkochan) have been supporting this field for about half a year now.

Why is it important?

With npm still being roughly half the Javascript ecosystem, some maintainers are found omitting to declare their peer dependencies altogether just to avoid the peer dependency warning to their users. It generates invalid dependency trees that may work in the best cases, but break when an unrelated package is added to the dependency tree and changes the way the hoisting works.

Why is it broken not to list peer dependencies?

It's convoluted, click to expand 🙂

Proof by example; imagine the following setup:

  • A depends on B@*, and has an unlisted peer dependency on X
  • your app depends on A and X@1
  • B@1 depends on nothing

In this situation, we get the following tree:

. - A - B@1
  \
    X@1

The package A is able to access X@1 (assuming a node_modules install), everything is ok. Now let's say that B gets a new release, B@2, which includes a dependency on X@2:

. - A - B@2 - X@2
  \
    X@1

But because of the hoisting, and because there is no conflict doing this, X will be moved from B to its parent folder. We'll now get the following:

. - A - B@2
  \   \ 
    X   X@2

Because of this hoisting, when A makes a require call to obtain X, it will no longer get the X@1 provided by its parent. Instead it will get X@2 provided by its child, breaking the original contract in a very subtle way.

If X had been listed as peer dependency of A, even optional, then the package manager would have been able to detect that hoisting X into A would have caused a conflict, and this problem would have been avoided.

Where are the tests?

This diff was manually tested by patching my local npm and trying it out. I also don't see how it could have any adverse effect, although I'm not familiar with the codebase.

Regarding automated tests, given that the fixtures are stored in a separate repository I figured you might want to integrate them yourself - although I can do it if you prefer. From a cursory glance, the relevant files to modify are:

@arcanis arcanis requested a review from a team as a code owner July 30, 2019 11:11
@isaacs isaacs added the semver:minor new backwards-compatible feature label Jul 31, 2019
@isaacs
Copy link
Contributor

isaacs commented Jul 31, 2019

This seems fine to me. In an ideal world, we would've maybe done dependenciesMeta for devDependencies, optionalDependencies, and bundleDependencies, or even supported something like dependencies:{<name>:{spec,optional}} but those ships sailed a long time ago. (The argument against is that it adds complexity, but it's a close call, imo.)

I think we can get this into 6.11. There'll probably be at least one or two more bug fix releases on 6.10 before then. If you feel motivated to send a PR to npm-registry-mock to add the optional peer dep fixture, that'd be great, but if not, I'll get to it when it's time to land.

This section of the code is all going away in v7, but I'll make sure to work the peerDependenciesMeta support into arborist. (In npm v7, peerDeps will be installed, and it'll enforce the constraint that they are not allowed nested under the package that peer depends on them, so hopefully we'll see fewer cases where they're unlisted due to the warnings.)

@arcanis
Copy link
Contributor Author

arcanis commented Jul 31, 2019

Sounds good - I've opened a PR at npm/npm-registry-mock#34 🙂

we would've maybe done dependenciesMeta for devDependencies, optionalDependencies, and bundleDependencies

For the record, we also have a dependenciesMeta field that can be, for example, used to prevent a package from being built. The optionality is also one of those settings, but mostly for internal code consistency: we still support optionalDependencies (and we would encourage using it over dependenciesMeta[].optional), but under the hood it gets converted to the new syntax.

In npm v7, peerDeps will be installed, and it'll enforce the constraint that they are not allowed nested under the package that peer depends on them

Do you have an rfc / design document? It sounds like a significant change - many packages have what are essentially optional peer dependencies, and them being automatically installed may have adverse size or behavioural effects. Maybe it would make sense to instead make this opt-in by adding a peerDependenciesMeta[].auto flag?

@isaacs
Copy link
Contributor

isaacs commented Aug 8, 2019

Implementation for npm v7: npm/arborist@5288b21

@isaacs isaacs closed this in a123410 Aug 20, 2019
Trott pushed a commit to npm/node that referenced this pull request Aug 20, 2019
Full release notes:

A few meaty bugfixes, and introducing `peerDependenciesMeta`.

FEATURES

* [`a12341088`](npm/cli@a123410)
  [nodejs#224](npm/cli#224) Implements
  peerDependenciesMeta ([@arcanis](https://github.com/arcanis))
* [`2f3b79bba`](npm/cli@2f3b79b)
  [nodejs#234](npm/cli#234) add new forbidden 403
  error code ([@claudiahdz](https://github.com/claudiahdz))

BUGFIXES

* [`24acc9fc8`](npm/cli@24acc9f)
  and
  [`45772af0d`](npm/cli@45772af)
  [nodejs#217](npm/cli#217)
  [npm.community#8863](https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863)
  [npm.community#9327](https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327,)
  do not descend into directory deps' child modules, fix shrinkwrap
  files that inappropriately list child nodes of symlink packages
  ([@isaacs](https://github.com/isaacs) and
  [@salomvary](https://github.com/salomvary))
* [`50cfe113d`](npm/cli@50cfe11)
  [nodejs#229](npm/cli#229) fixed typo in semver doc
  ([@gall0ws](https://github.com/gall0ws))
* [`e8fb2a1bd`](npm/cli@e8fb2a1)
  [nodejs#231](npm/cli#231) Fix spelling mistakes in
  CHANGELOG-3.md ([@XhmikosR](https://github.com/XhmikosR))
* [`769d2e057`](npm/cli@769d2e0)
  [npm/uid-number#7](npm/uid-number#7) Better
  error on invalid `--user`/`--group` configs. This addresses the issue
  when people fail to install binary packages on Docker and other
  environments where there is no 'nobody' user.
  ([@isaacs](https://github.com/isaacs))
* [`8b43c9624`](npm/cli@8b43c96)
  [nodejs#28987](nodejs#28987)
  [npm.community#6032](https://npm.community/t/npm-ci-doesnt-respect-npmrc-variables/6032)
  [npm.community#6658](https://npm.community/t/npm-ci-doesnt-fill-anymore-the-process-env-npm-config-cache-variable-on-post-install-scripts/6658)
  [npm.community#6069](https://npm.community/t/npm-ci-does-not-compile-native-dependencies-according-to-npmrc-configuration/6069)
  [npm.community#9323](https://npm.community/t/npm-6-9-x-not-passing-environment-to-node-gyp-regression-from-6-4-x/9323/2)
  Fix the regression where random config values in a .npmrc file are not
  passed to lifecycle scripts, breaking build processes which rely on
  them.  ([@isaacs](https://github.com/isaacs))
* [`8b85eaa47`](npm/cli@8b85eaa)
  save files with inferred ownership rather than relying on `SUDO_UID`
  and `SUDO_GID`. ([@isaacs](https://github.com/isaacs))
* [`b7f6e5f02`](npm/cli@b7f6e5f)
  Infer ownership of shrinkwrap files
  ([@isaacs](https://github.com/isaacs))
* [`54b095d77`](npm/cli@54b095d)
  [nodejs#235](npm/cli#235) Add spec to dist-tag
  remove function ([@theberbie](https://github.com/theberbie))

DEPENDENCIES

* [`dc8f9e52f`](npm/cli@dc8f9e5)
  `pacote@9.5.7`: Infer the ownership of all unpacked files in
  `node_modules`, so that we never have user-owned files in root-owned
  folders, or root-owned files in user-owned folders.
  ([@isaacs](https://github.com/isaacs))
* [`bb33940c3`](npm/cli@bb33940)
  `cmd-shim@3.0.0`:
  * [`9c93ac3`](npm/cmd-shim@9c93ac3)
    [#2](npm/cmd-shim#2)
    [npm#3380](npm/npm#3380) Handle
    environment variables properly
    ([@basbossink](https://github.com/basbossink))
  * [`2d277f8`](npm/cmd-shim@2d277f8)
    [nodejs#25](npm/cmd-shim#25)
    [nodejs#36](npm/cmd-shim#36)
    [nodejs#35](npm/cmd-shim#35) Fix 'no shebang' case
    by always providing `$basedir` in shell script
    ([@igorklopov](https://github.com/igorklopov))
  * [`adaf20b`](npm/cmd-shim@adaf20b)
    [nodejs#26](npm/cmd-shim#26) Fix `$*` causing an
    error when arguments contain parentheses
    ([@satazor](https://github.com/satazor))
  * [`49f0c13`](npm/cmd-shim@49f0c13)
    [nodejs#30](npm/cmd-shim#30) Fix paths for
    MSYS/MINGW bash ([@dscho](https://github.com/dscho))
  * [`51a8af3`](npm/cmd-shim@51a8af3)
    [nodejs#34](npm/cmd-shim#34) Add proper support
    for PowerShell ([@ExE-Boss](https://github.com/ExE-Boss))
  * [`4c37e04`](npm/cmd-shim@4c37e04)
    [#10](npm/cmd-shim#10) Work around quoted
    batch file names ([@isaacs](https://github.com/isaacs))
* [`a4e279544`](npm/cli@a4e2795)
  `npm-lifecycle@3.1.3` ([@isaacs](https://github.com/isaacs)):
    * fail properly if `uid-number` raises an error
    * [`7086a1809`](npm/cli@7086a18)
  `libcipm@4.0.3` ([@isaacs](https://github.com/isaacs))
* [`8845141f9`](npm/cli@8845141)
  `read-package-json@2.1.0` ([@isaacs](https://github.com/isaacs))
* [`51c028215`](npm/cli@51c0282)
  `bin-links@1.1.3` ([@isaacs](https://github.com/isaacs))
* [`534a5548c`](npm/cli@534a554)
  `read-cmd-shim@1.0.3` ([@isaacs](https://github.com/isaacs))
* [`3038f2fd5`](npm/cli@3038f2f)
  `gentle-fs@2.2.1` ([@isaacs](https://github.com/isaacs))
* [`a609a1648`](npm/cli@a609a16)
  `graceful-fs@4.2.2` ([@isaacs](https://github.com/isaacs))
* [`f0346f754`](npm/cli@f0346f7)
  `cacache@12.0.3` ([@isaacs](https://github.com/isaacs))
* [`ca9c615c8`](npm/cli@ca9c615)
  `npm-pick-manifest@3.0.0` ([@isaacs](https://github.com/isaacs))
* [`b417affbf`](npm/cli@b417aff)
  `pacote@9.5.8` ([@isaacs](https://github.com/isaacs))

TESTS

* [`b6df0913c`](npm/cli@b6df091)
  [nodejs#228](npm/cli#228) Proper handing of
  /usr/bin/node lifecycle-path test
  ([@olivr70](https://github.com/olivr70))
* [`aaf98e88c`](npm/cli@aaf98e8)
  `npm-registry-mock@1.3.0` ([@isaacs](https://github.com/isaacs))
isaacs added a commit to npm/node that referenced this pull request Aug 21, 2019
Full changelog:

6.11.1 (2019-08-20):

Fix a regression for windows command shim syntax.

* [`37db29647`](npm/cli@37db296)
  `cmd-shim@3.0.2` ([@isaacs](https://github.com/isaacs))

v6.11.0 (2019-08-20):

A few meaty bugfixes, and introducing `peerDependenciesMeta`.

FEATURES

* [`a12341088`](npm/cli@a123410)
  [nodejs#224](npm/cli#224) Implements
  peerDependenciesMeta ([@arcanis](https://github.com/arcanis))
* [`2f3b79bba`](npm/cli@2f3b79b)
  [nodejs#234](npm/cli#234) add new forbidden 403 error
  code ([@claudiahdz](https://github.com/claudiahdz))

BUGFIXES

* [`24acc9fc8`](npm/cli@24acc9f)
  and
  [`45772af0d`](npm/cli@45772af)
  [nodejs#217](npm/cli#217)
  [npm.community#8863](https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863)
  [npm.community#9327](https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327,)
  do not descend into directory deps' child modules, fix shrinkwrap files
  that inappropriately list child nodes of symlink packages
  ([@isaacs](https://github.com/isaacs) and
  [@salomvary](https://github.com/salomvary))
* [`50cfe113d`](npm/cli@50cfe11)
  [nodejs#229](npm/cli#229) fixed typo in semver doc
  ([@gall0ws](https://github.com/gall0ws))
* [`e8fb2a1bd`](npm/cli@e8fb2a1)
  [nodejs#231](npm/cli#231) Fix spelling mistakes in
  CHANGELOG-3.md ([@XhmikosR](https://github.com/XhmikosR))
* [`769d2e057`](npm/cli@769d2e0)
  [npm/uid-number#7](npm/uid-number#7) Better
  error on invalid `--user`/`--group` configs. This addresses the issue
  when people fail to install binary packages on Docker and other
  environments where there is no 'nobody' user.
  ([@isaacs](https://github.com/isaacs))
* [`8b43c9624`](npm/cli@8b43c96)
  [nodejs#28987](nodejs#28987)
  [npm.community#6032](https://npm.community/t/npm-ci-doesnt-respect-npmrc-variables/6032)
  [npm.community#6658](https://npm.community/t/npm-ci-doesnt-fill-anymore-the-process-env-npm-config-cache-variable-on-post-install-scripts/6658)
  [npm.community#6069](https://npm.community/t/npm-ci-does-not-compile-native-dependencies-according-to-npmrc-configuration/6069)
  [npm.community#9323](https://npm.community/t/npm-6-9-x-not-passing-environment-to-node-gyp-regression-from-6-4-x/9323/2)
  Fix the regression where random config values in a .npmrc file are not
  passed to lifecycle scripts, breaking build processes which rely on them.
  ([@isaacs](https://github.com/isaacs))
* [`8b85eaa47`](npm/cli@8b85eaa)
  save files with inferred ownership rather than relying on `SUDO_UID` and
  `SUDO_GID`. ([@isaacs](https://github.com/isaacs))
* [`b7f6e5f02`](npm/cli@b7f6e5f)
  Infer ownership of shrinkwrap files
  ([@isaacs](https://github.com/isaacs))
* [`54b095d77`](npm/cli@54b095d)
  [nodejs#235](npm/cli#235) Add spec to dist-tag remove
  function ([@theberbie](https://github.com/theberbie))

DEPENDENCIES

* [`dc8f9e52f`](npm/cli@dc8f9e5)
  `pacote@9.5.7`: Infer the ownership of all unpacked files in
  `node_modules`, so that we never have user-owned files in root-owned
  folders, or root-owned files in user-owned folders.
  ([@isaacs](https://github.com/isaacs))
* [`bb33940c3`](npm/cli@bb33940)
  `cmd-shim@3.0.0`:
  * [`9c93ac3`](npm/cmd-shim@9c93ac3)
    [#2](npm/cmd-shim#2)
    [npm#3380](npm/npm#3380) Handle environment
    variables properly ([@basbossink](https://github.com/basbossink))
  * [`2d277f8`](npm/cmd-shim@2d277f8)
    [nodejs#25](npm/cmd-shim#25)
    [nodejs#36](npm/cmd-shim#36)
    [nodejs#35](npm/cmd-shim#35) Fix 'no shebang' case by
    always providing `$basedir` in shell script
    ([@igorklopov](https://github.com/igorklopov))
  * [`adaf20b`](npm/cmd-shim@adaf20b)
    [nodejs#26](npm/cmd-shim#26) Fix `$*` causing an
    error when arguments contain parentheses
    ([@satazor](https://github.com/satazor))
  * [`49f0c13`](npm/cmd-shim@49f0c13)
    [nodejs#30](npm/cmd-shim#30) Fix paths for MSYS/MINGW
    bash ([@dscho](https://github.com/dscho))
  * [`51a8af3`](npm/cmd-shim@51a8af3)
    [nodejs#34](npm/cmd-shim#34) Add proper support for
    PowerShell ([@ExE-Boss](https://github.com/ExE-Boss))
  * [`4c37e04`](npm/cmd-shim@4c37e04)
    [#10](npm/cmd-shim#10) Work around quoted
    batch file names ([@isaacs](https://github.com/isaacs))
* [`a4e279544`](npm/cli@a4e2795)
  `npm-lifecycle@3.1.3` ([@isaacs](https://github.com/isaacs)):
    * fail properly if `uid-number` raises an error
* [`7086a1809`](npm/cli@7086a18)
  `libcipm@4.0.3` ([@isaacs](https://github.com/isaacs))
* [`8845141f9`](npm/cli@8845141)
  `read-package-json@2.1.0` ([@isaacs](https://github.com/isaacs))
* [`51c028215`](npm/cli@51c0282)
  `bin-links@1.1.3` ([@isaacs](https://github.com/isaacs))
* [`534a5548c`](npm/cli@534a554)
  `read-cmd-shim@1.0.3` ([@isaacs](https://github.com/isaacs))
* [`3038f2fd5`](npm/cli@3038f2f)
  `gentle-fs@2.2.1` ([@isaacs](https://github.com/isaacs))
* [`a609a1648`](npm/cli@a609a16)
  `graceful-fs@4.2.2` ([@isaacs](https://github.com/isaacs))
* [`f0346f754`](npm/cli@f0346f7)
  `cacache@12.0.3` ([@isaacs](https://github.com/isaacs))
* [`ca9c615c8`](npm/cli@ca9c615)
  `npm-pick-manifest@3.0.0` ([@isaacs](https://github.com/isaacs))
* [`b417affbf`](npm/cli@b417aff)
  `pacote@9.5.8` ([@isaacs](https://github.com/isaacs))

TESTS

* [`b6df0913c`](npm/cli@b6df091)
  [nodejs#228](npm/cli#228) Proper handing of
  /usr/bin/node lifecycle-path test
  ([@olivr70](https://github.com/olivr70))
* [`aaf98e88c`](npm/cli@aaf98e8)
  `npm-registry-mock@1.3.0` ([@isaacs](https://github.com/isaacs))
@zkochan
Copy link

zkochan commented Aug 22, 2019

I still see warnings when installing deps with optional peers (using npm v6.11).

@arcanis
Copy link
Contributor Author

arcanis commented Aug 22, 2019

I quickly checked - it seems that the warning is now properly hidden but only on subsequent installs. I'm not sure why, but the initial install seems to follow a different codepath with regard to the peer dependency validation 🤔

@arcanis
Copy link
Contributor Author

arcanis commented Aug 22, 2019

Further debug seem to indicate that tree.package doesn't contain all the fields during the initial install. Maybe the peerDependenciesMeta field needs to be explicitly forwarded when using the metadata from the registry?

@isaacs
Copy link
Contributor

isaacs commented Aug 22, 2019

Ahhh, duh. I think I know what's happening here. The peerDependenciesMeta field needs to be added to the trimmed down manifest that the registry sends when fetching with the application/vnd.npm.install-v1+json accept header.

I think the cli is doing the right thing, and this is a server-side fix.

Do you have an example handy of a package with this field set, so I can point them to it?

@arcanis
Copy link
Contributor Author

arcanis commented Aug 22, 2019

https://www.npmjs.com/package/ts-pnp has an optional peer dependency on typescript

dfernandez-asapp added a commit to dfernandez-asapp/prettier-package-json that referenced this pull request Jun 12, 2020
When sorting package.json entries it makes sense to put `peerDependenciesMeta` after `peerDependencies`.

The field `peerDependenciesMeta` comes from Yarn and it was implemented NPM: npm/cli#224
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver:minor new backwards-compatible feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants