Dependency Conflict on mem-fs for yeoman-generator 7.x

[madorb]

Forgive me for any mistaken terminology or assumptions, I am not a node developer...

Upgrading my generator to use yeoman-generator@7.1.0 causes the following issue:

npm ERR! While resolving: @yeoman/types@1.1.1
npm ERR! Found: mem-fs@4.0.0
npm ERR! node_modules/mem-fs
npm ERR!   mem-fs@"^4.0.0" from the root project
npm ERR!   peer mem-fs@"^4.0.0" from yeoman-generator@7.1.0
npm ERR!   node_modules/yeoman-generator
npm ERR!     yeoman-generator@"^7.1.0" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peerOptional mem-fs@"^3.0.0" from @yeoman/types@1.1.1
npm ERR! node_modules/@yeoman/types
npm ERR!   peer @yeoman/types@"^1.1.1" from yeoman-generator@7.1.0
npm ERR!   node_modules/yeoman-generator
npm ERR!     yeoman-generator@"^7.1.0" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: mem-fs@3.0.0
npm ERR! node_modules/mem-fs
npm ERR!   peerOptional mem-fs@"^3.0.0" from @yeoman/types@1.1.1
npm ERR!   node_modules/@yeoman/types
npm ERR!     peer @yeoman/types@"^1.1.1" from yeoman-generator@7.1.0
npm ERR!     node_modules/yeoman-generator
npm ERR!       yeoman-generator@"^7.1.0" from the root project

Again, not a node developer, but this reads to me as yeoman-generator depending on mem-fs@^4.0.0 as well as @yeoman/types@1.1.1 but @yeoman/types@1.1.1 specifically requiring mem-fs@^3.0.0.

This seems like a fundamental conflict that's controlled entirely within the yeoman ecosystem - I'm not sure what the recommended solution is.

[mshima]

@yeoman/types have acceptDependencies.
https://github.com/yeoman/yeoman-api/blob/af49cc9f1c276ecb0b8ee334a8570c9596221da2/workspaces/types/package.json#L60

Maybe you should update yeoman-test and yeoman-environment together.

[madorb]

@yeoman/types have acceptDependencies. https://github.com/yeoman/yeoman-api/blob/af49cc9f1c276ecb0b8ee334a8570c9596221da2/workspaces/types/package.json#L60

Maybe you should update yeoman-test and yeoman-environment together.

I'm likely missing something - but it's not clear to me how yeoman-test or yeoman-environment are related to the issue? I don't have either in my package.json nor do they show up in the error output.

I saw that @yeoman/types had acceptDependencies but it certainly doesn't seem to be working as intended. I wonder if it's because 4.0.0-beta.1 isn't valid semver?

In case it's useful, here is my dependencies section from package.json:

  "dependencies": {
    "semver": "^7.5.2",
    "update-notifier": "^6.0.2",
    "yeoman-generator": "^7.1.0",
    "yosay": "^3.0.0",
    "chalk": "^5.3.0",
    "mem-fs": "^4.0.0",
  },

[mshima]

I’ve created a new folder and run npm install yeoman-generator, no conflicts there using npm@latest.
You should try to remove node_modules, package-lock, and update npm.

[madorb]

I get errors on completely new projects using various different configurations. I've asked other folks on my team to try this out as well and they get same error.

Node 18:

❯ node --version
v18.18.0
❯ npm --version
9.8.1
❯ mkdir yeoman-test
❯ cd yeoman-test
❯ npm install yeoman-generator
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: undefined@undefined
npm ERR! Found: mem-fs@3.0.0
npm ERR! node_modules/mem-fs
npm ERR!   peerOptional mem-fs@"^3.0.0" from @yeoman/types@1.1.2
npm ERR!   node_modules/@yeoman/types
npm ERR!     peer @yeoman/types@"^1.1.1" from yeoman-generator@7.1.1
npm ERR!     node_modules/yeoman-generator
npm ERR!       yeoman-generator@"*" from the root project
npm ERR!   mem-fs@"^3.0.0" from mem-fs-editor@10.0.3
npm ERR!   node_modules/mem-fs-editor
npm ERR!     peerOptional mem-fs-editor@"^10.0.2" from @yeoman/types@1.1.2
npm ERR!     node_modules/@yeoman/types
npm ERR!       peer @yeoman/types@"^1.1.1" from yeoman-generator@7.1.1
npm ERR!       node_modules/yeoman-generator
npm ERR!         yeoman-generator@"*" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer mem-fs@"^4.0.0" from yeoman-generator@7.1.1
npm ERR! node_modules/yeoman-generator
npm ERR!   yeoman-generator@"*" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

Node 20 w/ out-of-the-box npm version from nodenv:

❯ node --version
v20.9.0
❯ npm --version
10.1.0
❯ mkdir yeoman-test-n20
❯ cd yeoman-test-n20
❯ npm install yeoman-generator
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: undefined@undefined
npm ERR! Found: mem-fs@3.0.0
npm ERR! node_modules/mem-fs
npm ERR!   peerOptional mem-fs@"^3.0.0" from @yeoman/types@1.1.2
npm ERR!   node_modules/@yeoman/types
npm ERR!     peer @yeoman/types@"^1.1.1" from yeoman-generator@7.1.1
npm ERR!     node_modules/yeoman-generator
npm ERR!       yeoman-generator@"*" from the root project
npm ERR!   mem-fs@"^3.0.0" from mem-fs-editor@10.0.3
npm ERR!   node_modules/mem-fs-editor
npm ERR!     peerOptional mem-fs-editor@"^10.0.2" from @yeoman/types@1.1.2
npm ERR!     node_modules/@yeoman/types
npm ERR!       peer @yeoman/types@"^1.1.1" from yeoman-generator@7.1.1
npm ERR!       node_modules/yeoman-generator
npm ERR!         yeoman-generator@"*" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer mem-fs@"^4.0.0" from yeoman-generator@7.1.1
npm ERR! node_modules/yeoman-generator
npm ERR!   yeoman-generator@"*" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

Node 20 with latest npm

❯ node --version
v20.9.0
❯ npm --version
10.2.3
❯ mkdir yeoman-test-n20-npm-102
❯ cd yeoman-test-n20-npm-102
❯ npm install yeoman-generator
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: undefined@undefined
npm ERR! Found: mem-fs@3.0.0
npm ERR! node_modules/mem-fs
npm ERR!   peerOptional mem-fs@"^3.0.0" from @yeoman/types@1.1.2
npm ERR!   node_modules/@yeoman/types
npm ERR!     peer @yeoman/types@"^1.1.1" from yeoman-generator@7.1.1
npm ERR!     node_modules/yeoman-generator
npm ERR!       yeoman-generator@"*" from the root project
npm ERR!   mem-fs@"^3.0.0" from mem-fs-editor@10.0.3
npm ERR!   node_modules/mem-fs-editor
npm ERR!     peerOptional mem-fs-editor@"^10.0.2" from @yeoman/types@1.1.2
npm ERR!     node_modules/@yeoman/types
npm ERR!       peer @yeoman/types@"^1.1.1" from yeoman-generator@7.1.1
npm ERR!       node_modules/yeoman-generator
npm ERR!         yeoman-generator@"*" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer mem-fs@"^4.0.0" from yeoman-generator@7.1.1
npm ERR! node_modules/yeoman-generator
npm ERR!   yeoman-generator@"*" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

[madorb]

well. turns out it looks like it's something in our environment / internal repo because:

npm install yeoman-generator --registry=https://registry.npmjs.com/

works fine.

sorry for the confusion.

[madorb]

Well. It is on our side... but we're definitely not the only folks who are gonna run into this.

acceptedDependencies is not defined as being returned on the abbreviated metadata object, and as such it's not returned by our artifactory instance when installing. It appears that npm repo though does return it.

If i hack my local npm's arborist/build-ideal-tree.js to request the full metadata from artifactory it works properly.

https://github.com/npm/registry/blob/master/docs/responses/package-metadata.md#abbreviated-version-object

It seems to me like acceptedDependencies should be considered unusable until it's part of the documented metadata "spec" so that non-npm repositories know they should be implementing it?

[mshima]

Thanks for the follow up.

Unfortunately moving the dependency to dependencies instead of acceptDependencies would be a breaking change since mem-fs@4 would be installed by default and it requires node 18 while types requires node 16.
Maybe there would be others side effects to the types.

You can try to use overrides to workaround the problem.

[madorb]

Ah, thank you! not being a node developer - i was fully unaware of overrides, but that seems to have done the trick.

I've opened an MR to try to get acceptDependencies documented npm/registry#24 so that other vendors can ensure their products return it in abbreviated metadata. Appreciate your help

[LandonSchropp]

@madorb Could you please share how to set up overrides to fix your issue?