Ensure/check single version of a dependency package throughout workspace

[wclr]

I think it is quite a common issue, if you have multiple packages/app in you monorepo that may depend on the same packge dependency (for as common example mongodb), it is reasonable to have only single version thought the project (though there maybe need for exeptions sometimes). But eventually you may add this dep to new package, and it will install newer version, but you may not notice this and forget to update the rest.

What is the best way to approach this issue? I thought about using hook, to check lockfile after resolution. Another approach would be declare such dep in the root of the project, so everyone will use it, but even if all your packages/apps (which is rare case) depend on this, this still doesn't seem a nice, clean and lean solution.

[zkochan]

What if we will add something like a linter for package.json files. For instance, we can have something like this in a pnpmfile.js file:

module.exports = {
  lintPackage (pkg) {
    if (pkg.dependencies?.mongodb) {
      pkg.dependencies.mongodb = '^3.0.0'
    }
    return pkg
  }
}

pnpm can use this for two things.

1. it can run this function on each project's manifest and verify that the object return by the function equals the one that was passed in. If they do not match, there is a linting error
2. it can fix packages by writing the returned manifests to the filesystem. So we may have a command like pnpm fix.

Yarn v2 has something similar. It is called constraints and they use Prolog for describing these rules.

cc @pnpm/collaborators

Also, I am not sure it should be part of pnpm. It can be probably a separate project that supports pnpm, npm, yarn.

---

it is not a linter, but for pnpm I have created this script that normalizes package.json files: https://github.com/pnpm/pnpm/blob/master/utils/updater/src/index.ts
and it also generates tsconfig files.

[aparajita]

Can't readPackage do this already?

[zkochan]

The changes of readPackage are not saved to the file system

[aparajita]

Another possible approach:

• Add a setting to .npmrc, something like pinWorkspacePackages = <spec>, where spec is a comma-delimited list of package names which can use * as a wildcard. * alone would match all packages.

• When pnpm install/update is executed, pinWorkspacePackages is consulted. If any existing packages elsewhere in the workspace match <spec> and have the same name as the one being installed/updated and have a different version, they are updated to the same version.

[wclr]

Also, I am not sure it should be part of pnpm. It can be probably a separate project that supports pnpm, npm, yarn.

I think such logic should be out of scope of concern of pnpm itself, but pnpm should allow to achive such linting things with some kind of plugins (which I belive will be mostly implemented via hooks).

What if we will add something like a linter for package.json files.

pkg.dependencies.mongodb = '^3.0.0'

But does it prevent one package from having 3.1.2 and another 3.3.0 installed later and which are fixed in lock file and not updated non explicitly? I'm not sure how it works, but I belive when lockfile is used installed dep exact version if fixed just after install and will not change until explicit update operation on this package?

[aparajita]

I think such logic should be out of scrope of pnpm itself

pnpm is a... package manager. Which excels at working with monorepos. How is managing package versions across the monorepo out of the scope of pnpm?

[wclr]

I think such logic should be out of scrope of pnpm itself

pnpm is a... package manager. Which excels at working with monorepos. How is managing package versions across the monorepo out of the scope of pnpm?

Basic logic of pnpm should be as simple as possible, should concern such things which is not achievalbe in userland, hooks API is a great thing which potentially can allow many things out of scope of basic packages resolution/linking logic which pnpm is responsible for.

[aparajita]

Basic logic of pnpm should be as simple as possible

It's a little too late for that. 😂

[wclr]

Basic logic of pnpm should be as simple as possible

It's a little too late for that. 😂

Well what things does pnpm do which could be actually (potentially) achivable by user using avaialbe API, but not innner pnpm logic?

[ExE-Boss]

Didn’t resolution‑strategy use to do this?

resolution-strategy = fewer-dependencies

[zkochan]

We need two things:

1. packages with the same spec should always be resolved to the same version. So if one project has foo@^3.0.0 and another one has foo@^3.0.0, in both projects pnpm should resolve foo to the same version, to 3.1.0, for instance. This is how Yarn works but Yarn does it on all levels (in subdependencies as well), we'll probably guarantee this only for project dependencies for now. This probably needs a new issue.
2. some linting solution should be possible for pnpm monorepos. It can be a separate tool to integrated to pnpm. Maybe we will just provide a high level API.

[zkochan]

Didn’t resolution‑strategy use to do this?

no. The change introduces by fewer-dependencies was that when dependencies were resolved, versions of already resolved packages from the parent packages were preferred.

[bdchauvette]

I've found syncpack to be pretty handy for ensuring version consistency and standard formatting for package.json files.

It would be great if it had first-party support for pnpm workspaces, but globbing works ok.

[aparajita]

I've found syncpack to be pretty handy

Thanks for the tip!

[aparajita]

I'm working on pnpm support for syncpack, @zkochan one thing I want to be clear on: does pnpm search each entry in pnpm-workspaces.yaml recursively, even if it doesn't end with **?

[zkochan]

maybe you can use this: https://github.com/Thinkmill/manypkg/tree/master/packages/get-packages

does pnpm search each entry in pnpm-workspaces.yaml recursively, even if it doesn't end with **?

no

[aparajita]

maybe you can use this: https://github.com/Thinkmill/manypkg/tree/master/packages/get-packages

That may be just what I'm looking for, thanks.

[aparajita]

@zkochan My modified version of syncpack is ready if you want to try it.

git clone https://github.com/aparajita/syncpack.git
cd syncpack
pnpm install
pnpm build
cd /path/to/pnpm-source
/path/to/syncpack/dist/bin.js list-mismatches

You may find the results interesting.

[zkochan]

I see. There are a few issues.

[aparajita]

Good incentive to implement package syncing in pnpm. 😁

[aparajita]

My PR to add pnpm support to syncpack has been merged and is now available via pnpm i -g syncpack.

[evantahler]

Just chiming in that the linter idea would be great for @grouparoo! On incoming PRs we could fail a test if the version of a dependency doens't match the rest of the monorepo.

[patroza]

@remorses #2713 (comment)
this is cool, thanks!

Im contemplating if adding a marker to package.json that we can pick up, would be a good solution.
"shouldBeUnique": true or so.

[aphex]

@zkochan did anything like inherit ever land for this? I think it is a solid idea when you want to sync packages up when the root workspace also requires the package. So root workspace package.json has xyz:^1.0.4, so any subpackage with xyz: 'inherit' would pick that up. this would solve some current frustrations for me.

However this isn't a great setup for keeping package versions in sync that are not required in the root workspace package.json, its a bit harder to figure out which one to reference in this situation, but you could imagine a monorepo with a front-end and back-end workspace. Maybe you want to share react versions just in the front-end packages.

[dsilvasc]

Gradle does this with a "version catalog" file: https://docs.gradle.org/current/userguide/platforms.html#sub::toml-dependencies-format

A similar approach for pnpm might be:

versions.yaml or versions.toml:

versions:
  react: ^18.0.2
packages:
  # react and react-dom versions should align
  react: react
  react-dom: react
  typescript: ^8.4.0

apps/myapp/package.json:

{
  "dependencies": {
     "react": "catalog:*"
   },
  "devDependencies": {
      "typescript": "catalog:*"
  }
}

Where the catalog protocol looks at a versions file at the root of the repo, and * means use the default versions file (maybe you can have multiple).

[gluxon]

Hello fellow pnpm users! 👋

I wanted to share pnpm/rfcs#1, which has a lot of the same ideas here. The RFC proposes a new workspace-consistent: protocol, which would behave similar to the prior mentioned "inherits" and "catalog:*" ideas. I only saw this issue today, but I find it cool how much of the ideas overlap.

The RFC adds a few features on top of a consistent root versions catalog.

• A change to the pnpm-lock.yaml using these consistent version placeholders can reduce pnpm-lock.yaml and package.json git merge conficts as these common versions are upgraded.
• Gives a mechanism for pnpm to know about and better enforce 1 version of a package transitively. (And not just direct project dependencies.)

The RFC got stalled a bit because of the complexity of the extra mile efforts above, but is something I'm still very invested in. Please feel free to drop ideas on the RFC or give suggestions.

(Disclaimer: I'm not a pnpm maintainer. Just a user that's submitted pull requests in the past.)

[mfliri]

I'm currently experiencing the same issue.
I would love to use @remorses solution but… #4388
I created a repo with the error here
Check this line and this one. Different references to the same library :(

[zkochan]

In Bit there is a concept known as environments (cc @GiladShoham). I think we can support something similar in pnpm. It would be not as powerful as in Bit but it could allow to sync dependencies between projects (even if they don't stay in the same project). For Bit it would also be beneficial as we would mention that the idea was borrowed from Bit and for more powerful envs people can use bit cli.

This is how I think it could work. In package.json we could add a new field. For instance:

{
  "name": "foo",
  "version": "1.0.0",
  "pnpm": {
     "environment": "@comp/react-env@1.0.0"
  }
}

This field would contain the exact name and version of a package called an "environment package". This package would be a regular package with dependencies and peer dependencies in its package.json. Then we could add some fields to inherit fields from this env package. For instance:

{
  "name": "foo",
  "version": "1.0.0",
  "pnpm": {
     "environment": "@comp/react-env@1.0.0",
     "dependenciesFromEnv": ["react"],
     "devDependenciesFromEnv": ["eslint"],
     "peerDependenciesFromEnv": ["react-dom"]
  }
}

pnpm would then lookup the versions of these dependencies in the env package.json and place them to regular dependencies fields. This is the package.json file of the @comp/react-env@1.0.0:

{
  "name": "@comp/react-env",
  "version": "1.0.0",
  "dependencies": {
    "react": "16",
    "eslint": "4",
    "react-dom": "17",
    "jest": "22"
  },
  "peerDependencies": {
    "react-dom": "17"
    "react": "16"
  }
}

In memory, this is out project's package.json that will be updated with selected dependencies from the env:

{
  "name": "foo",
  "version": "1.0.0",
  "dependencies": {
    "react": "16"
  },
  "devDependencies": {
    "eslint": "4"
  },
  "peerDependencies": {
    "react-dom": "17"
  },
  "pnpm": {
     "environment": "@comp/react-env@1.0.0",
     "dependenciesFromEnv": ["react"],
     "devDependenciesFromEnv": ["eslint"],
     "peerDependenciesFromEnv": ["react-dom"]
  }
}

[aphex]

@zkochan Heck ya now this is starting to look good. Could we use the the reference package name as a key though, so there could be multiple? Something like this.

{
  "name": "foo",
  "version": "1.0.0",
  "pnpm": {
    "environment": {
      "@comp/react-env@1.0.0": {
        "dependencies": ["react"],
        "devDependencies": ["eslint"],
        "peerDependencies": ["react-dom"]
      }
    }
  }
}

If someone adds the same dependency from two different reference packages I would say we just use a "last in wins" strategy.

[zkochan]

I don't know if it makes sense to support multiple. This is inspired by Bit, which supports one env per component. I think one env should be enough because you can create envs that inherit other envs.

[aphex]

I would think multiple would be handy in a larger monorepo when I want my front end projects to all share a vue version, but then a few of them share some packages with the backend or the root.

If I am following you would solve this by creating new packages in the mono repo that are just used as a environment for other packages? Just a collection of all the packages within the monorepo that should have consistent versioning?

[zkochan]

If I am following you would solve this by creating new packages in the mono repo that are just used as a environment for other packages? Just a collection of all the packages within the monorepo that should have consistent versioning?

yes, I think so

[aphex]

Alright I am following ya now. That's not bad. I think my issue is personal aesthetics, which clearly is not a solid reason to change anything haha.

Something about 4 sibling properties in an object all with "env" attached to them to indicate they are related vs keeping them all tucked away inside a "env" property. 🤷‍♂️

Either way getting a feature like this would help a lot, whichever way you prefer to configure it :)

[evelant]

@aphex it feels a little odd at first because in most setups there's a lot of friction to adding new packages in a monorepo but having separate packages for envrionments required by your other packages is exactly what Bit does. After spending some time with it I think I can say it's a very useful pattern, especially with Bit since it reduces that friction to near zero. Bringing some of that funcionality into pnpm IMO would be great.

[aphex]

@evelant No I get it, I am onboard and definitely want to see the functionality. If Bit has a solid pattern it should be followed. My aesthetic thoughts on the config design are not critical here.

[zkochan]

Something about 4 sibling properties in an object all with "env" attached to them to indicate they are related vs keeping them all tucked away inside a "env" property.

I don't know how this will be configured and how the properties will be named. It is just an example to explain the concept. Bit doesn't have such properties at all because it can automatically detect what packages from the env are used by the component.

[sneko]

Until there is an automatic way to find out duplicated packages I listed some steps that helped me: #6055 (reply in thread)

[dtothefp]

I've recently started to use PNPM in a monorepo and it solves a lot of problems but at the same time I'm experiencing this issue with singleton packages. I modified the afterAllResolved solution above and found that I have 392 duplicated dependencies and this is in a monorepo that is only moderate in size. For NodeJS projects maybe this is not a problem but for tools such as Eslint and for module bundlers like Webpack this presents issues. Is the only solution to manually find these duplicated dependencies and then specify the pnpm.overrides in package.json?

[neongreen]

I made a .pnpmfile.cjs to ensure some packages have only one version after every install [...]

@remorses's solution is great, but unfortunately only allows checking for the entire workspace. Sometimes you care about individual projects in the workspace having singleton packages (eg. @emotion), but it's ok for two different projects in the workspace to have different emotion versions.

[zkochan]

This feature is close to completion thanks to @gluxon. You can already try it using the branch from the next PR: #8122

Just place your versions in the catalog section of pnpm-workspace.yaml. For instance:

catalog:
  react: 16

Then in your package.json use the catalog: version:

{
  "dependencies": {
    "react": "catalog:"
  }
}

If you check it out please leave your experiences. We want to release it soon.

[remorses]

Catalogs don't really solve this problem, usually i get multiple instances of next for example when they depend on peer dependencies that resolve to different versions based on which workspace it is installed in (or if it is a transitive dependency based on which parent dependency).

For example when i update next i will also have to update all of it's peer dependencies and make sure pnpm doesn't duplicate the package in pnpm-lock.yaml

Here is an example of next getting duplicated because babel-plugin-macros resolves to different versions:

  next@14.2.4(@babel/core@7.24.6)(@playwright/test@1.43.1)(babel-plugin-macros@2.8.0)(react-dom@18.3.1(react@18.3.1))(react@18.3.1):
    dependencies:
      '@next/env': 14.2.4
      '@swc/helpers': 0.5.5
      busboy: 1.6.0
      caniuse-lite: 1.0.30001638
      graceful-fs: 4.2.11
      postcss: 8.4.31
      react: 18.3.1
      react-dom: 18.3.1(react@18.3.1)
      styled-jsx: 5.1.1(@babel/core@7.24.6)(babel-plugin-macros@2.8.0)(react@18.3.1)
    optionalDependencies:
      '@next/swc-darwin-arm64': 14.2.4
      '@next/swc-darwin-x64': 14.2.4
      '@next/swc-linux-arm64-gnu': 14.2.4
      '@next/swc-linux-arm64-musl': 14.2.4
      '@next/swc-linux-x64-gnu': 14.2.4
      '@next/swc-linux-x64-musl': 14.2.4
      '@next/swc-win32-arm64-msvc': 14.2.4
      '@next/swc-win32-ia32-msvc': 14.2.4
      '@next/swc-win32-x64-msvc': 14.2.4
      '@playwright/test': 1.43.1
    transitivePeerDependencies:
      - '@babel/core'
      - babel-plugin-macros

  next@14.2.4(@playwright/test@1.43.1)(babel-plugin-macros@3.1.0)(react-dom@18.3.1(react@18.3.1))(react@18.3.1):
    dependencies:
      '@next/env': 14.2.4
      '@swc/helpers': 0.5.5
      busboy: 1.6.0
      caniuse-lite: 1.0.30001638
      graceful-fs: 4.2.11
      postcss: 8.4.31
      react: 18.3.1
      react-dom: 18.3.1(react@18.3.1)
      styled-jsx: 5.1.1(babel-plugin-macros@3.1.0)(react@18.3.1)
    optionalDependencies:
      '@next/swc-darwin-arm64': 14.2.4
      '@next/swc-darwin-x64': 14.2.4
      '@next/swc-linux-arm64-gnu': 14.2.4
      '@next/swc-linux-arm64-musl': 14.2.4
      '@next/swc-linux-x64-gnu': 14.2.4
      '@next/swc-linux-x64-musl': 14.2.4
      '@next/swc-win32-arm64-msvc': 14.2.4
      '@next/swc-win32-ia32-msvc': 14.2.4
      '@next/swc-win32-x64-msvc': 14.2.4
      '@playwright/test': 1.43.1
    transitivePeerDependencies:
      - '@babel/core'
      - babel-plugin-macros

It would be great if there was a way to tell pnpm to install next only once and ignore different peer dependencies resolutions

[zkochan]

🚢 9.5.0-beta.0

@remorses your issue is unrelated to this issue.