feat(groups): target instances by specifier type

[moltar]

Description

Our use case is a bit of an edge case, and is not a problem with syncpack. But I am wondering if a solution could be introduced as a feature to deal with this.

A tool we use for repo management has a workflow that goes like this:

1. The user adds a dependency to the tool config (not using npm install foo)
2. Tool renders package.json file with the new dependency using * as a placeholder.

"dependencies": {
  "my-package": "*"
}

3. Tool automatically runs {pnpm,npm,yarn} install, to install the * version, which is a SemVer way of saying "latest".
4. Once the dep is installed, then tool runs a function to get the actual version of the installed package. In essence, all it does is require.resolve('my-package') to get the package location and then reads package.json to get the version.
5. Finally, once the tool knows the actual version of the package, it rewrites our package.json again, and sets the actual version.

"dependencies": {
  "my-package": "^1.2.3"
}

6. Tool runs install once again to update the lock files.

This can be also reproduced and visualized without a particular tool, with just npm and CLI:

## start a new package
npm init -y

## add `lodash@*` dep without installing it
npm pkg set 'dependencies.lodash'='*'

## runs install
npm i

## update `package.json` to list _actual_ version, instead of `*`
npm pkg set 'dependencies.lodash'=$(npm ls --depth=0 lodash --json | jq -r '.version')

## run install again, to update `package-lock.json`
npm i

The problem occurs because of the way we use syncpack. It runs as postinstall script. And there's no way to discriminate the first install from the second install. So it runs twice.

And when it runs the first time, the version is still *. Syncpack will set this version across the whole workspace (monorepo). Correctly so, of course, as it is ultimately the highest version, as far as SemVer is concerned.

But in our case, it is only transient, but it affects all other packages.

The temporary fix for us was to set preferVersion to lowestSemver, but this introduces other problems, that I won't get into, as they are not really important here.

Suggested Solution

I'd like to be able to ignore certain SemVers, notably *.

Help Needed

N/A

[JamieMason]

Something roughly like this could potentially be possible using some new internals coming soon.

For a totally made up example the new functionality might look something like this on a versionGroup:

{
  "versionGroups": [
    {
      # for any dependency (react, lodash etc)
      "dependencies": ["**"],
      # in devDependencies and peerDependencies only
      "dependencyTypes": ["dev", "peer"],
      # in every package in the monorepo
      "packages": ["**"],
+     # using version specifier "*" only
+     "specifierTypes": ["latest"],
      # ignore them
      "isIgnored": true
    }
  ]
}

The internal types which could be exposed as possible values are:

• Alias
• Delete
• File
• HostedGit
• Latest
• Range
• Tag
• Unsupported
• Url
• Version
• WorkspaceProtocol

In your case though, especially because this will take a while, might you be better to set and check some environment variables to help decide when to run certain tasks or not?

Thanks a lot for the detailed issue.

Finally, if you or anyone reading this finds syncpack useful, please tell people about it – it's completely free and has been a ton of work, please help us spread the word – thanks!

[moltar]

I like the specifierTypes approach! 👍🏼

might you be better to set and check some environment variables to help decide when to run certain tasks or not

I did look into this, but did not find a way. The tool does not do this. And otherwise, there is no way to distinguish one npm i from another npm i.

My simplified example using CLI is almost literally all that the tool does under the hood.

Imagine if my CLI example was wrapped into a single shell script, that you have no influence of.

The only changes I could make are to our environment, e.g. edit package.json in some way, or not run syncpack in postinstall.

Finally, if you or anyone reading this finds syncpack useful, please tell people about it – it's completely free and has been a ton of work, please help us spread the word – thanks!

Thank you for all your hard work, Jamie. I am certainly talking about the tool :)

[moltar]

One way I could achieve this, perhaps, is to read every pacakge.json and if any version is set to * then skip syncpack run.

Something along the lines of:

check-versions && syncpack || true

[JamieMason]

Checking if the output of syncpack list contains a *? Yeah that could work. I would tweak that example slightly though as I think in the next version I might have changed it to output help instead of running list by default:

- check-versions && syncpack || true
+ check-versions && syncpack list || true

EDIT: I see what you mean now, check-versions will do that.

[moltar]

Right, my example was oversimplified, we run it with exact command and config file flag and all that 😁

[JamieMason]

Released in 12.2.0.

👋 If anyone reading this finds syncpack useful, please tell people about it.