Add optional snapTo property in versionGroups

[bombillazo]

Description

Perhaps this is doable with the current configuration options but seems like it is not:

I have an autogenerated package.json that automatically sets some versions to "*". I want to run fix-mismatches such that the version set on all files matches the highest none "*" version.

Suggested Solution

Have a new option on fix-mismatches that replaces all deps with "*" with the latest semver range found. At the moment I am getting my wanted behavior by commenting out the if (raw === '*' || rawHighest === '*') check in the getHighestVersion function.

[JamieMason]

Hey Hector, could you go into some more detail on what the overall task you're doing here is? It helps if I understand the wider system, right now I would be considering exposing a new piece of config to users but not be able to explain to them why it exists, when to use it, and what it's for etc.

Thanks.

[bombillazo]

Hey, sure thing. In my specific case, some package.json files are modified by code generators within a monorepo, always generating dependencies with "*". I have other package.json files that have specific versions, so I would like to ignore "*" as the highest valid version and instead use the specific semver found in the other files. This goes a bit in hand with #88 since it would be also useful to specify a file from which some or all package.json files must adhere to if "*" is found.

[JamieMason]

OK I think I get it, I'll think about it for a while but right now I'm thinking maybe, a bit like pinVersion, there is an optional snapTo property of versionGroups which causes this behaviour. We'd need to think it through but on the face of it it seems like it could work.

[JamieMason]

I think #87 and #88 might potentially be duplicates and we have one issue to track this one thing which handles your use case? Or are there more than one use case?

[bombillazo]

I think we can track it under (this) one

[bombillazo]

@JamieMason Sorry to hi-jack this to ask: is there a way to ignore certain dependencies from being matched during fix-mismatches? Like keeping the version present in their respective package.json.

[bombillazo]

In the case of snapTo, I'm thinking it can be an array of packages ranked by priority (first has more precedence). A dependency that matches the versionGroup will "snap" their version to the version found in the first package in the snapTo array that has that dependency specified.

[JamieMason]

@JamieMason Sorry to hi-jack this to ask: is there a way to ignore certain dependencies from being matched during fix-mismatches? Like keeping the version present in their respective package.json.

This should be possible with versionGroups, if you create a versionGroup containing only these versions you want treating differently, they'll be isolated from other groups. If it's a group containing only one package, there'll be no other packages it could mismatch with.

In the case of snapTo, I'm thinking it can be an array of packages ranked by priority (first has more precedence). A dependency that matches the versionGroup will "snap" their version to the version found in the first package in the snapTo array that has that dependency specified.

was this related to your previous comment or this issue? I'm a bit lost.

[bombillazo]

What I mean was having somthing like this:

{
"versionGroups": [
    {
      "dependencies": ["@myDeps/**"],
      "snapTo": ["native-app", "web-app"],
      "packages": ["**"]
    }
  ]
}

This will sync all @myDep/** versions within all packages to the version found in the native-app package.json first, if not found there, it'll check in the web-app package.json, else then by the regular highest version.

That is what I meant earlier by the "driver" file, the root package.json is driving the versions for the rest of the packages, regardless if it has a higher version or not.

* EDIT *
Update example for clarity.

Im assuming passing "" or a special keyword could be used to snap those dependencies to the root package.json.

{
"versionGroups": [
    {
      "dependencies": ["@myDeps/**"],
      "snapTo": ["native-app", "", "_ROOT_"],
      "packages": ["**"]
    }
  ]
}

[JamieMason]

Although I've never been in a situation that has needed it, I could understand a single file being used as a source of truth of some sort and some kind of snapTo property might be able to support that if we decide to go ahead.

But:

1. Why would you have an array?
2. What does the repo you're working on look like?
3. What's the workflow problem you have?
4. How does this feature potentially fix it?
5. What is it about the packages in that array that makes those a point of reference for the others, version wise?
6. Tell me a bit about the package which is 2nd in the array, why is that one 2nd and not first?

I'm generally just looking to understand how all of this fits together. Right now this seems like it could be a niche problem specific to your stack, but let me know.

[bombillazo]

Sure,

1. An array would be to specify more than one file as fallbacks to the main "driving" file. If it was an empty array syncpack would behave like it now does. More than one and a loop is started to pin the version to the version found in the first file that contains that package.
2. It is an Nx monorepo with very different types of apps within. I'm talking backend APIs, Front end Web apps, React Native (Expo) app, + internal packages, and libraries. All project dependencies should be set in the root level package.json, BUT installed from the app level package.json. Since they are installed from within each app folder via toolings and clis, the version in this folder will be the true correct version that needs to be synced out to the root package. Both versions and deps need to be synced (any package existing inside an app folder must exist in the root package.json, I am taking care of dependency syncing already and using syncpack for version syncing).

I should note that none of this would be necessary IF watchman (used by React Native) supported symlink, which is what yarn workspaces uses to connect and share dependencies across monorepos.

3. React Native with Expo is very contingent on specific dependency versions to work and build properly. One single incorrect version can turn the build and troubleshooting process into a real nightmare. I want to force my whole monorepo to respect those dependency versions first over all other apps. Keeping track manually of all the versions supported and which are being used to set them globally is very error-prone and tedious.
4. This would set my RN (and potentially other apps) as the "setters" of the versions instead of us manually specifying them via configuration.
5. These packages are repo-level tools or app-shared packages that need to have the same versions to ensure consistency and compatibility.
6. In that case it is a fallback if the version is not found in the first package. This could be useful for example if I set this:

"snapTo": ["native-app", "web-app"],

Here any version found in the native app has priority as the "correct" version. If not specified there, the "web-app" version is the "correct" version. For example, imagine both apps share react, then the react version set by the native app should be used across all. But if react-foo only exists in web-app, that version is used across. If not found anywhere, the regular rules apply. This way the root level package.json + app package.jsons is set with the correct versions.

[JamieMason]

Great explanation, thanks. Leave it with me a while, I've a much better idea of what's going on now.

[bombillazo]

Thank you so much for your consideration and interest! Looking forward to what you can come up with 😀

[JamieMason]

Released in 9.7.4

Docs: https://jamiemason.github.io/syncpack/config/version-groups#snapto-string

/cc @javascripter @chbdetta (👍🏻'd this issue)

[bombillazo]

EPIC! Thank you so much for your work.

[JamieMason]

You're welcome, let me know if it works as you'd expect.

[pastelsky]

@JamieMason If I understand correctly, the snapTo to reference root package.json is to work as follows?

{
      // Snapped
      "label": "snap everything to root package.json",
      "dependencies": ["**"],
      "packages": ["**"],
      "snapTo": ["root-package-name"]
    },

However, this does not seem to work with the latest version of syncpack either and I run into —

versionGroup.snapTo does not match any package versions

errors. Is the root workspace made available to snapTo?

[JamieMason]

I'm not sure this is it but try adding "dependencyTypes": ["**"], as well. Otherwise do you have a repro I could look at please? Could be a bug

[pastelsky]

So upon adding logs, I figured the error is thrown for local dependencies that don't have a version in root package.json — which is expected.

Shouldn't snapTo in that case gracefully skip local dependencies?

I also tried adding a dependencyTypes filter, which didn't seem to have an effect —

{
      // Snapped
      "label": "snap everything to root package.json",
      "dependencies": ["**"],
      "dependencyTypes": ["!local"]
      "packages": ["**"],
      "snapTo": ["root-package-name"]
}

[JamieMason]

Thanks a lot @pastelsky that's useful, could you create a new issue to track this please? and I'll take a look

[JamieMason]

@pastelsky that issue where a version is missing should be handled now in 12.0.0-alpha.0

[silvaj8]

@JamieMason I'm testing the newest version 12.0.0-alpha.0 and I'm still facing the issue @pastelsky referred above (if I understood it correctly). If you prefer I can create a separate issue for this, but since this is related to this thread of comments and related to the testing of an alpha version, I thought that discussing it here would suffice.

Description

I need all local packages to use whatever version some specific local packages (a.k.a. "apps") are using of the local packages. And for the other local packages (that are not used by any "app"), I need them to use workspace protocol (workspace:*). So, I approached it like this:

{
  label: "Ensure all packages use whatever version the apps are using",
  packages: ["**"],
  dependencies: ["$LOCAL"],
  dependencyTypes: ["**"],
  snapTo: ["appA", "appB", "appC"],
},
{
  label: "Ensure all packages use the workspace protocol for the local packages",
  packages: ["**"],
  dependencies: ["$LOCAL"],
  dependencyTypes: ["!local"],
  pinVersion: "workspace:*",
},

Output

But this causes an error when a local package is not used by an app, something like this:

! failed to get snapped to version for packageX using ["appA","appB","appC"]
✘ packageX 0.7.1 → ??? packages/packageX/package.json > version [missing snapTo version]
  no package in this groups .snapTo array depend on packageX
✘ packageX workspace:* → ??? packages/packageY/package.json > devDependencies [missing snapTo version]
  no package in this groups .snapTo array depend on packageX
✘ packageX workspace:* → ??? packages/packageZ/package.json > devDependencies [missing snapTo version]
  no package in this groups .snapTo array depend on packageX

Expected output

I was expecting for snapTo to gracefully skip the local packages that are not used by an app, so I don't need to enumerate each local package used inside the apps on the dependencies array, which is my current workaround.

Context

I can't share any repo, but I will try to describe my context in the best and simplest way possible to check if I'm approaching my necessity incorrectly. In my shared monorepo I have almost 30 packages and 3 "apps", and I use the workspace protocol for every package, with some exceptions in the apps. These apps have some particularities:

• They are not like packages, as they are not to be used by other packages;
• They can use any of the local packages, but they cannot use the workspace protocol, only semVer (for some irrelevant reason that I can't control related to NPM vs Yarn). And the same applies for the packages that the apps use, it's recursive. Example: appA uses packageA, and packageA uses packageB, so in the package.json of appA, packageA needs to use semVer, and in the package.json of packageA, packageB also needs to use semVer. In summary, we can't use the workspace protocol in the package.json of the apps and in its dependencies' package.json;
• Since we want to use the same version for the same local package across the monorepo, these packages will have to use semVer even in packages that are not used by an app.