Feature Request: Flag to work with yarn workspace

[expelledboy]

I am trying to use this with firebase-tools in a yarn monorepo.

To solve the issue where firebase assumes I have @org/common pointing directly to the workspace files:../common.

$ yarn generate-lockfile --package ./package.json --lockfile ../../yarn.lock
Error: Error: Could not find: @org/common@file:../common
✨  Done in 0.22s.

[ajainvivek]

The issue seems to be around symlink packages.

[varsis]

@expelledboy I see the issue.

Currently the cli is attempting to resolve all dependencies in the package.json from the provided yarn.lock file. However as this is using yarn workspaces there is no entry and it is only sym linked.

If I add an option or flag would the expected behaviour be to ignore any workspace items? This would remove the locking for that particular package.

[expelledboy]

So I have run some tests. Indeed the workspace is not locked in top level yarn.lock. If we assume this monorepo behavior I would have said yes.

However what we are trying to achieve is generating a lock file as if not in a workspace, so are file://... entries are locked ordinarily? Upon a quick test

/tmp ⌚ 11:46:07
$ mkdir -p /tmp/pkg; cd $_
Found existing alias for "mkdir -p". You should use: "md"

/tmp/pkg ⌚ 11:46:23
$ yarn add file://$HOME/repos/betaplum/che-cha/packages/types
yarn add v1.22.5
info No lockfile found.
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning "file:///Users/anthony/repos/betaplum/che-cha/packages/types > @firebase/firestore-types@2.1.0" has unmet peer dependency "@firebase/app-types@0.x".
[4/4] 🔨  Building fresh packages...

success Saved lockfile.
success Saved 2 new dependencies.
info Direct dependencies
└─ @che-cha/types@1.0.0
info All dependencies
├─ @che-cha/types@1.0.0
└─ @firebase/firestore-types@2.1.0
✨  Done in 2.37s.

/tmp/pkg ⌚ 11:46:26
$ cat yarn.lock
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1


"@che-cha/types@file:///Users/anthony/repos/betaplum/che-cha/packages/types":
  version "1.0.0"
  dependencies:
    "@firebase/firestore-types" "^2.1.0"

"@firebase/firestore-types@^2.1.0":
  version "2.1.0"
  resolved "https://registry.yarnpkg.com/@firebase/firestore-types/-/firestore-types-2.1.0.tgz#ad406c6fd7f0eae7ea52979f712daa0166aef665"
  integrity sha512-jietErBWihMvJkqqEquQy5GgoEwzHnMXXC/TsVoe9FPysXm1/AeJS12taS7ZYvenAtyvL/AEJyKrRKRh4adcJQ==

Looks like there are. So you would need to inspect the version pointed to by the file://..?

[varsis]

@expelledboy I don't think pointing to a local file is the normal use case here. I will see what I can pull together in a sample repo that may work for this, but there are a few edge cases I am worried about. For example:

Do we want to resolve to the version at install time or the version that is in the repo when we run our tests / build each time? Or do we expect some other behaviour?

[expelledboy]

Well as we dont actually have the ability to "use" a locked version I would say when you run generate-lockfile. Also I am not sure what you mean by install vs tests/build?

[eric-burel]

Hey folks, any idea if this package should work these days in a monorepo? That's still the best solution to yarnpkg/berry#1223 and yarnpkg/yarn#5428, there is also this package but a big too complex: https://github.com/JanVoracek/yarn-plugin-entrypoint-lockfiles
It's becoming relevant again as monorepo with Yarn 3 workspaces are getting popular. I intend to use this plugin to generate a yarn.lock for a starter app from the root yarn.lock.

[varsis]

@eric-burel I'm currently using this in a mono repo. That said each item is bundled / packaged locally into a single file using rollup. This package is to create a lock file for external dependencies.

That said if you wanted to "lock" in a local package you need to either publish it and install it locally or zip your local version and install using a file path to the zip.

Is the use case here to lock in the dependencies in the local package?

[eric-burel]

This package is to create a lock file for external dependencies.

Glad that you still use it then!
My use case is as follow: my monorepo contains reusable NPM packages, no problem with that. But it also contains a Remix app and a Next app, that I publish in a separate read-only repo using git subtree.
The corresponding folders in my monorepo must not have a yarn.lock, because otherwise Yarn treats them as independent project and won't be able to use my local reusable packages.
But I need to publish a yarn.lock in the subtree version, so that when people download those apps they have the same version as I do in the monorepo.

So my idea is to generate yarn.remix.lock for instance just for the Remix app, and store it in the folder. Then I can add a presinstall script or whatever that renames it yarn.lock when people actually use this app.

[varsis]

This package is to create a lock file for external dependencies.

Glad that you still use it then! My use case is as follow: my monorepo contains reusable NPM packages, no problem with that. But it also contains a Remix app and a Next app, that I publish in a separate read-only repo using git subtree. The corresponding folders in my monorepo must not have a yarn.lock, because otherwise Yarn treats them as independent project and won't be able to use my local reusable packages. But I need to publish a yarn.lock in the subtree version, so that when people download those apps they have the same version as I do in the monorepo.

So my idea is to generate yarn.remix.lock for instance just for the Remix app, and store it in the folder. Then I can add a presinstall script or whatever that renames it yarn.lock when people actually use this app.

@eric-burel You should be able to do this with the --write flag

[eric-burel]

Hi,

I am htting an error:

yarn generate-lockfile --lockfile ./yarn.lock --package ./starters/remix/ --write --dev
Error: SyntaxError: Unknown token: { line: 3, col: 2, type: 'INVALID', value: undefined } 3:2 in lockfile

You can reproduce by running this command in this repo: https://github.com/VulcanJS/vulcan-npm (after a yarn install).

It seems related to the yarn.lock not being readable, at least the first few lines:

# This file is generated by running "yarn install" inside your project.
# Manual changes might be lost - proceed with caution!

__metadata:
  version: 6
  cacheKey: 8

"@actions/core@npm:^1.5.0":

I am using Yarn 3 so maybe the structure changed over time

[varsis]

Hi,

I am htting an error:

yarn generate-lockfile --lockfile ./yarn.lock --package ./starters/remix/ --write --dev
Error: SyntaxError: Unknown token: { line: 3, col: 2, type: 'INVALID', value: undefined } 3:2 in lockfile

You can reproduce by running this command in this repo: https://github.com/VulcanJS/vulcan-npm (after a yarn install).

It seems related to the yarn.lock not being readable, at least the first few lines:

# This file is generated by running "yarn install" inside your project.
# Manual changes might be lost - proceed with caution!

__metadata:
  version: 6
  cacheKey: 8

"@actions/core@npm:^1.5.0":

I am using Yarn 3 so maybe the structure changed over time

@eric-burel I took a small look at support v3 but I was unable to get it fully working in the limited time I had. If you want to take a look / contribute to the WIP branch go for it -> https://github.com/varsis/generate-lockfile/pull/9/files

It's currently able to resolve the majority of the dependencies but I am still missing a few of them from the lockfile. Maybe they can be ignored?

For the future of this project / idea I think it would be better suited as a yarn plugin (https://yarnpkg.com/features/plugins)

[eric-burel]

If you plan to create a Yarn plugin, maybe it would be better to contribute to: https://github.com/JanVoracek/yarn-plugin-entrypoint-lockfiles

It mostly works, except that the generated lock seems slightly "out of date" or off, it doesn't work with frozen-lockfile.

See yarnpkg/berry#1223 (comment) for more details, and I demo a setup here: https://github.com/VulcanJS/vulcan-npm/pull/132/files