Support yarn v2

[oliversalzburg]

In a repository using yarn 2, vsce package will fail trying to enumerate dependencies:

 ERROR  Command failed: yarn list --prod --json

To reproduce, in an extension repository, use:

yarn set version berry
yarn install
vsce package

[joaomoreno]

What does yarn list --prod --json return?

Can I clone your repo and try it out?

[oliversalzburg]

It returns Usage Error: Couldn't find a script named "list"., as it's not a valid yarn2 command: https://yarnpkg.com/cli/

My reproduction should work in any extension repository, but I created a fresh one that should showcase it perfectly: https://github.com/oliversalzburg/vsce-yarn2-support

[SheepReaper]

Likewise, yarn 2 omits the run command since it's not needed. Yarn 2 is not meant to be installed globally (yet) If there is yarn available in global, then it's probably v1. I don't know if vscode is running an embedded yarn or global yarn, but ordinarily yarn v1 checks if yarn 2 (or any yarn) is installed in the repository and runs it instead.

In my case, running an npm script from the side panel results in this:

The terminal process "C:\Program Files\PowerShell\7\pwsh.exe -Command yarn run dev" terminated with exit code: 1.

Which is incorrect, it should just be yarn dev

[jewlexx]

I had the same issue, here was my temporary fix, although it can be annoying,

1. Optional: Checkout a new temp branch so that package-lock.json and other npm files to carry over
2. Run npm i
3. Run vsce package --no-yarn

[latipun7]

just info, as of now yarn list still not implemented in yarn berry: https://yarnpkg.com/getting-started/migration#not-implemented-yet

[SchoofsKelvin]

Not the cleanest/best solution, but this is how I circumvented this issue:

• Configure your .yarnrc.yml with a lockfileFilename different from yarn.lock
• Run vsce with the --yarn parameter (since it won't detect Yarn using the yarn.lock anymore)

This will make vsce skip its whole process of looking at your dependencies and partially including node_modules in the packaged extension. Skipping this process also skips its internal yarn list (and because of --yarn also npm ls) which prevents this error from happening.

The only issue with this is that vsce might bundle some items in node_modules if it isn't in .vscodeignore or something similar. You'd lose that "bundling" of course (which you would lose anyway if using Yarn PnP), but if your extension doesn't have any dependencies or you bundle it using e.g. Webpack, this is a non-issue.

Of course, having vsce support Yarn 2 natively is still better.

[itspngu]

With the freeze of yarn v1 this should perhaps be re-prioritized.

[YXL76]

With the freeze of yarn v1 this should perhaps be re-prioritized.

See #439. I think yarn v2 is supported.

[ssbarnea]

This ticket is already two years old and we still do not have full support for yarn 2.

For example vsce ls command is fully broken because:

$ npx vsce ls
 INFO  Detected presence of yarn.lock. Using 'yarn' instead of 'npm' (to override this pass '--no-yarn' on the command line).
 ERROR  Command failed: yarn list --prod --json

There is no list in yarn 2 and if you try to pass --no-yarn to vsce, you endup with the other problems and failures.

I guess that for the moment I will need to replace vsce ls with zipinfo -1 *.vsix.

[oliversalzburg]

In general, it seems like very poor practice to rely on a specific third party shell application to perform anything of meaning in a NodeJS project, especially when that third party application is a package manager.

Which is why I have given up on vsce a long time ago. Please open a new issue to track the total absence of progress on this issue, if you need to.

[joaomoreno]

Contributions here are more than welcome! Yarn v2 just isn't that big of a deal for us and most of our users (only 8 upvotes here), so this issue never makes it to the front page. Also, and I'm sure you know, we're constantly busy with plenty of other work.

[ssbarnea]

@joaomoreno Thanks for clarifying as not everyone understands what is expected to be fixed by core team and what goes into welcoming fixes from outside. Some might get frustrated waiting for something to happen that nobody is working on.

At least knowing that the core team does not see yarn v2 as a priority, could motivate some to propose fixes. I fixed one yarn bug last week and I may have to fix this one too.

I did check the outcome of vsce package with yarn v2 and the extension seem to be correctly built, so the incompatibilities are kinda minor, more around on the missing or wrong command line arguments.

I was recently forced to switch to yarn because npm link-ing was never fixed and apparently they are not going to do it. One extra benefit was that operations seem to be like 5x faster if not even more and managing dependencies become much easier. I really do not want to go back to npm.

And yep, I am closely watching code team release notes each month, I am giving them as a good example on how to deliver fast and also inform users about what changed. Great work!

[oliversalzburg]

If you need to keep track of your lack of interest to work on this subject, please create your own internal ticket. I don't want to be bothered by activity on this repository.

[ssbarnea]

@oliversalzburg You already mentioned that you are no longer interested in following it up and you could have just pressed the unsubscribe button. That is an open-source project and it is not nice to undo maintainers work, I even seen people being temporarily banned for not playing nice.

Regarding broken ls command, the workaround is to use npx vsce ls --no-dependencies --no-yarn, which seems to be working with yarn2.

[joaomoreno]

@oliversalzburg, please unsubscribe.

[arendjr]

I also ran into this issue, but resolved it by creating a Yarn plugin that implements the list command. I have only tested this on our own repository, so I may not have covered all edge cases, but it works for us.

See: https://gist.github.com/arendjr/415747652a8c79f12b005ce3cfb2f808

If there is interest in contributing this to Yarn itself, please let me know.

[lordrip]

Hi @arendjr, thanks for sharing your plugin. I think that it's a good idea of having your plugin listed on the yarn website.

How can I help to make this? I guess that we would need to put the plugin in a dedicated package perhaps.

[lordrip]

Hi @joaomoreno, from your point of view, how would you envision this issue moving forward? I might be lacking context but I think that having @arendjr plugin installed on yarn berry repositories it's good and probably it will evolve to support the previous use cases from yarn list from yarn v1, but at the same time, maybe vscode-vsce shouldn't depend on the users installing the plugin but rather have this functionality embedded.

Maybe another way to handle it could be listing @arendjr's plugin as a prerequisite of vscode-vsce for supporting yarn berry.

edit: I'm happy to help, I just wanted to understand what would be the best approach to follow 😄

[ivangabriele]

Thank you so much for working on this @lordrip :). Fyi, it seems that yarn list has simply be renamed/restructured to yarn info --recursive in case that helps you with your WIP.

[lordrip]

Hi @ivangabriele I haven't worked on this yet, I would like to make sure that this is still needed first. Thanks for the heads-up about the renaming, I gave it a try and it seems that the JSON output is not exactly right:

Using:

yarn info --recursive --json

this is the output, notice how it will need a little bit of adjustment before using it:

{
    "value":"@4tw/cypress-drag-drop@npm:2.2.4",
    "children":{
        "Instances":1,
        "Version":"2.2.4"
    }
}{
    "value":"@adobe/css-tools@npm:4.2.0",
    "children":{
        "Version":"4.2.0"
    }
}{
    "value":"@ampproject/remapping@npm:2.2.1",
    "children":{
        "Version":"2.2.1",
        "Dependencies":[
            {
                "descriptor":"@jridgewell/gen-mapping@npm:^0.3.0",
                "locator":"@jridgewell/gen-mapping@npm:0.3.3"
            },
            {
                "descriptor":"@jridgewell/trace-mapping@npm:^0.3.9",
                "locator":"@jridgewell/trace-mapping@npm:0.3.18"
            }
        ]
    }
}

[ivangabriele]

@lordrip Indeed I didn't check in detail how it's used yet. Normally I think that could be a one-shot fix by updating this function and detect Yarn 2/3 via the .yarn directory existence.

I can at least confirm I had the same issue today on a brand new extension using Yarn stable (v3.6.1).

In the meantime I used @arendjr script.

I can try to poke a PR if you wish ^^.

[lordrip]

Sure, I'm not the maintainer but I'm confident that it will help a lot.

If you happen to submit a pull request I can lend you a hand if needed.

If it helps, I adjusted a bit @arendjr 's plugin here to incorporate the yarn link workflow as well

[arendjr]

Would it help if I moved my script into its own repository?

[lordrip]

Hi @arendjr, as I mentioned in a few comments above, I think that there's value in trying to publish your script in the yarn plugin list.

Plenty of plugin examples exist, for instance, yarn-plugin-env-vars.

That being said, I don't think that both efforts are mutually exclusive, at least how I see it, it would be quicker to have a public plugin, installed through the CLI, like:

yarn add arendjr-plugin

while we wait for a dedicated implementation in this repository, which probably would be the best approach in the long run 😃

EDIT: I took your cool script and adjusted it a little bit in order to support linked packages, the ones that you get when you do a:

yarn link ~/my-package

[arendjr]

Alrighty, this should do the trick :) https://github.com/arendjr/yarn-plugin-list

I have incorporated your support for Yarn resolutions as well, @lordrip

[ivangabriele]

Thanks again here! Fyi I will try to open a PR-fix today or tomorrow for this issue.

[anbnyc]

I was blocked by this issue for a while, but in case it is helpful to others, my workaround was to use esbuild to bundle, then pass --no-dependencies to my package command. This skips the outdated yarn list command. Credit for this idea: #421 (comment)

For emphasis:

there is no need to include npm dependencies when package and publish, all things is in your bundle

As an additional aside, it looks like the compatibility issue looks like it was introduced in v2.9.2: https://github.com/microsoft/vscode-vsce/releases/tag/v2.9.2. If you are OK using an earlier version of vsce, this might help too.

[joaomoreno]

Closing this as out of scope. As part of our recurring maintenance milestone, we've decided to limit the amount of supported package managers to npm and yarn v1 only.

That being said, I'm glad workarounds exist for anyone who is blocked.

[BasixKOR]

Have you considered using an off-the-shelf solution for resolving Node.js dependencies instead? It seems like the package manager-dependent code is just used for listing which node_modules should be put into the VSIX package, and I think that could be done by using something like https://github.com/vercel/nft

I think resolving dependencies from the code would lead to better accuracy (since you cannot be so sure all the package maintainers are managing devDependencies and dependencies well) and better interoperability overall without juggling over every package manager VSCE may need to support.