1.35.3 breaks when using yarn 3

[dhmw]

Describe the bug

Updating to 1.35.3 does not work with yarn 3. Specifically, I am using 3.1.1.

1.35.2 works as expected.

To Reproduce

Use serverless-esbuild 1.35.3 or greater with a project using yarn 3.1.1

Expected behavior

Build completes successfully

Screenshots or Logs

Packaging [REDACTED] for stage test (eu-west-1)
Warning: No external modules needed
Environment: linux, node 18.14.0, framework 3.27.0 (local), plugin 6.2.3, SDK 4.3.2
Credentials: Local, environment variables
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
Error: Command failed with exit code 1: yarn list --json --production
Usage Error: Couldn't find a script named "list".

$ yarn run [--inspect] [--inspect-brk] [-T,--top-level] [-B,--binaries-only] <scriptName> ...
    at makeError (/home/[REDACTED]/node_modules/execa/lib/error.js:60:11)
    at handlePromise (/home/[REDACTED]/node_modules/execa/index.js:118:26)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Yarn.getProdDependencies (/home/[REDACTED]/node_modules/serverless-esbuild/dist/packagers/yarn.js:39:35)
    at async EsbuildServerlessPlugin.pack (/home/[REDACTED]/node_modules/serverless-esbuild/dist/pack.js:116:53)
    at async before:package:createDeploymentArtifacts (/home/[REDACTED]/node_modules/serverless-esbuild/dist/index.js:73:17)
    at async PluginManager.runHooks (/home/[REDACTED]/node_modules/serverless/lib/classes/plugin-manager.js:530:9)
    at async PluginManager.invoke (/home/[REDACTED]/node_modules/serverless/lib/classes/plugin-manager.js:563:9)
    at async PluginManager.run (/home/[REDACTED]/node_modules/serverless/lib/classes/plugin-manager.js:604:7)
    at async Serverless.run (/home/[REDACTED]/node_modules/serverless/lib/serverless.js:179:5)
    at async /home/[REDACTED]/node_modules/serverless/scripts/serverless.js:812:9

Versions (please complete the following information):

Environment: linux, node 18.14.0, framework 3.27.0 (local), plugin 6.2.3, SDK 4.3.2

serverless-esbuild any >= 1.35.3

[lukevers]

hey @track0x1 are you seeing this error now? It's odd that I didn't last night when I tried your branch, but now after trying the updated version bump I get this one. It does make sense as an error because it looks like yarn list does not exist as command (see not implemented yet).

I was able to tinker with this a bit to get it in a better state, but the major problem here is the entire tree and finding function will need to be updated because the structure is much different.

So there is yarn info which is similar but seems to be a different structure (I'm not sure the v1 structure, I just saw it in the type here). The second problem is yarn info returns the data as an NDJSON stream, so what I did was basically just split and slurp it up like if I was using yarn info --json | jq -s . With some minor tweaks I got this:

diff --git a/src/packagers/yarn.ts b/src/packagers/yarn.ts
index 1df808f..8924f24 100644
--- a/src/packagers/yarn.ts
+++ b/src/packagers/yarn.ts
@@ -71,7 +71,11 @@ export class Yarn implements Packager {
 
   async getProdDependencies(cwd: string, depth?: number): Promise<DependenciesResult> {
     const command = /^win/.test(process.platform) ? 'yarn.cmd' : 'yarn';
-    const args = ['list', depth ? `--depth=${depth}` : null, '--json', '--production'].filter(isString);
+    const version = await this.getVersion(cwd);
+    const args = (version.isBerry
+      ? ['info', '--json', depth ? `--depth=${depth}` : null]
+      : ['list', depth ? `--depth=${depth}` : null, '--json', '--production']
+    ).filter(isString);
 
     // If we need to ignore some errors add them here
     const ignoredYarnErrors: Array<{
@@ -84,7 +88,12 @@ export class Yarn implements Packager {
     try {
       const processOutput = await spawnProcess(command, args, { cwd });
 
-      parsedDeps = JSON.parse(processOutput.stdout) as YarnDeps;
+      if (version.isBerry) {
+        const rows = processOutput.stdout.split(/\n|\n\r/).filter(Boolean);
+        parsedDeps = rows.map((row: string) => JSON.parse(row));
+      } else {
+        parsedDeps = JSON.parse(processOutput.stdout) as YarnDeps;
+      }
     } catch (err) {
       if (err instanceof SpawnError) {
         // Only exit with an error if we have critical npm errors for 2nd level inside

And this fails when we start to parse the trees because there are no data attributes named "trees" in the data. I ran mine on this and then plopped it into here to compare types:

export interface YarnInfo {
    value:    string;
    children: Children;
}

export interface Children {
    Version:              string;
    Dependencies?:        Dependency[];
    "Exported Binaries"?: string[];
    Instances?:           number;
}

export interface Dependency {
    descriptor: string;
    locator:    string;
}

Compared to the v1 trees:

interface YarnTree {
  name: string;
  color: 'bold' | 'dim' | null;
  children?: YarnTree[];
  hint?: null;
  depth?: number;
  shadow?: boolean;
}
export interface YarnDeps {
  type: 'tree';
  data: {
    type: 'list';
    trees: YarnTree[];
  };
}

@track0x1 @floydspace anyway, all of this to say that I think v1 and v2+ will need a separate function here, or maybe we can shoehorn the yarn v2+ tree into the v1 structure and then keep going. Thoughts?

Side note...even though I'm using yarn and yarn workspaces in my project, interestingly it seems like if I tell my serverless config for esbuild-serverless to use packager: npm .. it lets me package it up without failing. I'm unsure if there are unintended side effects for doing that, but just wanted to throw that out there.

[kabo]

I'm also getting this issue.

Environment: linux, node 18.16.1, framework 3.34.0 (local), plugin 6.2.3, SDK 4.3.2
Yarn 3.6.3
serverless-esbuild 1.46.0

[dhmw]

I can confirm this is still an issue with the latest versions.

[jounii]

This is little bit blocker now. I noticed that as soon as you add the "external" configuration option, it breaks as it begins to require the "yarn list". I probably can make workaround for now, but definitely needs to be fixed if later yarn versions are to be supported properly.

serverless-esbuild 1.49.0
yarn 4.0.2

[jpgupta]

@jounii / @dhmw - did either of you find a suitable workaround for this? Running into the same issue

[jounii]

@jpgupta I cannot remember anymore. Did quick check and it might be that adding the packager: 'yarn' to the esbuild configuration file fixed it.

[jpgupta]

Thanks @jounii I'll give that a go!

[mattkindy]

Still appears to be an issue