[Bug]: yarn v3 "@esbuild/linux-x64" is not found when running yarn build-storybook (Docker)

[bobmazy]

Describe the bug

When I run storybook build with yarn (v3) inside a docker container (with node:19-alpine3.17) i get an error message that the package @esbuild/linux64 is missing and needed by esbuild. I have already tried different node images.

When I add the package manually, everything works fine. It also works when using npm.

To Reproduce

https://github.com/bobmazy/storybook-test

System

System:
  OS: Linux 5.10 Alpine Linux
  CPU: (12) x64 Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz
Binaries:
  Node: 19.8.1 - /usr/local/bin/node
  Yarn: 3.5.0 - /usr/local/bin/yarn
  npm: 9.5.1 - /usr/local/bin/npm
npmPackages:
  @storybook/addon-essentials: ^7.0.2 => 7.0.2
  @storybook/addon-interactions: ^7.0.2 => 7.0.2
  @storybook/addon-links: ^7.0.2 => 7.0.2
  @storybook/blocks: ^7.0.2 => 7.0.2
  @storybook/react: ^7.0.2 => 7.0.2
  @storybook/react-vite: ^7.0.2 => 7.0.2
  @storybook/testing-library: ^0.1.0 => 0.1.0

Additional context

Console Output

4.140 @storybook/cli v7.0.2
=> Cleaning outputDir: /storybook-static
Error: The package "@esbuild/linux-x64" could not be found, and is needed by esbuild.

If you are installing esbuild with npm, make sure that you don't specify the
"--no-optional" or "--omit=optional" flags. The "optionalDependencies" feature
of "package.json" is used by esbuild to install the correct binary executable
for your current platform.
at generateBinPath (/app/.yarn/unplugged/esbuild-npm-0.17.14-fb0460a0aa/node_modules/esbuild/lib/main.js:1885:15)
at esbuildCommandAndArgs (/app/.yarn/unplugged/esbuild-npm-0.17.14-fb0460a0aa/node_modules/esbuild/lib/main.js:1955:33)
at ensureServiceIsRunning (/app/.yarn/unplugged/esbuild-npm-0.17.14-fb0460a0aa/node_modules/esbuild/lib/main.js:2119:25)
at startSyncServiceWorker (/app/.yarn/unplugged/esbuild-npm-0.17.14-fb0460a0aa/node_modules/esbuild/lib/main.js:2322:19)
at Object. (/app/.yarn/unplugged/esbuild-npm-0.17.14-fb0460a0aa/node_modules/esbuild/lib/main.js:2363:3)
at Module._compile (node:internal/modules/cjs/loader:1275:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1329:10)
at require$$0.Module._extensions..js (/app/.pnp.cjs:21863:33)
at Module.load (node:internal/modules/cjs/loader:1133:32)
at require$$0.Module._load (/app/.pnp.cjs:21682:22)
Error: The package "@esbuild/linux-x64" could not be found, and is needed by esbuild.

If you are installing esbuild with npm, make sure that you don't specify the
"--no-optional" or "--omit=optional" flags. The "optionalDependencies" feature
of "package.json" is used by esbuild to install the correct binary executable
for your current platform.
at generateBinPath (/app/.yarn/unplugged/esbuild-npm-0.17.14-fb0460a0aa/node_modules/esbuild/lib/main.js:1885:15)
at esbuildCommandAndArgs (/app/.yarn/unplugged/esbuild-npm-0.17.14-fb0460a0aa/node_modules/esbuild/lib/main.js:1955:33)
at ensureServiceIsRunning (/app/.yarn/unplugged/esbuild-npm-0.17.14-fb0460a0aa/node_modules/esbuild/lib/main.js:2119:25)
at startSyncServiceWorker (/app/.yarn/unplugged/esbuild-npm-0.17.14-fb0460a0aa/node_modules/esbuild/lib/main.js:2322:19)
at Object. (/app/.yarn/unplugged/esbuild-npm-0.17.14-fb0460a0aa/node_modules/esbuild/lib/main.js:2363:3)
at Module._compile (node:internal/modules/cjs/loader:1275:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1329:10)
at require$$0.Module._extensions..js (/app/.pnp.cjs:21863:33)
at Module.load (node:internal/modules/cjs/loader:1133:32)
at require$$0.Module._load (/app/.pnp.cjs:21682:22)

[IanVS]

I wonder if this is due to creating your lockfile on one platform, and then installing the node_modules in another. ESBuild can have trouble with that sometimes...

One thing you can try is to remove your yarn.lock and node_modules, and re-install to re-generate the lockfile.

[bobmazy]

Unfortunately, this does not work either.
I also tried to create the yarn.lock in the Docker container and copy only the package.json into the container.

[mrredr]

You can try one these solutions https://esbuild.github.io/getting-started/#simultaneous-platforms

[fbell123]

Having the same issue, I'm running yarn on Mac and then uploading to a Linux CI (node:16-alpine3.17)

I've tried the supported architectures as suggested by esbuild and its the same issue there, even tried specifying the package extensions. Also tried the nuclear option of removing yarn.lock, clean cache, rebuild and install but no luck either.

The CI doesn't seem to find the esbuild package, as when I intercept the pipeline and run yarn add -D @esbuild/linux-x64 it removes the uploaded package and re-downloads from the registry as can be seen in the Fetch step below. It then 'fails' the install due to esbuild not being built successfully. I've quoted fails because I've run yarn storybook build on the CI and it works even though yarn errored.

I tried adding yarn add -D @esbuild/linux-x64 to the CI step before running Storybook but it exits before running due to the yarn error.

➤ YN0000: ┌ Fetch step
➤ YN0013: │ @esbuild/linux-x64@npm:0.18.11 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ @esbuild/linux-x64@npm:0.17.19 can't be found in the cache and will be fetched from the remote registry
➤ YN0019: │ @esbuild-linux-x64-npm-0.17.19-08a7136aa6-03ff211ae0.zip appears to be unused - removing
➤ YN0019: │ @esbuild-linux-x64-npm-0.18.11-b588f18037-8.zip appears to be unused - removing
➤ YN0000: └ Completed in 3s 845ms

➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0007: │ @apollo/protobufjs@npm:1.2.6 must be built because it never has been before or the last one failed
➤ YN0007: │ @apollo/protobufjs@npm:1.2.7 must be built because it never has been before or the last one failed
➤ YN0007: │ core-js@npm:3.8.3 must be built because it never has been before or the last one failed
➤ YN0007: │ esbuild@npm:0.18.11 must be built because it never has been before or the last one failed
➤ YN0007: │ core-js@npm:3.6.4 must be built because it never has been before or the last one failed
➤ YN0007: │ protobufjs@npm:6.11.3 must be built because it never has been before or the last one failed
➤ YN0007: │ esbuild@npm:0.17.19 must be built because it never has been before or the last one failed
➤ YN0007: │ @swc/core@npm:1.3.53 [b02d5] must be built because it never has been before or the last one failed
➤ YN0007: │ msw@npm:1.2.1 [b02d5] must be built because it never has been before or the last one failed
➤ YN0009: │ esbuild@npm:0.18.11 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-4463af5b/build.log)
➤ YN0009: │ esbuild@npm:0.17.19 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-388c1661/build.log)
➤ YN0007: │ now-xtv@workspace:. must be built because it never has been before or the last one failed
➤ YN0000: └ Completed in 5s 467ms
➤ YN0000: Failed with errors in 10s 331ms

Log from the failed `esbuild` step

# This file contains the result of Yarn building a package (esbuild@npm:0.17.19)
# Script name: postinstall

[esbuild] Failed to find package "@esbuild/linux-x64" on the file system

This can happen if you use the "--no-optional" flag. The "optionalDependencies"
package.json feature is used by esbuild to install the correct binary executable
for your current platform. This install script will now attempt to work around
this. If that fails, you need to remove the "--no-optional" flag to use esbuild.

[esbuild] Trying to install package "@esbuild/linux-x64" using npm
node:internal/errors:857
  const err = new Error(message);
              ^

Error: Command failed: /usr/local/bin/node /tmp/build/1e48fc67/project/.yarn/unplugged/esbuild-npm-0.17.19-f690397756/node_modules/esbuild/bin/esbuild --version
/tmp/build/1e48fc67/project/.pnp.cjs:43379
  return Object.defineProperties(new Error(message), {
                                 ^

Error: Qualified path resolution failed: we looked for the following paths, but none could be accessed.

Source path: /tmp/build/1e48fc67/project/.yarn/unplugged/esbuild-npm-0.17.19-f690397756/node_modules/esbuild/bin/esbuild
Not found: /tmp/build/1e48fc67/project/.yarn/unplugged/esbuild-npm-0.17.19-f690397756/node_modules/esbuild/bin/esbuild

    at makeError (/tmp/build/1e48fc67/project/.pnp.cjs:43379:34)
    at resolveUnqualified (/tmp/build/1e48fc67/project/.pnp.cjs:45194:13)
    at resolveRequest (/tmp/build/1e48fc67/project/.pnp.cjs:45235:14)
    at Object.resolveRequest (/tmp/build/1e48fc67/project/.pnp.cjs:45291:26)
    at resolve$1 (file:///tmp/build/1e48fc67/project/.pnp.loader.mjs:1991:25)
    at nextResolve (node:internal/modules/esm/loader:163:28)
    at ESMLoader.resolve (node:internal/modules/esm/loader:837:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)
    at ESMLoader.import (node:internal/modules/esm/loader:521:22)
    at node:internal/modules/run_main:58:28

    at checkExecSyncError (node:child_process:861:11)
    at Object.execFileSync (node:child_process:896:15)
    at validateBinaryVersion (/tmp/build/1e48fc67/project/.yarn/unplugged/esbuild-npm-0.17.19-f690397756/node_modules/esbuild/install.js:98:28)
    at /tmp/build/1e48fc67/project/.yarn/unplugged/esbuild-npm-0.17.19-f690397756/node_modules/esbuild/install.js:283:5 {
  status: 1,
  signal: null,
  output: [
    null,
    Buffer(0) [Uint8Array] [],
    Buffer(1154) [Uint8Array] [
       47, 116, 109, 112,  47,  98, 117, 105, 108, 100,  47,  49,
      101,  52,  56, 102,  99,  54,  55,  47, 112, 114, 111, 106,
      101,  99, 116,  47,  46, 112, 110, 112,  46,  99, 106, 115,
       58,  52,  51,  51,  55,  57,  10,  32,  32, 114, 101, 116,
      117, 114, 110,  32,  79,  98, 106, 101,  99, 116,  46, 100,
      101, 102, 105, 110, 101,  80, 114, 111, 112, 101, 114, 116,
      105, 101, 115,  40, 110, 101, 119,  32,  69, 114, 114, 111,
      114,  40, 109, 101, 115, 115,  97, 103, 101,  41,  44,  32,
      123,  10,  32,  32,
      ... 1054 more items
    ]
  ],
  pid: 166,
  stdout: Buffer(0) [Uint8Array] [],
  stderr: Buffer(1154) [Uint8Array] [
     47, 116, 109, 112,  47,  98, 117, 105, 108, 100,  47,  49,
    101,  52,  56, 102,  99,  54,  55,  47, 112, 114, 111, 106,
    101,  99, 116,  47,  46, 112, 110, 112,  46,  99, 106, 115,
     58,  52,  51,  51,  55,  57,  10,  32,  32, 114, 101, 116,
    117, 114, 110,  32,  79,  98, 106, 101,  99, 116,  46, 100,
    101, 102, 105, 110, 101,  80, 114, 111, 112, 101, 114, 116,
    105, 101, 115,  40, 110, 101, 119,  32,  69, 114, 114, 111,
    114,  40, 109, 101, 115, 115,  97, 103, 101,  41,  44,  32,
    123,  10,  32,  32,
    ... 1054 more items
  ]
}

On the CI - ls .yarn/cache/ | grep esbuild

@esbuild-darwin-arm64-npm-0.17.19-64d69299ed-8.zip
@esbuild-darwin-arm64-npm-0.18.11-e0ec586879-8.zip
@esbuild-darwin-x64-npm-0.17.19-30afb0190b-8.zip
@esbuild-darwin-x64-npm-0.18.11-0c67bcca93-8.zip
@esbuild-linux-arm64-npm-0.17.19-6f31af48bb-8.zip
@esbuild-linux-arm64-npm-0.18.11-5e3b51d24f-8.zip
@esbuild-linux-x64-npm-0.17.19-08a7136aa6-03ff211ae0.zip
@esbuild-linux-x64-npm-0.18.11-b588f18037-8.zip
@fal-works-esbuild-plugin-global-externals-npm-2.1.2-b50ef0c6e7-c59715902b.zip
@yarnpkg-esbuild-plugin-pnp-npm-3.0.0-rc.15-b916c218b3-04da15355a.zip
esbuild-npm-0.17.19-f690397756-ac11b1a5a6.zip
esbuild-npm-0.18.11-7ebdd30a31-fbd981388f.zip
esbuild-plugin-alias-npm-0.2.1-8906d98875-afe2d2c8b5.zip
esbuild-register-npm-3.4.2-c873c16acc-f65d1ccb58.zip

After adding the esbuild package manually

@esbuild-darwin-arm64-npm-0.17.19-64d69299ed-8.zip
@esbuild-darwin-arm64-npm-0.18.11-e0ec586879-8.zip
@esbuild-darwin-x64-npm-0.17.19-30afb0190b-8.zip
@esbuild-darwin-x64-npm-0.18.11-0c67bcca93-8.zip
@esbuild-linux-arm64-npm-0.17.19-6f31af48bb-8.zip
@esbuild-linux-arm64-npm-0.18.11-5e3b51d24f-8.zip
@esbuild-linux-x64-npm-0.17.19-08a7136aa6-8.zip
@esbuild-linux-x64-npm-0.18.11-b588f18037-797c07c0ab.zip
@fal-works-esbuild-plugin-global-externals-npm-2.1.2-b50ef0c6e7-c59715902b.zip
@yarnpkg-esbuild-plugin-pnp-npm-3.0.0-rc.15-b916c218b3-04da15355a.zip
esbuild-npm-0.17.19-f690397756-ac11b1a5a6.zip
esbuild-npm-0.18.11-7ebdd30a31-fbd981388f.zip
esbuild-plugin-alias-npm-0.2.1-8906d98875-afe2d2c8b5.zip
esbuild-register-npm-3.4.2-c873c16acc-f65d1ccb58.zip