diff --git a/CHANGELOG.md b/CHANGELOG.md index 51c223ede75..677898ba23e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -43,6 +43,12 @@ An aside: You may be wondering why esbuild is trying to escape the `e` at all and why it doesn't just pass through the original source code unmodified. The reason why esbuild does this is that, for robustness, esbuild's AST generally tries to omit semantically-unrelated information and esbuild's code printers always try to preserve the semantics of the underlying AST. That way the rest of esbuild's internals can just deal with semantics instead of presentation. They don't have to think about how the AST will be printed when changing the AST. This is the same reason that esbuild's JavaScript AST doesn't have a "parentheses" node (e.g. `a * (b + c)` is represented by the AST `multiply(a, add(b, c))` instead of `multiply(a, parentheses(add(b, c)))`). Instead, the printer automatically inserts parentheses as necessary to maintain the semantics of the AST, which means all of the optimizations that run over the AST don't have to worry about keeping the parentheses up to date. Similarly, the CSS AST for the dimension token stores the actual unit and the printer makes sure the unit is properly escaped depending on what value it's placed after. All of the other code operating on CSS ASTs doesn't have to worry about parsing escapes to compare units or about keeping escapes up to date when the AST is modified. Hopefully that makes sense. +* Attempt to avoid creating the `node_modules/.cache` directory for people that use Yarn 2+ in Plug'n'Play mode ([#2685](https://github.com/evanw/esbuild/issues/2685)) + + When Yarn's PnP mode is enabled, packages installed by Yarn may or may not be put inside `.zip` files. The specific heuristics for when this happens change over time in between Yarn versions. This is problematic for esbuild because esbuild's JavaScript package needs to execute a binary file inside the package. Yarn makes extensive modifications to Node's file system APIs at run time to pretend that `.zip` files are normal directories and to make it hard to tell whether a file is real or not (since in theory it doesn't matter). But they haven't modified Node's `child_process.execFileSync` API so attempting to execute a file inside a zip file fails. To get around this, esbuild previously used Node's file system APIs to copy the binary executable to another location before invoking `execFileSync`. Under the hood this caused Yarn to extract the file from the zip file into a real file that can then be run. + + However, esbuild copied its executable into `node_modules/.cache/esbuild`. This is the [official recommendation from the Yarn team](https://yarnpkg.com/advanced/rulebook/#packages-should-never-write-inside-their-own-folder-outside-of-postinstall) for where packages are supposed to put these types of files when Yarn PnP is being used. However, users of Yarn PnP with esbuild find this really annoying because they don't like looking at the `node_modules` directory. With this release, esbuild now sets `"preferUnplugged": true` in its `package.json` files, which tells newer versions of Yarn to not put esbuild's packages in a zip file. There may exist older versions of Yarn that don't support `preferUnplugged`. In that case esbuild should still copy the executable to a cache directory, so it should still run (hopefully, since I haven't tested this myself). Note that esbuild setting `"preferUnplugged": true` may have the side effect of esbuild taking up more space on the file system in the event that multiple platforms are installed simultaneously, or that you're using an older version of Yarn that always installs packages for all platforms. In that case you may want to update to a newer version of Yarn since Yarn has recently changed to only install packages for the current platform. + ## 0.15.14 * Fix parsing of TypeScript `infer` inside a conditional `extends` ([#2675](https://github.com/evanw/esbuild/issues/2675)) diff --git a/lib/npm/node-platform.ts b/lib/npm/node-platform.ts index 2144792ea86..8c65c230665 100644 --- a/lib/npm/node-platform.ts +++ b/lib/npm/node-platform.ts @@ -210,48 +210,66 @@ for your current platform.`); } } - // The esbuild binary executable can't be used in Yarn 2 in PnP mode because - // it's inside a virtual file system and the OS needs it in the real file - // system. So we need to copy the file out of the virtual file system into - // the real file system. + // This code below guards against the unlikely case that the user is using + // Yarn 2+ in PnP mode and that version is old enough that it doesn't support + // the "preferUnplugged" setting. If that's the case, then the path to the + // binary executable that we got above isn't actually a real path. Instead + // it's a path to a zip file with some extra stuff appended to it. // - // You might think that we could use "preferUnplugged: true" in each of the - // platform-specific packages for this instead, since that tells Yarn to not - // use the virtual file system for those packages. This is not done because: + // Yarn's PnP mode tries hard to patch Node's file system APIs to pretend + // that these fake paths are real. So we can't check whether it's a real file + // or not by using Node's file system APIs (e.g. "fs.existsSync") because + // they have been patched to lie. But we can't return this fake path because + // Yarn hasn't patched "child_process.execFileSync" to work with fake paths, + // so attempting to execute the binary will fail. // - // * Really early versions of Yarn don't support "preferUnplugged", so package - // installation would break on those Yarn versions if we did this. + // As a hack, we use Node's file system APIs to copy the file from the fake + // path to a real path. This will cause Yarn's hacked file system to extract + // the binary executable from the zip file and turn it into a real file that + // we can execute. // - // * Earlier Yarn versions always installed all optional dependencies for all - // platforms even though most of them are incompatible. To minimize file - // system space, we want these useless packages to take up as little space - // as possible so they should remain unzipped inside their ".zip" files. + // This is only done when both ".zip/" is present in the path and the + // "pnpapi" package is present, which is a strong indication that Yarn PnP is + // being used. There is no API at all for telling whether something is a real + // file or not as far as I can tell. Even Yarn's own code just checks for + // whether ".zip/" is present in the path or not. // - // We have to explicitly pass "preferUnplugged: false" instead of leaving - // it up to Yarn's default behavior because Yarn's heuristics otherwise - // automatically unzip packages containing ".exe" files, and we don't want - // our Windows-specific packages to be unzipped either. - // - let pnpapi: any; - try { - pnpapi = require('pnpapi'); - } catch (e) { - } - if (pnpapi) { - const root = pnpapi.getPackageInformation(pnpapi.topLevel).packageLocation; - const binTargetPath = path.join( - root, - 'node_modules', - '.cache', - 'esbuild', - `pnpapi-${pkg}-${ESBUILD_VERSION}-${path.basename(subpath)}`, - ); - if (!fs.existsSync(binTargetPath)) { - fs.mkdirSync(path.dirname(binTargetPath), { recursive: true }); - fs.copyFileSync(binPath, binTargetPath); - fs.chmodSync(binTargetPath, 0o755); + // Note to self: One very hacky way to tell if a path is under the influence + // of Yarn's file system hacks is to stat the file and check for the "crc" + // property in the result. If that's present, then the file is inside a zip + // file. However, I haven't done that here because it's not intended to be + // used that way. Who knows what Yarn versions it does or does not work on + // (including future versions). + if (/\.zip\//.test(binPath)) { + let pnpapi: any; + try { + pnpapi = require('pnpapi'); + } catch (e) { + } + if (pnpapi) { + // Copy the executable to ".cache/esbuild". The official recommendation + // of the Yarn team is to use the directory "node_modules/.cache/esbuild": + // https://yarnpkg.com/advanced/rulebook/#packages-should-never-write-inside-their-own-folder-outside-of-postinstall + // People that use Yarn in PnP mode find this really annoying because they + // don't like seeing the "node_modules" directory. These people should + // either work with Yarn to change their recommendation, or upgrade their + // version of Yarn, since newer versions of Yarn shouldn't stick esbuild's + // binary executables in a zip file due to the "preferUnplugged" setting. + const root = pnpapi.getPackageInformation(pnpapi.topLevel).packageLocation; + const binTargetPath = path.join( + root, + 'node_modules', + '.cache', + 'esbuild', + `pnpapi-${pkg}-${ESBUILD_VERSION}-${path.basename(subpath)}`, + ); + if (!fs.existsSync(binTargetPath)) { + fs.mkdirSync(path.dirname(binTargetPath), { recursive: true }); + fs.copyFileSync(binPath, binTargetPath); + fs.chmodSync(binTargetPath, 0o755); + } + return { binPath: binTargetPath, isWASM }; } - return { binPath: binTargetPath, isWASM }; } return { binPath, isWASM }; diff --git a/npm/@esbuild/android-arm/package.json b/npm/@esbuild/android-arm/package.json index 63d46a60c7f..a95fc26d615 100644 --- a/npm/@esbuild/android-arm/package.json +++ b/npm/@esbuild/android-arm/package.json @@ -4,7 +4,7 @@ "description": "A WebAssembly shim for esbuild on Android ARM.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/@esbuild/linux-loong64/package.json b/npm/@esbuild/linux-loong64/package.json index a97569d8418..af0cfd9063f 100644 --- a/npm/@esbuild/linux-loong64/package.json +++ b/npm/@esbuild/linux-loong64/package.json @@ -4,7 +4,7 @@ "description": "The Linux LoongArch 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-android-64/package.json b/npm/esbuild-android-64/package.json index 1edc0b28dd2..571c9b4ac1e 100644 --- a/npm/esbuild-android-64/package.json +++ b/npm/esbuild-android-64/package.json @@ -4,7 +4,7 @@ "description": "A WebAssembly shim for esbuild on Android x64.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-android-arm64/package.json b/npm/esbuild-android-arm64/package.json index 71da4c89c23..a6e50feab6c 100644 --- a/npm/esbuild-android-arm64/package.json +++ b/npm/esbuild-android-arm64/package.json @@ -4,7 +4,7 @@ "description": "The Android ARM 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-darwin-64/package.json b/npm/esbuild-darwin-64/package.json index 4e7cf9b5def..5d3cb183d1e 100644 --- a/npm/esbuild-darwin-64/package.json +++ b/npm/esbuild-darwin-64/package.json @@ -4,7 +4,7 @@ "description": "The macOS 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-darwin-arm64/package.json b/npm/esbuild-darwin-arm64/package.json index 910876d136e..7b8a31ed1fc 100644 --- a/npm/esbuild-darwin-arm64/package.json +++ b/npm/esbuild-darwin-arm64/package.json @@ -4,7 +4,7 @@ "description": "The macOS ARM 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-freebsd-64/package.json b/npm/esbuild-freebsd-64/package.json index 9cecac49732..4786f9f5897 100644 --- a/npm/esbuild-freebsd-64/package.json +++ b/npm/esbuild-freebsd-64/package.json @@ -4,7 +4,7 @@ "description": "The FreeBSD 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-freebsd-arm64/package.json b/npm/esbuild-freebsd-arm64/package.json index c6daa7776cd..b1a186ada16 100644 --- a/npm/esbuild-freebsd-arm64/package.json +++ b/npm/esbuild-freebsd-arm64/package.json @@ -4,7 +4,7 @@ "description": "The FreeBSD ARM 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-linux-32/package.json b/npm/esbuild-linux-32/package.json index 56e3dea95f9..93d7254cc9e 100644 --- a/npm/esbuild-linux-32/package.json +++ b/npm/esbuild-linux-32/package.json @@ -4,7 +4,7 @@ "description": "The Linux 32-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-linux-64/package.json b/npm/esbuild-linux-64/package.json index e130d7bc512..92545c52bd7 100644 --- a/npm/esbuild-linux-64/package.json +++ b/npm/esbuild-linux-64/package.json @@ -4,7 +4,7 @@ "description": "The Linux 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-linux-arm/package.json b/npm/esbuild-linux-arm/package.json index 5d186757e56..eafb7732549 100644 --- a/npm/esbuild-linux-arm/package.json +++ b/npm/esbuild-linux-arm/package.json @@ -4,7 +4,7 @@ "description": "The Linux ARM binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-linux-arm64/package.json b/npm/esbuild-linux-arm64/package.json index 010956a403d..b12819b9684 100644 --- a/npm/esbuild-linux-arm64/package.json +++ b/npm/esbuild-linux-arm64/package.json @@ -4,7 +4,7 @@ "description": "The Linux ARM 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-linux-mips64le/package.json b/npm/esbuild-linux-mips64le/package.json index d62275a34c3..c10db53e1e7 100644 --- a/npm/esbuild-linux-mips64le/package.json +++ b/npm/esbuild-linux-mips64le/package.json @@ -4,7 +4,7 @@ "description": "The Linux MIPS 64-bit Little Endian binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-linux-ppc64le/package.json b/npm/esbuild-linux-ppc64le/package.json index cb2ff8a7503..691a4421a9f 100644 --- a/npm/esbuild-linux-ppc64le/package.json +++ b/npm/esbuild-linux-ppc64le/package.json @@ -4,7 +4,7 @@ "description": "The Linux PowerPC 64-bit Little Endian binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-linux-riscv64/package.json b/npm/esbuild-linux-riscv64/package.json index 6c9f6cf518f..6150c81ea76 100644 --- a/npm/esbuild-linux-riscv64/package.json +++ b/npm/esbuild-linux-riscv64/package.json @@ -4,7 +4,7 @@ "description": "The Linux RISC-V 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-linux-s390x/package.json b/npm/esbuild-linux-s390x/package.json index 0324f8e8755..d043ab4f2b9 100644 --- a/npm/esbuild-linux-s390x/package.json +++ b/npm/esbuild-linux-s390x/package.json @@ -4,7 +4,7 @@ "description": "The Linux IBM Z 64-bit Big Endian binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-openbsd-64/package.json b/npm/esbuild-openbsd-64/package.json index 5bef5d01eb8..1ab1208d7f3 100644 --- a/npm/esbuild-openbsd-64/package.json +++ b/npm/esbuild-openbsd-64/package.json @@ -4,7 +4,7 @@ "description": "The OpenBSD 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-sunos-64/package.json b/npm/esbuild-sunos-64/package.json index 297e0019f01..a5d977cafc2 100644 --- a/npm/esbuild-sunos-64/package.json +++ b/npm/esbuild-sunos-64/package.json @@ -4,7 +4,7 @@ "description": "The illumos 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-windows-32/package.json b/npm/esbuild-windows-32/package.json index 6f40e822ed3..72ddd5d7e60 100644 --- a/npm/esbuild-windows-32/package.json +++ b/npm/esbuild-windows-32/package.json @@ -4,7 +4,7 @@ "description": "The Windows 32-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-windows-64/package.json b/npm/esbuild-windows-64/package.json index 7fe432a508a..51b01e2175e 100644 --- a/npm/esbuild-windows-64/package.json +++ b/npm/esbuild-windows-64/package.json @@ -4,7 +4,7 @@ "description": "The Windows 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" }, diff --git a/npm/esbuild-windows-arm64/package.json b/npm/esbuild-windows-arm64/package.json index 7f806caf689..b484bc4d0ef 100644 --- a/npm/esbuild-windows-arm64/package.json +++ b/npm/esbuild-windows-arm64/package.json @@ -4,7 +4,7 @@ "description": "The Windows ARM 64-bit binary for esbuild, a JavaScript bundler.", "repository": "https://github.com/evanw/esbuild", "license": "MIT", - "preferUnplugged": false, + "preferUnplugged": true, "engines": { "node": ">=12" },