fix(next): remove turbopack build support to fix bundle size regression (#14696)

## Background

The following PRs attempted to add support for Turbopack build:
- #14475
- #14473

## The Fundamental Problem

Payload's database adapters (e.g., `@payloadcms/db-postgres`) depend on
packages with native dependencies that cannot be bundled (e.g.,
`drizzle-kit`, which imports `esbuild`). We need to externalize these
packages.

**Why we can't externalize them directly:**

With pnpm, externalizing a package like `drizzle-kit` generates
`require('drizzle-kit')` calls in the bundle. However, `drizzle-kit` is
not in the user's `package.json` (it's a transitive dependency installed
by `db-postgres`). pnpm's strict dependency isolation prevents importing
dependencies of dependencies, causing runtime failures.

**The attempted workaround:**

Instead of externalizing `drizzle-kit`, we tried externalizing the
entry-point package `@payloadcms/db-postgres` (which users DO install)
via `serverExternalPackages`. This works in development, but creates
severe issues in production builds.

## Why the Workaround Failed

When you externalize `@payloadcms/db-postgres`:

1. **Everything it imports becomes external**, including `payload`
itself
2. This creates **two copies of `payload`**:
   - One bundled (from user's direct imports)
   - One external in node_modules (from db-postgres's imports)
3. **Bundle size explodes** due to:
   - Disabled tree-shaking for externalized packages
   - Duplicate package installations
   - Loss of code-splitting optimizations

**Another example of duplication:**
```
@payloadcms/richtext-lexical (bundled) → qs-esm (bundled)
payload (external) → qs-esm (external)
Result: Two copies of qs-esm in production
```

This issue was reported on our discord
[here](https://discord.com/channels/967097582721572934/1422639568808841329/1440689060015374437).

## The Solution (This PR)

**Short term:** Disable Turbopack build support until Next.js provides a
proper solution.

### Why Webpack Works

Webpack has `webpack.externals`, which can externalize **any** package
regardless of whether it's in the user's `package.json`:

- We externalize `drizzle-kit` directly via `webpack.externals`
- Webpack generates `require('drizzle-kit')` calls in the bundle
- At runtime, Node.js resolves these just fine - we're not yet sure why
that is
- We avoid externalizing entry-point packages, preventing the
duplication problem

### Why Turbopack Build Doesn't Work

Turbopack only has `serverExternalPackages` (similar to
webpack.externals but with restrictions):

- **The constraint**: Packages must be resolvable from the project root
(i.e., in the user's `package.json`)
- If a package isn't directly installed by the user, Next.js **ignores
the externalization rule** and tries to bundle it anyway
- This forces us to externalize entry-point packages (`db-postgres`),
which causes the duplication and bundle size problems described above

### Why Turbopack Dev Works

Turbopack dev has the same `serverExternalPackages` constraint, BUT:

- **In dev, we can afford the trade-off** of externalizing entry-point
packages because:
  - Bundle size doesn't matter in development
  - Faster compilation speed is more important
  - We're not shipping to production
- The duplication problem still exists, but it's acceptable for the dev
experience

**Changes made:**

1. **Throw error for Turbopack builds** - Prevent production builds with
Turbopack until Next.js fixes the underlying issue
2. **Restore webpack.externals** - Use webpack-specific externals for
problematic transitive dependencies (`drizzle-kit`, `sharp`, `libsql`,
etc.) that aren't in user's package.json
3. **Simplify serverExternalPackages** - Only externalize packages
resolvable from project root (`graphql`, `@sentry/nextjs`)
4. **Clean up unnecessary config** - Remove webpack configurations that
are no longer justifiable. Any configuration we have left now comes with
a comment block explaining why we need it
5. **enable devBundleServerPackages optimization by default** - there
have not been any reported issues since this was introduced, and this
setting is now **necessary** for turbopack support during dev

## Future

In order to properly support Turbopack Build, Next.js will have to
implement one of these solutions:

- **Option 1**: Implement webpack.externals-like functionality for
Turbopack (no package.json constraint)
- **Option 2**: Remove the need for declaring all externals as direct
dependencies in the application

We're tracking Next.js's progress on this issue.

Author: AlessioGr

packages/next/src/withPayload.js

@@ -1,7 +1,7 @@
 /**
  * @param {import('next').NextConfig} nextConfig
  * @param {Object} [options] - Optional configuration options
- * @param {boolean} [options.devBundleServerPackages] - Whether to bundle server packages in development mode. @default true
+ * @param {boolean} [options.devBundleServerPackages] - Whether to bundle server packages in development mode. @default false
  *
  * @returns {import('next').NextConfig}
  * */
@@ -52,6 +52,16 @@ export const withPayload = (nextConfig = {}, options = {}) => {
     }
   }
 
+  const isBuild = process.env.NODE_ENV === 'production'
+  const isTurbopackNextjs15 = process.env.TURBOPACK === '1'
+  const isTurbopackNextjs16 = process.env.TURBOPACK === 'auto'
+
+  if (isBuild && (isTurbopackNextjs15 || isTurbopackNextjs16)) {
+    throw new Error(
+      'Payload does not support using Turbopack for production builds. If you are using Next.js 16, please use `next build --webpack` instead.',
+    )
+  }
+
   const poweredByHeader = {
     key: 'X-Powered-By',
     value: 'Next.js, Payload',
@@ -106,42 +116,62 @@ export const withPayload = (nextConfig = {}, options = {}) => {
       ]
     },
     serverExternalPackages: [
+      // serverExternalPackages = webpack.externals, but with turbopack support and an additional check
+      // for whether the package is resolvable from the project root
       ...(nextConfig.serverExternalPackages || []),
-      // These packages always need to be external, both during dev and production. This is because they install dependencies
-      // that will error when trying to bundle them (e.g. drizzle-kit, libsql, esbuild etc.).
-      // We cannot externalize those problem-packages directly. We can only externalize packages that are manually installed
-      // by the end user. Otherwise, the require('externalPackage') calls generated by the bundler would fail during runtime,
-      // as you cannot import dependencies of dependencies in a lot of package managers like pnpm. We'd have to force users
-      // to install the dependencies directly.
-      // Thus, we externalize the "entry-point" = the package that is installed by the end user, which would be our db adapters.
-      //
+      // Can be externalized, because we require users to install graphql themselves - we only rely on it as a peer dependency => resolvable from the project root.
       //
-      // External because it installs mongoose (part of default serverExternalPackages: https://github.com/vercel/next.js/blob/canary/packages/next/src/lib/server-external-packages.json => would throw warning if we don't exclude the entry-point package):
-      '@payloadcms/db-mongodb',
-      // External because they install dependencies like drizzle, libsql, esbuild etc.:
-      '@payloadcms/db-postgres',
-      '@payloadcms/db-sqlite',
-      '@payloadcms/db-vercel-postgres',
-      '@payloadcms/drizzle',
-      '@payloadcms/db-d1-sqlite',
-      // External because they install @aws-sdk/client-s3:
-      '@payloadcms/payload-cloud',
-      // External, because it installs import-in-the-middle and require-in-the-middle - both in the default serverExternalPackages list.
-      '@sentry/nextjs',
-      // Can be externalized, because we require users to install graphql themselves - we only rely on it as a peer dependency.
       // WHY: without externalizing graphql, a graphql version error will be thrown
       // during runtime ("Ensure that there is only one instance of \"graphql\" in the node_modules\ndirectory.")
       'graphql',
-      // TODO: We need to externalize @payloadcms/storage-s3 as well, once Next.js has the ability to exclude @payloadcms/storage-s3/client from being externalized.
-      // Do not bundle additional server-only packages during dev to improve compilation speed
-      ...(process.env.NODE_ENV === 'development' && options.devBundleServerPackages === false
-        ? [
+      // External, because it installs import-in-the-middle and require-in-the-middle - both in the default serverExternalPackages list.
+      '@sentry/nextjs',
+      ...(process.env.NODE_ENV === 'development' && options.devBundleServerPackages !== true
+        ? /**
+           * Unless explicitly disabled by the user, by passing `devBundleServerPackages: true` to withPayload, we
+           * do not bundle server-only packages during dev for two reasons:
+           *
+           * 1. Performance: Fewer files to compile means faster compilation speeds.
+           * 2. Turbopack support: Webpack's externals are not supported by Turbopack.
+           *
+           * Regarding Turbopack support: Unlike webpack.externals, we cannot use serverExternalPackages to
+           * externalized packages that are not resolvable from the project root. So including a package like
+           * "drizzle-kit" in here would do nothing - Next.js will ignore the rule and still bundle the package -
+           * because it detects that the package is not resolvable from the project root (= not directly installed
+           * by the user in their own package.json).
+           *
+           * Instead, we can use serverExternalPackages for the entry-point packages that *are* installed directly
+           * by the user (e.g. db-postgres, which then installs drizzle-kit as a dependency).
+           *
+           *
+           *
+           * We should only do this during development, not build, because externalizing these packages can hurt
+           * the bundle size. Not only does it disable tree-shaking, it also risks installing duplicate copies of the
+           * same package.
+           *
+           * Example:
+           * - @payloadcms/richtext-lexical (in bundle) -> installs qs-esm (bundled because of importer)
+           * - payload (not in bundle, external) -> installs qs-esm (external because of importer)
+           * Result: we have two copies of qs-esm installed - one in the bundle, and one in node_modules.
+           *
+           * During development, these bundle size difference do not matter much, and development speed /
+           * turbopack support are more important.
+           */
+          [
             'payload',
+            '@payloadcms/db-mongodb',
+            '@payloadcms/db-postgres',
+            '@payloadcms/db-sqlite',
+            '@payloadcms/db-vercel-postgres',
+            '@payloadcms/db-d1-sqlite',
+            '@payloadcms/drizzle',
             '@payloadcms/email-nodemailer',
             '@payloadcms/email-resend',
             '@payloadcms/graphql',
+            '@payloadcms/payload-cloud',
             '@payloadcms/plugin-redirects',
             // TODO: Add the following packages, excluding their /client subpath exports, once Next.js supports it
+            // see: https://github.com/vercel/next.js/discussions/76991
             //'@payloadcms/plugin-cloud-storage',
             //'@payloadcms/plugin-sentry',
             //'@payloadcms/plugin-stripe',
@@ -162,7 +192,51 @@ export const withPayload = (nextConfig = {}, options = {}) => {
 
       return {
         ...incomingWebpackConfig,
-        externals: [...(incomingWebpackConfig?.externals || []), 'require-in-the-middle'],
+        externals: [
+          ...(incomingWebpackConfig?.externals || []),
+          /**
+           * See the explanation in the serverExternalPackages section above.
+           * We need to force Webpack to emit require() calls for these packages, even though they are not
+           * resolvable from the project root. You would expect this to error during runtime, but Next.js seems to be able to require these just fine.
+           *
+           * This is the only way to get Webpack Build to work, without the bundle size caveats of externalizing the
+           * entry point packages, as explained in the serverExternalPackages section above.
+           */
+          'drizzle-kit',
+          'drizzle-kit/api',
+          'sharp',
+          'libsql',
+          'require-in-the-middle',
+        ],
+        resolve: {
+          ...(incomingWebpackConfig?.resolve || {}),
+          alias: {
+            ...(incomingWebpackConfig?.resolve?.alias || {}),
+          },
+          fallback: {
+            ...(incomingWebpackConfig?.resolve?.fallback || {}),
+            /*
+             * This fixes the following warning when running next build with webpack (tested on Next.js 16.0.3 with Payload 3.64.0):
+             *
+             * ⚠ Compiled with warnings in 8.7s
+             *
+             * ./node_modules/.pnpm/mongodb@6.16.0/node_modules/mongodb/lib/deps.js
+             * Module not found: Can't resolve 'aws4' in '/Users/alessio/Documents/temp/next16p/node_modules/.pnpm/mongodb@6.16.0/node_modules/mongodb/lib'
+             *
+             * Import trace for requested module:
+             * ./node_modules/.pnpm/mongodb@6.16.0/node_modules/mongodb/lib/deps.js
+             * ./node_modules/.pnpm/mongodb@6.16.0/node_modules/mongodb/lib/client-side-encryption/client_encryption.js
+             * ./node_modules/.pnpm/mongodb@6.16.0/node_modules/mongodb/lib/index.js
+             * ./node_modules/.pnpm/mongoose@8.15.1/node_modules/mongoose/lib/index.js
+             * ./node_modules/.pnpm/mongoose@8.15.1/node_modules/mongoose/index.js
+             * ./node_modules/.pnpm/@payloadcms+db-mongodb@3.64.0_payload@3.64.0_graphql@16.12.0_typescript@5.7.3_/node_modules/@payloadcms/db-mongodb/dist/index.js
+             * ./src/payload.config.ts
+             * ./src/app/my-route/route.ts
+             *
+             **/
+            aws4: false,
+          },
+        },
         plugins: [
           ...(incomingWebpackConfig?.plugins || []),
           // Fix cloudflare:sockets error: https://github.com/vercel/next.js/discussions/50177