Broken on Next.js 10.2.0 which uses Webpack 5 when running a custom server to support HTTPS locally

[gavrichards]

Bug description

When I upgrade Next.js to 10.2.0 which uses Webpack 5 and then run node server.js to run my app locally using HTTPS, I get this error:

error - ./prisma/client/runtime/index.js:24474:15
Module not found: Can't resolve '_http_common'
  24472 | var require_http_parser = __commonJS2((exports2, module2) => {
  24473 |   "use strict";
> 24474 |   var common = require("_http_common");
        |               ^
  24475 |   if (common.HTTPParser) {
  24476 |     module2.exports = common.HTTPParser;
  24477 |   } else {

How to reproduce

Typically you run Next.js apps locally by running npm run dev and that's it, but if you want to use HTTPS locally then you have to do something a bit different which involves creating a file usually called "server.js" which attaches the SSL certificate and then runs the server. I have no idea if this particularly issue is related to that or if it would happen without HTTPS, as I only use this method now. I'm hoping the error message alone might make the issue apparent to anyone who understands what it's saying, and why Webpack 5 would cause this.

Expected behavior

Everything works just as it would on Next.js <10.2.0 / Webpack 4.

Environment & setup

• OS: macOS Big Sur
• Database: MySQL / Amazon Aurora
• Node.js version: 14.16.0
• Prisma version:

prisma               : 2.22.0
@prisma/client       : 2.22.0
Current platform     : darwin
Query Engine         : query-engine 60cc71d884972ab4e897f0277c4b84383dddaf6c (at node_modules/@prisma/engines/query-engine-darwin)
Migration Engine     : migration-engine-cli 60cc71d884972ab4e897f0277c4b84383dddaf6c (at node_modules/@prisma/engines/migration-engine-darwin)
Introspection Engine : introspection-core 60cc71d884972ab4e897f0277c4b84383dddaf6c (at node_modules/@prisma/engines/introspection-engine-darwin)
Format Binary        : prisma-fmt 60cc71d884972ab4e897f0277c4b84383dddaf6c (at node_modules/@prisma/engines/prisma-fmt-darwin)
Default Engines Hash : 60cc71d884972ab4e897f0277c4b84383dddaf6c
Studio               : 0.379.0

[millsp]

Related to #6564, we need to upgrade to undici@4 which is currently in alpha.

[gavrichards]

It would be great to have an update on this. I'm stuck on Next.js v10.1.3 and Prisma v2.23.0 until this is resolved. Thanks!

[millsp]

This is in the making (currently works) #8842. If you want, you can give it a try by installing this version instead 2.30.0-integration-undici-4.2. Feedback welcome :)

[gavrichards]

@millsp thanks, I gave it a try.
I got this error:

error - ./prisma/client/runtime/index.js:29838:23
Module not found: Can't resolve 'stream/web'
  29836 |   function ReadableStreamFrom(iterable) {
  29837 |     if (!ReadableStream) {
> 29838 |       ReadableStream = require("stream/web").ReadableStream;
        |                       ^
  29839 |     }
  29840 |     if (ReadableStream.from) {
  29841 |       return ReadableStream.from(iterable);

[millette]

@gavrichards node v16.5.0 is required according to https://nodejs.org/api/webstreams.html#webstreams_new_readablestream_underlyingsource_strategy

[millsp]

Thanks for the feedback, and sorry about this. undici@4's package.json states that they support node >=12.18, so I'll check up on this.

[millsp]

@gavrichards can you please provide a repro? Which node version are you using? I could not reproduce this on node 14.16.0.

[gavrichards]

I'm on 14.16.0. I'll get back to you regarding a repro.

Is this not the case in your original message?

Alright, undici is written in commonjs modules which esbuild does not attempt to tree-shake. So all of undici ends up in our bundle, and that causes all the bundled modules to load at once (including unsused stuff that's reserved for node >= 16).

[millsp]

It does get all bundled, but I could not reproduce this locally, so my first assumption that it gets all loaded at once might not hold as I could not reproduce this locally. This also never shows in our tests.

That only seems to be used on their fetch api
https://github.com/nodejs/undici/search?q=stream%2Fweb
As stated by undici in their docs. We don't use fetch
https://github.com/nodejs/undici#undicifetchinput-init-promise

[gavrichards]

Weird because as you can see in my earlier comment, the use of stream/web comes from prisma/client/runtime/index.js
If Prisma doesn't use this functionality, how could it be ending up in there?
Would it help if I sent you that file?

[belgattitude]

@gavrichards If it helps I upgraded to nextjs 11.1.0 / webpack 5, prisma 2.30.0 and this next.config.js trick still works with node 14.x:

  webpack: (config, { defaultLoaders, isServer }) => {
    // thx  https://github.com/prisma/prisma/issues/6899#issuecomment-849126557
    if (isServer) {
      config.externals.push('_http_common');
    }
    return config;
  },

PS: a full config is here https://github.com/belgattitude/nextjs-monorepo-example/blob/main/apps/web-app/next.config.js#L127

[millsp]

@gavrichards Yes, all gets bundled because esbuild does not attempt to tree-shake commonjs. But besides that, that require is guarded by and if, so it should not be reached. Did you find the time to build a repro?

[gavrichards]

Ok I updated to Prisma 2.30.0 and Next 11.1.0 and everything's working ok. 🤯
I tried the next.config.js trick that @belgattitude mentioned but it doesn't seem to be needed so I've left it out.
The one thing I did need to change was to drop output = "./client" from schema.prisma. I presume this was necessary when I started the project, but it doesn't seem to be anymore. This is presumably what was causing the issue @millsp was unable to reproduce.
I'll let you know if I come across any other issues, but everything appears good so far. I'm confused because I'm not using 2.30.0-integration-undici-4.2, just the regular version.

[belgattitude]

@gavrichards yes seems working fine without 😄

Test here: belgattitude/nextjs-monorepo-example#327

[finnatsea]

For what it's worth, I simply needed to remove target: "serverless" from next.config.js.

[thinkjrs]

For what it's worth, I simply needed to remove target: "serverless" from next.config.js.

You are a life saver. Exact same issue here and removing the serverless target did the trick. Thanks for sharing!

[millsp]

Hey everyone, this has been fixed in @prisma/client@dev if you want to give it a try. Please let us know if you find anything unexpected. Thanks.

(cc @gavrichards @belgattitude @finnatsea)

[RobinLbt]

@millsp is this fixed in the 3.11.1 version or do we need to wait?

[janpio]

As Pierre wrote, this is fixed in dev and can be tested and confirmed there @RobinLbt. It will later be part of 3.12.0.

[MGough]

@millsp thanks, I gave it a try. I got this error:

error - ./prisma/client/runtime/index.js:29838:23
Module not found: Can't resolve 'stream/web'
  29836 |   function ReadableStreamFrom(iterable) {
  29837 |     if (!ReadableStream) {
> 29838 |       ReadableStream = require("stream/web").ReadableStream;
        |                       ^
  29839 |     }
  29840 |     if (ReadableStream.from) {
  29841 |       return ReadableStream.from(iterable);

I've ended up here as I'm seeing a similar issue but only after upgrading to 3.12.0 from 3.11.1:

ModuleNotFoundError: Module not found: Error: Can't resolve 'stream/web' in '/<my path>/node_modules/@prisma/client/runtime'


> Build error occurred
Error: > Build failed because of webpack errors
    at /<my path>/node_modules/next/dist/build/index.js:397:19
    at async Span.traceAsyncFn (/<my path>/node_modules/next/dist/telemetry/trace/trace.js:60:20)
    at async Object.build [as default] (/<my path>node_modules/next/dist/build/index.js:77:25)
info  - Loaded env from /<my path>/.env.local
info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
info  - Checking validity of types...

I'm not sure if this is a different issue, I wanted to flag it just in case. Happy to open up a ticket for it if needed. I'm currently investigating a separate problem so I'm not in a rush to resolve this one. Once I have more time I'm happy to do some investigation.

[janpio]

That is a follow up issue to use finally merging Undici 4 I would guess, so a new issue with full information would be most welcome. THanks @MGough.

[millsp]

Hey @MGough, are you generating in a custom location?

[sdgluck]

We are using Prisma 4.4.0 (latest at time of writing) and are seeing the issue reported here when including the Prisma runtime as part of a webpack bundle:

Module not found: Can't resolve 'stream/web'

To get around it we added "stream/web" and "util/types" to the externals array in our webpack configuration, as well as externalPresets.node=true and target="node":

target: "node",
externals: ["stream/web", "util/types"],
externalPresets: { node: true },

The above was all that was required for us to successfully bundle Prisma 4.4.0 with webpack.

Whether it runs OK is another thing... I will try and update this message once we validate that. 🙏

[millsp]

This error feels weird to me, webpack should treat this as a native module. I assume that you're bundling on Node 14 then?

[sdgluck]

We are using Node v16.16.0

[millsp]

Thanks. I am not sure why webpack wouldn't treat this as a native module. I have found a similar issue in the dependency responsible for this. However, it was closed as "Not Planned". If you have the time, could you work on a reproduction? I could then push forward for fixing this.