Custom server renderToHTML does not work with next 13.4

[thilohaas]

Link to the code that reproduces this issue or a replay of the bug

https://github.com/thilohaas/nextjs-custom-server/tree/main
https://codesandbox.io/p/github/thilohaas/nextjs-custom-server/main

To Reproduce

1. Start a new nextjs project with npx create-next-app@latest
2. Add a server.js

const { createServer } = require('http');
const { parse } = require('url');
const next = require('next');

const dev = process.env.NODE_ENV !== "production";
const hostname = 'localhost';
const port = process.env.PORT || 3001;
// when using middleware `hostname` and `port` must be provided below
const app = next({ hostname, port, dev });
const handle = app.getRequestHandler();

app.prepare().then(() => {
  createServer(async (req, res) => {
    try {
      const parsedUrl = parse(req.url, true);
      const { pathname, query } = parsedUrl;

      if (pathname.startsWith('/_next') || pathname.startsWith('/__next')) {
        await handle(req, res, parsedUrl);
      } else {
        const html = await app.renderToHTML(req, res, pathname, query);
        res.end(html);
      }
    } catch (err) {
      console.error('Error occurred handling', req.url, err);
      res.statusCode = 500;
      res.end('internal server error');
    }
  }).listen(port, (err) => {
    if (err) throw err;
    console.log(`> Ready on http://${hostname}:${port}`);
  });
});

3. Change the package.json dev script to: "dev": "node server.js",
4. run npm run dev
5. open http://localhost:3001

Current vs. Expected behavior

I expect it to display the welcome page but instead i get an error from the next server:

Error: Invariant ensurePage called outside render worker

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
      Platform: linux
      Arch: x64
      Version: #1 SMP PREEMPT_DYNAMIC Sun Aug  6 20:05:33 UTC 2023
    Binaries:
      Node: 16.17.0
      npm: 8.15.0
      Yarn: 1.22.19
      pnpm: 7.1.0
    Relevant Packages:
      next: 13.4.20-canary.16
      eslint-config-next: N/A
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 5.1.3
    Next.js Config:
      output: N/A

Which area(s) are affected? (Select all that apply)

Not sure

Additional context

No response

[sreimer15]

I'm seeing this error as well, when using next-dev-https

[KODerFunk]

For me this problem is also related to this one: #53466
Obstacles have appeared in large projects with their own infrastructure (Kube).
These problems are associated with the use of Custom-Server, which is a great need for additional collection of metrics, custom logging, control of response headers and status, etc. I also need the ability to run in one thread so that the AsyncLocalStorage context is saved and other global mechanics work.

On versions higher than 13.4.12 (I confirm, the problem appears with 13.4.13-canary.0) it became impossible to create a custom-server. Now when calling methods, and even render, I get errors when calling server methods: Invariant ... called outside render worker.

The Next server is initializing, but it seems to be isolated from my Fastify server, or the new code is broken.
@ijjk It seems to me that you added these changes that affected an important part of the community.

[JaviCodx]

We are having the same issue with a custom Express server

[salman-mazhar]

We are facing the same issue when using renderToHTML with a custom http server, is there a temporary work around?

[KODerFunk]

@salman-mazhar We are waiting for @ijjk.

[linhucontorion]

@jiebaibai please use English to report your issue, so others can understand, thanks

[jiebaibai]

@jiebaibai please use English to report your issue, so others can understand, thanks

@ijjk 13.5.5 also has this problem, after customizing server.js, app.renderToHTML exception，catch error is：
TypeError: Cannot read properties of undefined (reading 'ensurePage')

[QianKuang8]

Also face a similar issue in Custom Server. Our AssetPrefix is partially working.

[saurabhgupta050890]

Same issue with Next 14.0.3 with custom server. We are having a hapi server

[gustavoalvesm]

Same problem here with custom server + renderToHTML + hydrate components 😞 any temporary solution?

[ESCxjl]

We are facing the same issue when using renderToHTML with a custom http server, anyone can help me ?

[chngjunjie]

Any update for this issue ? we are facing the same issue with hapi js. renderToHTML issue , version 13.4

The isRenderWorker is always undefined in next-dev-server. So it will work if we turn off the development mode, build the dist folder and run it without next server.

[SouthLink]

Same issue with Next 14.0.4 with custom server.

[emreozkandev]

do not use custom next serve. manage routers via next.config.js

[redguard1]

I have a same problem with next 14.1.0 with custom server. Any update for this issue ?

[mildfuzz]

Also seeing this error

[emreozkandev]

I solved my problem by managing the routes in next.config.js

[mildfuzz]

@duzgunemreozkan yeah, you mentioned.

Maybe you might elaborate on how you are able to replicate the functionality of Express in the next.config.js, that would be helpful

[azraelgarden]

Same error here (Next 14. while trying to work with Hydrate and Stencil (Web Components).
TypeError: Cannot read properties of undefined (reading 'ensurePage')

[mildfuzz]

Looking through the code, seems like it *might" only be an issue with the dev server. Might be worth seeing if it will run in production mode

…

On Fri, 16 Feb 2024, 14:03 Azrael Garden, ***@***.***> wrote: Same error here (Next 14. while trying to work with Hydrate and Stencil (Web Components). TypeError: Cannot read properties of undefined (reading 'ensurePage') — Reply to this email directly, view it on GitHub <#54977 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABQAYKWTU3GIBD7U55TDBDYT5RKPAVCNFSM6AAAAAA4KOUVACVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBYGQ2DCNRWGU> . You are receiving this because you commented.Message ID: <vercel/next. ***@***.***>

[vaishnavi-jadhav-amla]

Iam facing the same issue. Iam not able to use renderToHTML on custom server. Iam using next 13.4. Any workaround for this issue. I also tried on the latest version but it still not worked.

[rtrys]

I'm seeing the same error with custom server (I'm using Hapi)

[ChangWei-Chang]

Any update or workaround on this ? We are planning to upgrade to next 14.2.3 from 12 and see this issue.

[Rylab]

We ran into this problem too, it's with renderToHtml only in dev mode. We got around it by calling app.render in dev mode instead.

[rv-dswanson]

We ran into this problem too, it's with renderToHtml only in dev mode. We got around it by calling app.render in dev mode instead.

This worked for my team also, going from 13.2.3 to 14.2.4

Nvm saw errors in server

[mayerraphael]

A workaround is to directly access the internal Dev/Node server inside the app object.

const appHtml = await (app as any).renderServer.server.renderToHTML(req, res, pathname!, query);

This works with 14.2.4.

Problem is the getServer() method. Which is called on the app ( CustomServer ) object, but not on the already created DevNextServer. Therefor another new server is created with an undefined bundlerService. app (which is a CustomServer), has no serverPromise.

app.renderServer or app.renderServer.server are the correct objects.

[lzkui2013]

Aslo face the same thing，when use customServer to get the page html to cache in redis。

const html = await app.renderToHTML(req, res, pathname, query);

Alway error: Invariant: dynamic responses cannot be unchunked. This is a bug in Next.js

Nextjs version: 14.2.3