How to change limit? API response for ${req.url} exceeds 4MB. This will cause the request to fail in a future version.

[700software]

What version of Next.js are you using?

11.1.0

How are you deploying your application?

Other platform

Describe the Bug

When streaming a large file in an API response, are getting this warnings on the terminal.

(this change was first introduced in PR #26831 and modified to 4mb in #26887)

API response for /api/... exceeds 4MB. This will cause the request to fail in a future version. https://nextjs.org/docs/messages/api-routes-body-size-limit

But we are not using Vercel. We are not using AWS serverless functions at all. So to throw a warning is not relevant to us.

I am concerned that we will update Next.js and our file downloads will start throwing runtime exceptions in production as wording of #26831

In a future version this scenario will throw an error.

Possible solution(s)

I propose a config option to suppress this warning (future error) for non-Vercel users of Next.js.

Alternatively, I'm wondering if middleware would make this a non-issue?

We are hoping to avoid spinning up a custom server, but will do if required.

[balazsorban44]

Hi, could you try adding a minimal reproduction that would make this easier to reason about?

[700software]

reproduction

// pages/api/test-download.js
import crypto from 'crypto'

export default async function (req, res) {
  res.setHeader('Content-Disposition', 'attachment; filename=random.dat');
  // create random buffer 10mb
  res.end(crypto.randomBytes(10 * 1024 * 1024)) // above the current 4mb limit of nextjs
  // can also be reproduced with `ReadableStream.pipe(res)`
}

[balazsorban44]

So this is actually intentional.

In your use case, sending large files from a single Next.js instance isn't performant as the user could be far away or the connection gets broken so it's better to use a CDN which is designed for handling large assets.

The error document points this out as well:

If you need to support sending large files to the client, you should consider using a dedicated media host for those assets.

https://nextjs.org/docs/messages/api-routes-body-size-limit

[700software]

@balazsorban44, Prior to opening this issue I read through all the relevant documentation. I'm aware that it is intentional.

However, in our use case, these are private files (requires authentication check), a lesser-used feature in larger application, and all users are in the same country. Therefore we could not justify a CDN. Even a dedicated media host would be overkill. As we grow this may change but not any time soon.

We could certainly start a custom server but that seems like a workaround for red tape. Plus we'd lose the npm run dev hot reloading advantage for those particular endpoints.

On the other hand, it would be so easy to add a config switch, and many developers are in use cases like ours, where, nobody benefits from overengineering. That is, until they can grow their company to a larger scale.

So, I'm just thinking through the Next.js team's motivation for adding this restriction. What your team has done makes sense in certain use cases, but not for everyone. I'm particularly concerned about the Error coming in a future version.

My team would be glad to create a PR for a config switch. What do you think?

[jaymascarenas]

My app would also greatly benefit from being able to override this restriction. We too are not using serverless functions and are ok with larger response payloads. I really like the Next API and don't want to have to create a custom server and express API.

[ImTemporaryHere]

My app would also greatly benefit from being able to override this restriction. We too are not using serverless functions and are ok with larger response payloads. I really like the Next API and don't want to have to create a custom server and express API.

fully support , I dont use vercel, there is no need to limit my app... pls.

[700software]

Again, my team would be glad to create a PR introducing a next.config.js option. I just need to know it will be merged and not rejected. :)

[Tetedeiench]

@balazsorban44 Please consider reopening the issue or participating in the conversation here.

I'm also afraid of the time where this will produce an error.

Not everyone is using Vercel / Cloud services, there is a world beyond this : i'm self-hosting on a dedicated server, thus i'm fully responsible for the resource usage on my server. There can be a warning to tell me, I have no issue with that. But I would like to be fully responsible for my actions and thus :

• Be able to disable that warning if I judge it doesn't apply
• Still be able to use that function if it fits my use case

Especially since it's an easy solution to implement without going all out for a CDN which is probably overkill in many, many situations. It adds an extra layer of complexity and cost for little to no gain in many use cases.

My personal use case is gating the downloads behind an authentication mechanism ( next-auth ).

To give you some numbers :

• The gated files are about 30MB and are downloaded 20 000 times per day in my case (which amounts to about 600GB per day)
• I don't have any performance or CPU usage issues (quite the contrary, it's really low, congrats !). Same with network. It's just working like a charm.
• I have NO complains about users having slow downloads or issue with this mechanism
• Website is working like a charm

I agree with @700software here : give us a way to disable this warning, and forbid or do whatever you want on Vercel to prevent it for being used. Restricting it for everyone is just too broad.

I personally think you're implementing a Vercel/Cloud oriented warning inside Next.js, which is not the place for that limit to be.

[crice88]

@balazsorban44 Hoping you can reopen this issue and allow a PR to be made. I do not use Vercel and have a use case in streaming large files back to the client.

Trying to load large files first in memory or moving them from S3 and back again just won't work for me.

We allow users to download their folders/files from S3 (kinda like Dropbox). When a folder download request comes in we pull from S3, create directories and zip the whole package up and stream it back to the client. For us to take that zip, push it back out to S3 and request a presigned URL is just not as performant (not to mention it can get costly) as streaming the data across.

I agree with both @Tetedeiench and @700software that there should be a way to configure this option.