How to stop Next.js from caching notFound() renders for invalid dynamic route params?

[aliasgary1]

Summary

Title: notFound() results for bot-scanned invalid dynamic params get permanently cached, bloating disk — tried Cache Components, issue persists

Environment

• Next.js 16.2.12 (App Router, Turbopack)
• Self-hosted with next build && next start, single Node.js process (PM2, fork mode)
• Route in question: app/[slug]/page.jsx, ~2,200 legitimate song pages (a
  music streaming site), with the catalog growing continuously

The original problem

We had a straightforward dynamic route with the classic static-generation
model:

// app/[slug]/page.jsx
export const dynamic = 'force-static';

async function getSong(slug) {
  const res = await fetch(`${INTERNAL_API}/song/${slug}`, {
    headers: { ... },
    next: { revalidate: 60 * 60 * 24, tags: [`song-${slug}`] },
  });
  return res.json();
}

export default async function SongPage({ params }) {
  const { slug } = await params;
  const song = await getSong(slug);
  if (!song || song.error) {
    notFound();
  }
  // ... render
}

We deliberately did not use generateStaticParams with the full list of
~2,200+ slugs, because the catalog grows continuously and pre-rendering the
entire set at build time would add substantial and ever-increasing build
time. Instead we relied on dynamic = 'force-static' with the default
dynamicParams: true behavior — i.e. an ISR/fallback: blocking-style
pattern: new/unlisted slugs get rendered and cached on their first real
request.

What we discovered

Any public-facing dynamic route without a fixed, enumerable set of valid
params inevitably gets hit by bots/scanners probing for things like
wp-login.php, .env, backup.sql, or simply malformed/garbled slugs
(URL-encoded garbage, etc.). Every one of these is a unique param value that
resolves to notFound().

We found that Next.js persisted a cache entry (.html / .meta / .rsc
under .next/server/app) for every one of these invalid, bot-generated
slugs, exactly as if they were successful pages. Since bots generate an
effectively unbounded stream of unique garbage URLs, our .next cache grew
to ~2GB for a site that actually only has ~2,200 real pages, and kept
growing indefinitely with no natural ceiling.

Approaches we considered before touching caching config, and why we
didn't use them

1. generateStaticParams returning the full list of valid slugs, with
   dynamicParams: false. Rejected for the reason above (unbounded,
   continuously growing build-time cost as the catalog grows).

2. dynamic = 'force-dynamic' / opting the whole route out of static
   rendering. Rejected because it discards instant, cached delivery (and
   the associated performance/SEO benefits) for the ~2,200 legitimate
   pages, which is the entire reason we wanted static/ISR-style rendering
   in the first place. We only want to avoid persisting invalid params,
   not give up caching for valid ones.

3. Redirecting invalid slugs to a canonical /404 page instead of
   calling notFound(). Considered, but rejected because (a) it doesn't
   avoid the underlying issue — a unique cache entry still gets generated
   per invalid param, just for a redirect instead of an HTML page — and
   (b) it produces a soft-404 (redirect + 200 on the destination) instead
   of a proper 404 status, which is worse for SEO than the current
   behavior.

4. Relying on revalidate as a ceiling on how long a bad entry
   survives. This doesn't prevent the initial write and doesn't stop
   unbounded growth from a continuous stream of unique bot-generated
   slugs — each is a distinct param, so revalidation just refreshes each
   one individually rather than reducing their count.

We found this described as a known, long-standing behavior here:
#63483 (closed as not planned).

We then migrated to Cache Components, expecting it to solve this

Based on the mental model that "use cache is opt-in, everything else is
dynamic by default," we migrated the route to cacheComponents: true,
moving the data fetch into an explicitly cached function and calling
notFound() outside of it:

async function getSong(slug) {
  "use cache";
  cacheLife({ revalidate: 60 * 60 * 24 });
  cacheTag(`song-${slug}`);

  const res = await fetch(`${INTERNAL_API}/song/${slug}`, { headers: { ... } });
  if (!res.ok) return null;
  const song = await res.json();
  if (!song || song.error) return null;
  return song;
}

export async function generateStaticParams() {
  // small sample set only — same reasoning as before, cannot pre-render
  // the whole growing catalog at build time
  return [{ slug: "some-real-song" }, /* ... */];
}

export default function Page({ params }) {
  return (
    <Suspense fallback={...}>
      <SongContent params={params} />
    </Suspense>
  );
}

async function SongContent({ params }) {
  const { slug } = await params;
  const song = await getSong(slug);
  if (!song) {
    notFound(); // called OUTSIDE the 'use cache' boundary, intentionally
  }
  // ... render
}

Our expectation was that since notFound() runs outside any 'use cache'
scope, it would never be persisted — only the successful getSong() result
would be.

This did not solve the problem, and we found out why

For a slug that doesn't exist, the documented behavior for unlisted dynamic
params applies regardless of the render's outcome:

"Any other URL is served the App Shell instantly, then upgraded in the
background with its now-known params and cached for the next visitor."
— https://nextjs.org/docs/app/getting-started/caching

Empirically, for an invalid slug we still see:

• .html / .meta / .rsc files written under .next/server/app.
• A second request to the same invalid slug returns 304 Not Modified,
  confirming the not-found render itself was cached as the "upgraded
  result," not just the underlying data fetch.

So 'use cache' only governs the data cache (the getSong fetch) — it
does not govern whether the page-level render for a given dynamic param
gets persisted by the ISR/App-Shell-upgrade mechanism, which persists per
unique param regardless of success or notFound(). In other words, Cache
Components changed how the data is cached, but did not change the
underlying per-param persistence behavior that caused our original problem.

We also found that an equivalent to dynamicParams: false for
cacheComponents is explicitly requested but doesn't currently exist
(dynamicParams itself is rejected at build time when cacheComponents is
enabled):
#84991

Our current workaround

We're gating requests in middleware.js, checking the requested slug
against an in-memory allowlist (a Set populated from our backend and
refreshed on song create/delete) before the request ever reaches the
route's render pipeline, so invalid params never get a chance to enter the
App-Shell-upgrade/caching path at all. This works, but it means
reimplementing, entirely outside the framework, exactly the kind of
validation that dynamicParams: false used to give us for free before
Cache Components.

Question for the team

1. Is there a supported/recommended way, within the Cache Components model,
   to prevent a specific dynamic param's resolved render — specifically one
   that ends in notFound() — from being persisted by the App-Shell
   background-upgrade mechanism, without maintaining an external allowlist
   in Middleware?
2. Is the dynamicParams-equivalent requested in discussion `dynamicParams = false` for Cache Components & non-inheriting behavior #84991 on the
   roadmap? If so, is Middleware-based gating considered the interim
   recommended pattern until then, or is there a different idiom we should
   be using (e.g. something along the lines of the interceptor.ts sketch
   in that discussion)?

Happy to provide a minimal reproduction repo if useful.

Additional information

No response

Example

No response

[ryux1]

Your reading of the two cache layers is correct. In the current Cache Components model there is no per-result switch that means “cache successful params, but do not persist this notFound() result.”

The current docs make the boundaries fairly explicit:

• use cache controls the return value of a cached function/component.
• Unknown dynamic params are served with the App Shell, then ISR fills in and stores the concrete page after the first visit.
• dynamicParams was removed when Cache Components is enabled.
• Most importantly, the current notFound() docs say that Cache Components streams a shell for every dynamic route and that, when a real 404 must be decided before streaming, the check should run in proxy.

So for question 1, your gate is currently the supported shape. Have proxy.ts target only this route with a narrow matcher, reject obviously invalid slug syntax cheaply, then consult an authoritative fast lookup for the remaining candidates and return a 404 response before the request reaches the dynamic route. Valid new slugs pass through and still get on-demand ISR.

One adjustment: I would not make correctness depend on a mutable module-level Set. The Proxy docs warn that Proxy executes separately from render code and specifically say not to rely on shared modules or globals. A single PM2 process may make that appear stable today, but it becomes fragile across restarts, multiple workers, or a deployment change. A bounded shared KV/cache (or a backend existence endpoint), optionally preceded by syntax filtering and rate limiting, is safer. If this is already enforced at a CDN, doing the same check there is equivalent and avoids invoking Next.js for scanner traffic.

For question 2, I cannot find a public roadmap commitment. The linked #84991 still has no accepted or maintainer answer committing to an API, and the current route-segment documentation still records dynamicParams as removed under Cache Components. The interceptor.ts sketch therefore should not be treated as available or scheduled.

That leaves three supported trade-offs today:

1. Cache Components plus Proxy or CDN validation: keeps on-demand caching for valid slugs and prevents invalid requests entering the render or ISR path.
2. The previous cache model plus dynamicParams = false: requires enumerating accepted params.
3. Fully dynamic rendering: avoids per-param ISR persistence but gives up the page cache.

Given your requirements, option 1 is the only current combination that preserves on-demand ISR for a growing catalog while placing a hard boundary in front of unbounded negative params.

Official references:

• https://nextjs.org/docs/app/api-reference/functions/not-found#calling-notfound-after-streaming-has-started
• https://nextjs.org/docs/app/getting-started/caching#incremental-static-regeneration
• https://nextjs.org/docs/app/api-reference/file-conventions/route-segment-config
• https://nextjs.org/docs/app/api-reference/file-conventions/proxy#execution-order

[Ldenis06]

This is an outstanding write-up of a critical edge case in Next.js's App Router rendering pipeline.

The Root Cause: Why notFound() persists in Cache Components

Under cacheComponents (PPR / App Shell model), the route shell is rendered statically, and unlisted dynamic segments trigger a background upgrade pass. Because notFound() throws an internal control-flow signal (NEXT_NOT_FOUND) rather than a cache-abort signal, the background compilation treats the resulting 404 UI render as the canonical "upgraded" output for that route parameter, persisting .html, .meta, and .rsc payloads to disk.

1. The Short Answer regarding Framework Support

Currently, there is no built-in route-level directive in the Cache Components model to tell the App Shell upgrade engine: "abort persistence if this render throws notFound()".

As you noted, dynamicParams = false is disabled under cacheComponents because the routing model assumes dynamic param resolution happens at runtime via dynamic IO / streaming rather than upfront param enumeration.

2. Recommended Alternatives to an In-Memory Middleware Set

While your current Middleware Set approach works, it doesn't scale well across multiple PM2 instances or horizontally scaled nodes. Here are the two standard patterns for handling this without blowing up the .next disk cache:

Option A: Custom Incremental Cache Handler with Status Filter (Recommended)

Since you are self-hosting on a single Node/PM2 instance via next start, you can implement a custom Cache Handler (cacheHandler in next.config.js).

A custom cache handler intercepts all read/write operations to the disk cache. You can hook into the set method and ignore entries that resolve to a 404 / notFound status or metadata:

// cache-handler.js
const FileSystemCache = require('next/dist/server/lib/incremental-cache/file-system-cache').default;

class FilteredCacheHandler {
  constructor(options) {
    this.fsCache = new FileSystemCache(options);
  }

  async get(key) {
    return this.fsCache.get(key);
  }

  async set(key, data, ctx) {
    // Prevent persisting renders that resulted in notFound / 404 status
    if (data?.kind === 'APP_PAGE' && data?.pageData?.status === 404) {
      return;
    }
    // Alternatively inspect ctx or headers if status is stripped
    return this.fsCache.set(key, data, ctx);
  }

  async revalidateTag(tag) {
    return this.fsCache.revalidateTag(tag);
  }
}

module.exports = FilteredCacheHandler;

[icyJoseph]

We are working on an API to allow you to control this behavior, it is a bit early to share impl and API details, but it'll happen soon.

[MarvelCollin]

we had this same problem on a project with user generated content slugs. the core issue is that the caching layer and the rendering layer are two separate systems. use cache only controls the data fetching cache, meaning the return value of your cached function. but the App Shell upgrade mechanism operates at the routing level. when a request comes in for a dynamic param that wasnt prerendered, the router serves the shell immediately then does a background render for that specific param and persists the full result regardless of whether the page ended up calling notFound() or rendering successfully. from the routers perspective, a 404 response for that param IS the resolved result, so it caches it just like any other.

this means there is no way from inside the page component to tell the router not to persist the result. notFound() is a rendering outcome, but the persistence decision was already made at the routing layer before your component even ran.

your middleware approach is the right call. one optimization we found useful on top of the Set lookup is adding a fast regex pre-filter before hitting the Set. most bot traffic follows obvious patterns like slugs with file extensions, path traversals, or known scanner prefixes. filtering those with a cheap regex before doing the Set.has() lookup cut our middleware response time since the regex handles the bulk of bot junk without touching the allowlist.

for the dynamicParams equivalent under Cache Components, sounds like the team is working on something based on icyJosephs reply. until then middleware gating is the way to go.

[itzmitto]

Good news — this is a genuinely well-researched question, and I can confirm your diagnosis is accurate based on what's actually documented, plus point you to the real answer.

Your read on the mechanism is correct, and it's confirmed by Next.js's own Cache Components docs on ISR: once an unknown param resolves (App Shell → background upgrade), subsequent visits to that URL get served the upgraded result straight from cache, bypassing the App Shell entirely. Nothing in that doc distinguishes a successful render from a notFound() result — both are "the resolved outcome for this param," and both get persisted. So use cache only ever governed your data fetch; the page-level ISR/App-Shell-upgrade cache is a separate layer that persists per unique param regardless of what the render produced. You didn't miss an API — it currently doesn't exist.

On your two direct questions:

There's no supported way to prevent a specific param's notFound() render from being persisted by the App-Shell upgrade, short of stopping it from reaching the render pipeline at all (which is what your Middleware allowlist does).
Discussion #84991 is exactly the tracking thread for this gap, and it confirms dynamicParams: false has no Cache Components equivalent yet. The interesting part is the proposed interceptor.ts API sketched there — a validation step that runs before rendering starts, calling notFound() at that stage without opting the route into dynamic rendering. That's precisely the shape of what you're asking for: reject before render, don't cache the rejection. But as of that discussion, it's a proposal from the Next.js team member, not a shipped API.

So to answer directly: yes, Middleware-based gating is currently the correct interim pattern — you're not missing a supported alternative, you've essentially hand-rolled the interceptor.ts concept ahead of its release. One refinement that might reduce your allowlist maintenance burden: since most of your bot traffic is probably hitting obviously-invalid patterns (.php, .env, .sql, encoded garbage) rather than almost-valid slugs, you could pair a cheap regex/pattern reject in Middleware with your allowlist, so the allowlist only has to arbitrate the smaller set of plausible-looking slugs.

Given you've already got a clean repro and a well-articulated case, it's worth adding a comment to #84991 describing this bot-scanning scenario specifically — unbounded cache growth from adversarial traffic is a sharper motivating example than the i18n locale case that thread currently centers on, and could help push the interceptor API up the priority list.

[NeoTim]

I'd suggest the following approach:

1. Check if you're using the latest version - many issues are resolved in recent releases.
2. Review the official documentation for any breaking changes.
3. Try clearing your cache/node_modules and reinstalling dependencies.

If none of these work, providing a minimal reproduction repo would help the maintainers diagnose the issue faster.

[aliasgary1]

@NeoTim thanks for the suggestions — to make sure we're on the same page,
could you clarify how they apply here? The issue isn't a crash or a
version-specific bug we're trying to troubleshoot; it's a documented,
by-design behavior (confirmed by ryux1, MarvelCollin, itzmitto above, and
by icyJoseph that a new API is being worked on) where the App-Shell/ISR
persistence layer caches the resolved render for any dynamic param —
success or notFound() — independent of the Next.js version or dependency
state.

If you're saying you have a working setup that avoids this without an
external gate (Proxy/CDN), that would genuinely be useful to see, since it
contradicts what's documented and what the maintainers/collaborator have
said in this thread so far. Could you share:

1. The exact route structure you're using — e.g. is it a dynamic segment
   like src/app/[slug]/page.jsx with a large/growing, non-enumerable set
   of valid params (in our case ~2,200 song pages)?
2. Whether that route uses static/ISR-style caching (dynamic = 'force-static' pre-Cache-Components, or 'use cache' +
   cacheLife/cacheTag under cacheComponents: true) with periodic
   revalidate, similar to what's in our repro above?
3. Whether you're using proxy.ts (formerly middleware.ts) to gate/validate
   requests before they reach that route at all, or whether you're relying
   on something else entirely?
4. If proxy.ts is involved, the actual code — specifically how it decides
   a param is valid, and how/when that validation data gets loaded and
   kept in sync?

If it turns out proxy-based gating is what you're using too, that's a
useful confirmation that ryux1/itzmitto's read is the correct interim
pattern. If it's something different, we'd like to understand what.

[jhmartins1]

One practical way to make the current workaround safer is to move the validation into proxy.ts and avoid relying on a process-local Set.

For example, you can cheaply reject obvious scanner traffic first, then only call your authoritative lookup for plausible slugs:

import { NextRequest, NextResponse } from 'next/server'

const INVALID_PATTERNS = [
/.php$/i,
/.env$/i,
/.sql$/i,
/../,
]

export async function proxy(request: NextRequest) {
const pathname = request.nextUrl.pathname
const match = pathname.match(/^/([^/]+)$/)

if (!match) {
return NextResponse.next()
}

const slug = decodeURIComponent(match[1])

if (INVALID_PATTERNS.some((pattern) => pattern.test(slug))) {
return new NextResponse('Not Found', { status: 404 })
}

const response = await fetch(
${process.env.INTERNAL_API}/song/exists/${encodeURIComponent(slug)},
{
headers: {
authorization: Bearer ${process.env.INTERNAL_API_TOKEN},
},
}
)

if (!response.ok) {
return new NextResponse('Not Found', { status: 404 })
}

return NextResponse.next()
}

export const config = {
matcher: '/:slug',
}

I would prefer this over a mutable module-level allowlist because proxy.ts should not depend on process-local state staying synchronized across restarts or multiple workers.

This still isn't as nice as having a Cache Components equivalent of dynamicParams = false, but it prevents invalid bot-generated params from entering the App Shell/ISR path at all while preserving on-demand caching for valid slugs.

Based on the maintainer reply above, it sounds like this is the best interim option until the new control API lands.

[fernandomunizz89]

One issue with this example: !response.ok also catches 401, 429 and 5xx responses. Returning 404 for all of those would make existing songs appear missing during an API outage.
I'd return 404 only when the endpoint explicitly confirms the song doesn't exist, and handle upstream failures separately, e.g. with a 503. The same distinction applies to the original getSong(), which currently returns—and caches—null for any non-OK response.