Link prefetch requests are throwing 404 errors, but only in production.

[jaandrews]

Link to the code that reproduces this issue

https://codesandbox.io/p/sandbox/friendly-ganguly-qmrdl2

To Reproduce

The provided reproduction link doesn't reproduce the issue, as I only encounter the problem in the production environment, but it is a simplified version of the code that I am having issues with. A page with the issue can be seen here. To reproduce my production environment, use the following steps.

1. Deploy nextjs app with multiple Link tags on a page to azure static web app.
2. Put that web app behind an azure front door instance
3. Visit the site and check the logs to see whether the prefetch requests worked

Current vs. Expected behavior

The Link prefetch request throws a 404 error for a page that exists, when it should be returning the json data.

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

Get an error when I run next info because the package.json file isn't registered in npmjs.org (nor is it supposed to be).

Operating System
Not sure, as the documentation doesn't specify, but I suspect it's running Linux.

Binaries
node: 16.13.2
npm: 8.18.0

Packages
next: 13.5.1
babel-plugin-superjson-next: 0.4.3 (used to return date objects for typescript, since next leaves them as strings)

Which area(s) are affected? (Select all that apply)

Data fetching (gS(S)P, getInitialProps)

Additional context

I'm not sure why it's unable to retrieve the data, as the pages themselves resolve when visited and the data requests work fine in the development environment.

[TrustyTechSG]

Im having the same issue, after upgrade from 13.4.19 to 13.5.2, local dev all works fine, but after deploy to Vercel. keep having prefetch module not found error log in Vercel, and every client side prefetch link failed with 404. Even dynamic params somehow all appended with .prefetch.

[lewisking]

Yep, having similar issues on https://stableaudio.com

[JNer1]

Same issue as well. At least for me, it seams to only affect dynamic routes. It works on my local machine even if I run it with next start. But once I deploy it to my EC2 instance, I get the 404 error. Rolling back to 13.4.8 works for me.

[fasani-tx]

Do you have a CDN in place in production?

I have noticed the use of several headers in Next and am investigating the same issue, perhaps unknown headers being blocked by CDN.

[JNer1]

Same issue as well. At least for me, it seams to only affect dynamic routes. It works on my local machine even if I run it with next start. But once I deploy it to my EC2 instance, I get the 404 error. Rolling back to 13.4.8 works for me.

I figured out that what was causing my issue was how my proxy pass was configured for Nginx. My original configuration had a trailing slash. Removing it seemed to fix the problem of getting 404 errors on dynamic routes.

Would cause 404

location / {
        proxy_pass http://127.0.0.1:3001/;
        }

Fixes 404

location / {
        proxy_pass http://127.0.0.1:3001;
        }

[matthewwilson]

We are having a similar issue, when you click on the Link its taking you to https://myurl/mypagename.prefetch which is then causing a 404

In the Vercel logs I can see 404 warnings for pages and there is a double .prefetch.prefetch in the url

[rebeccapeltz]

same issue: I reported it here: #56724

I think this is a regression because we weren't seeing these 404 errors in earlier releases.

[digitalthjarta]

I have this on Azure Static Webapps

[jaandrews]

I confirmed that I still run into the problem without front door being involved. Also have a few more details about the problem. In my case the problem is that "_next" isn't being treated as a reserved route. When these requests come in they are run through [[...slug]].tsx instead of returning the json data and so end up making an api call that is supposed to result in an 404 error because no page starts with "_next" in the cms it pulls data from. The requests that fail have the structure "/_next/data/mScWIefhkWiUcXJNdpr9Q/local/path/to/example.json?slug=path&slug=to&slug=example."

[jaandrews]

I tried reverting to version 13.4.8, as suggested earlier and that did fix the problem for me (thought I already had tested it, but I updated from "^13.0.0" to 13.4.19). So the issue was introduced between 13.4.8 and 13.4.19. It hasn't been fixed in the most recent version of 13 either (13.5.6).

[beckypeltz-hdrinc]

I tried reverting to version 13.4.8, as suggested earlier and that did fix the problem for me (thought I already had tested it, but I updated from "^13.0.0" to 13.4.19). So the issue was introduced between 13.4.8 and 13.4.19. It hasn't been fixed in the most recent version of 13 either (13.5.6).

Have you tried v14? If so, did it fix this problem?

[jaandrews]

@beckypeltz-hdrinc I haven't tested version 14 because that includes breaking changes and so would be more time consuming to implement. Will likely test it at some point, but it's not a priority for me.

[maxijonson]

I believe I have a similar issue but using the App directory as well. The issue wasn't happening in dev or local build, but always on Vercel.

After a long day of debugging this issue, I found out that it's because prefetched requests append a .prefetch in the URL. So a route like /blog/some-blog-slug would actually be prefetching /blog/some-blog-slug.prefetch and my params.slug would be equal to "some-blog-slub.prefetch", which isn't a valid slug and would result in an error.

I was able to fix it by filtering the .prefetch suffix. However, this raises more concerns, because this is somewhat unpredictable behavior, especially since all <Link> have prefetching enabled. Either way, I believe this is a mistake. My params shouldn't have .prefetch suffix just because they happen to be at the end of the URL...

This .prefetch bug was identified on v14.0.3, but the behaviour was initially observed on v13.5.4

[pablohpsilva-forjnetwork]

@maxijonson it is happening in my project as well. Same exact thing you mentioned

[jaandrews]

I just tested my code with version 14.1.0 and this issue still exists there.

[aligajani]

@jaandrews Have they fixed it in later versions, I am on 14.1.0 too.

[jaandrews]

@aligajani I haven't tested with the latest version.

[rijk]

I have the same issue; filtering out .prefetch manually partly fixes the issue, but not all the way. This is because it also seems to impact the route resolution on Vercel. For example, given these two paths:

1. [course]/resources/page.tsx
   2.[course]/[lesson]/page.tsx

✅ When I request mysite.com/some-course/resources with RSC: 1
→ I get the page with X-Matched-Path: /[course]/resources.rsc

🚫 When I add the Next-Router-Prefetch: 1 header though
→ It fails 404 with X-Matched-Path: /[course]/[lesson].rsc

I believe this to be because the request is changed internally to something like mysite.com/some-course/resources.prefetch, which no longer resolves to the static resources path. So this is something I cannot solve by manually overriding the params.

[rijk]

I think I tracked it down to my use of a rewrite. Without it seems to work as designed.

I've made a minimal repro here: https://github.com/rijk/next-prefetch-issue

Deployed on https://next-prefetch-issue.vercel.app/

To reproduce:

• GET https://next-prefetch-issue.vercel.app/course/resources with RSC: 1 → good
• GET https://next-prefetch-issue.vercel.app/course/resources with RSC: 1 and Next-Router-Prefetch: 1 → bad, and note X-Matched-Path header
• GET https://next-prefetch-issue.vercel.app/course/lesson-1 with RSC: 1 → good
• GET https://next-prefetch-issue.vercel.app/course/lesson-1 with RSC: 1 and Next-Router-Prefetch: 1 → bad

It's not as bad as in my app, where I get a 404 status, which also shows up in the console and completely breaks client side routing. But you can already see the routing issue. I'll keep trying to reproduce that 404 as well.

[rijk]

This seems to have been fixed on Vercel's end, I just did a redeploy of my latest commit and the prefetch 404s are gone.

[jaandrews]

@rijk Are you using the app router or pages router? I'm still having the issue I mentioned earlier. Currently running nextjs 14.2.1, but I don't see any mentioned in the patch notes for later versions that looks like a fix for the issue.

[rijk]

I am using the app router. However, the issue seemed to be in Vercel's code, not Next, because it was fixed with a redeploy without any changes to my code. But it's still possible Vercel behavior is different for pages/app router I guess.

[jaandrews]

@rijk K. Thanks for the clarification.