Ensure we dedupe fetch requests properly during page revalidate

[ijjk]

When we are revalidating an entire page and a specific fetch cache entry is stale we refresh the data in the foreground so that the page cache doesn't contain the stale data. When this foreground fetch updating occurs we weren't de-duping the requests like we were when it was done in the background which could cause unexpected upstream requests. This copies over the deduping handling we have from the background handling to the foreground handling.

x-ref: slack thread

Closes NEXT-2966

[ijjk]

Stats from current PR

Default Build

General Overall increase ⚠️

	vercel/next.js canary	ijjk/next.js fix/dedupe-fetch-stale	Change
buildDuration	14s	14.1s	N/A
buildDurationCached	8.6s	6.3s	N/A
nodeModulesSize	199 MB	199 MB	⚠️ +50.1 kB
nextStartRea..uration (ms)	402ms	401ms	N/A

Client Bundles (main, webpack)

	vercel/next.js canary	ijjk/next.js fix/dedupe-fetch-stale	Change
2453-HASH.js gzip	31.4 kB	31.4 kB	N/A
3304.HASH.js gzip	181 B	181 B	✓
3f784ff6-HASH.js gzip	53.7 kB	53.7 kB	✓
8299-HASH.js gzip	5.04 kB	5.04 kB	N/A
framework-HASH.js gzip	45.2 kB	45.2 kB	✓
main-app-HASH.js gzip	242 B	241 B	N/A
main-HASH.js gzip	32.2 kB	32.2 kB	N/A
webpack-HASH.js gzip	1.68 kB	1.68 kB	N/A
Overall change	99 kB	99 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	ijjk/next.js fix/dedupe-fetch-stale	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	ijjk/next.js fix/dedupe-fetch-stale	Change
_app-HASH.js gzip	196 B	197 B	N/A
_error-HASH.js gzip	184 B	184 B	✓
amp-HASH.js gzip	505 B	505 B	✓
css-HASH.js gzip	324 B	325 B	N/A
dynamic-HASH.js gzip	2.5 kB	2.5 kB	N/A
edge-ssr-HASH.js gzip	258 B	258 B	✓
head-HASH.js gzip	352 B	352 B	✓
hooks-HASH.js gzip	370 B	371 B	N/A
image-HASH.js gzip	4.21 kB	4.21 kB	✓
index-HASH.js gzip	259 B	259 B	✓
link-HASH.js gzip	2.67 kB	2.67 kB	N/A
routerDirect..HASH.js gzip	314 B	312 B	N/A
script-HASH.js gzip	386 B	386 B	✓
withRouter-HASH.js gzip	309 B	309 B	✓
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	6.57 kB	6.57 kB	✓

Client Build Manifests

	vercel/next.js canary	ijjk/next.js fix/dedupe-fetch-stale	Change
_buildManifest.js gzip	481 B	484 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	ijjk/next.js fix/dedupe-fetch-stale	Change
index.html gzip	530 B	529 B	N/A
link.html gzip	542 B	542 B	✓
withRouter.html gzip	524 B	523 B	N/A
Overall change	542 B	542 B	✓

Edge SSR bundle Size

	vercel/next.js canary	ijjk/next.js fix/dedupe-fetch-stale	Change
edge-ssr.js gzip	95.3 kB	95.3 kB	N/A
page.js gzip	3.04 kB	3.04 kB	N/A
Overall change	0 B	0 B	✓

Middleware size

	vercel/next.js canary	ijjk/next.js fix/dedupe-fetch-stale	Change
middleware-b..fest.js gzip	626 B	625 B	N/A
middleware-r..fest.js gzip	151 B	151 B	✓
middleware.js gzip	25.5 kB	25.5 kB	N/A
edge-runtime..pack.js gzip	839 B	839 B	✓
Overall change	990 B	990 B	✓

Next Runtimes

	vercel/next.js canary	ijjk/next.js fix/dedupe-fetch-stale	Change
app-page-exp...dev.js gzip	170 kB	170 kB	✓
app-page-exp..prod.js gzip	97 kB	97 kB	✓
app-page-tur..prod.js gzip	98.8 kB	98.8 kB	✓
app-page-tur..prod.js gzip	93 kB	93 kB	✓
app-page.run...dev.js gzip	144 kB	144 kB	✓
app-page.run..prod.js gzip	91.5 kB	91.5 kB	✓
app-route-ex...dev.js gzip	21.4 kB	21.4 kB	N/A
app-route-ex..prod.js gzip	15.1 kB	15.2 kB	N/A
app-route-tu..prod.js gzip	15.1 kB	15.2 kB	N/A
app-route-tu..prod.js gzip	14.8 kB	14.9 kB	N/A
app-route.ru...dev.js gzip	21 kB	21.1 kB	N/A
app-route.ru..prod.js gzip	14.8 kB	14.9 kB	N/A
pages-api-tu..prod.js gzip	9.55 kB	9.55 kB	✓
pages-api.ru...dev.js gzip	9.82 kB	9.82 kB	✓
pages-api.ru..prod.js gzip	9.55 kB	9.55 kB	✓
pages-turbo...prod.js gzip	22.5 kB	22.5 kB	✓
pages.runtim...dev.js gzip	23.1 kB	23.1 kB	✓
pages.runtim..prod.js gzip	22.4 kB	22.4 kB	✓
server.runti..prod.js gzip	51 kB	51 kB	✓
Overall change	842 kB	842 kB	✓

build cache

	vercel/next.js canary	ijjk/next.js fix/dedupe-fetch-stale	Change
0.pack gzip	1.58 MB	1.58 MB	N/A
index.pack gzip	106 kB	106 kB	N/A
Overall change	0 B	0 B	✓

Diff details

Diff for middleware.js

Diff too large to display

Diff for app-route-ex..ntime.dev.js

Diff too large to display

Diff for app-route-ex..time.prod.js

Diff too large to display

Diff for app-route-tu..time.prod.js

Diff too large to display

Diff for app-route-tu..time.prod.js

Diff too large to display

Diff for app-route.runtime.dev.js

Diff too large to display

Diff for app-route.ru..time.prod.js

Diff too large to display

Commit: 2ca21ee

[ijjk]

Failing test suites

Commit: 2ca21ee

TURBOPACK=1 pnpm test-start test/e2e/prerender.test.ts (turbopack)

• Prerender > should on-demand revalidate for fallback: blocking with onlyGenerated if generated

Expand output

● Prerender › should on-demand revalidate for fallback: blocking with onlyGenerated if generated

expect(received).toMatch(expected)

Expected pattern: /(HIT|STALE)/
Received string:  "MISS"

  2297 |
  2298 |         expect(initialTime).toBe($2('#time').text())
> 2299 |         expect(res2.headers.get('x-nextjs-cache')).toMatch(/(HIT|STALE)/)
       |                                                    ^
  2300 |
  2301 |         const res3 = await fetchViaHTTP(
  2302 |           next.url,

  at Object.toMatch (e2e/prerender.test.ts:2299:52)

Read more about building and testing Next.js in contributing.md.