Replace withCoalescedInvoke with ResponseCache

[devknoll]

By itself, withCoalescedInvoke with a separate this.incrementalCache.set(...) isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.

This could potentially be addressed by moving the cache update, but then IncrementalCache itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.

As an added bonus, renderToResponseWithComponents becomes significantly simpler, and we delete some duplication.

[ijjk]

Failing test suites

Commit: 1975b62

test/integration/route-load-cancel/test/index.test.js

• next/dynamic > dev mode > should cancel slow page loads on re-navigation

Expand output

● next/dynamic › dev mode › should cancel slow page loads on re-navigation

NoSuchElementError: no such element: Unable to locate element: {"method":"css selector","selector":"#page-text"}
  (Session info: headless chrome=91.0.4472.114)

  31 |     await waitFor(1000)
  32 |
> 33 |     const text = await browser.elementByCss('#page-text').text()
     |                  ^
  34 |     expect(text).toMatch(/2/)
  35 |     expect(await browser.eval('window.routeCancelled')).toBe('yes')
  36 |   })

  at Object.throwDecodedError (../node_modules/selenium-webdriver/lib/error.js:550:15)
  at parseHttpResponse (../node_modules/selenium-webdriver/lib/http.js:565:13)
  at Executor.execute (../node_modules/selenium-webdriver/lib/http.js:491:26)
  at thenableWebDriverProxy.execute (../node_modules/selenium-webdriver/lib/webdriver.js:700:17)
  at Object.<anonymous> (integration/route-load-cancel/test/index.test.js:33:18)

[shuding]

The implementation of response-cache looks great to me, also the code structure is so much simpler now, thanks!

[ijjk]

Stats from current PR

Default Build (Decrease detected ✓)

General Overall increase ⚠️

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
buildDuration	12.5s	12.4s	-106ms
buildDurationCached	3s	2.9s	-55ms
nodeModulesSize	49.3 MB	49.3 MB	⚠️ +410 B

Page Load Tests Overall decrease ⚠️

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
/ failed reqs	0	0	✓
/ total time (seconds)	2.041	2.046	0
/ avg req/sec	1224.76	1222	⚠️ -2.76
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	1.103	1.126	⚠️ +0.02
/error-in-render avg req/sec	2266.7	2219.76	⚠️ -46.94

Client Bundles (main, webpack, commons)

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
359.HASH.js gzip	3.09 kB	3.09 kB	✓
framework-HASH.js gzip	42 kB	42 kB	✓
main-HASH.js gzip	20.9 kB	20.9 kB	✓
webpack-HASH.js gzip	1.49 kB	1.49 kB	✓
Overall change	67.6 kB	67.6 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
polyfills-HASH.js gzip	31.1 kB	31.1 kB	✓
Overall change	31.1 kB	31.1 kB	✓

Client Pages

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
_app-HASH.js gzip	803 B	803 B	✓
_error-HASH.js gzip	3.18 kB	3.18 kB	✓
amp-HASH.js gzip	526 B	526 B	✓
css-HASH.js gzip	329 B	329 B	✓
hooks-HASH.js gzip	903 B	903 B	✓
image-HASH.js gzip	5.69 kB	5.69 kB	✓
index-HASH.js gzip	261 B	261 B	✓
link-HASH.js gzip	1.66 kB	1.66 kB	✓
routerDirect..HASH.js gzip	319 B	319 B	✓
withRouter-HASH.js gzip	320 B	320 B	✓
bb14e60e810b..30f.css gzip	125 B	125 B	✓
Overall change	14.1 kB	14.1 kB	✓

Client Build Manifests

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
_buildManifest.js gzip	419 B	419 B	✓
Overall change	419 B	419 B	✓

Rendered Page Sizes

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
index.html gzip	531 B	531 B	✓
link.html gzip	544 B	544 B	✓
withRouter.html gzip	524 B	524 B	✓
Overall change	1.6 kB	1.6 kB	✓

---

Webpack 4 Mode (Decrease detected ✓)

General Overall increase ⚠️

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
buildDuration	10s	10.2s	⚠️ +203ms
buildDurationCached	4s	4s	-40ms
nodeModulesSize	49.3 MB	49.3 MB	⚠️ +410 B

Page Load Tests Overall decrease ⚠️

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
/ failed reqs	0	0	✓
/ total time (seconds)	2.07	2.081	⚠️ +0.01
/ avg req/sec	1207.93	1201.18	⚠️ -6.75
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	1.131	1.143	⚠️ +0.01
/error-in-render avg req/sec	2210.66	2187.15	⚠️ -23.51

Client Bundles (main, webpack, commons)

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
14.HASH.js gzip	3.11 kB	3.11 kB	✓
677f882d2ed8..HASH.js gzip	13.9 kB	13.9 kB	✓
framework.HASH.js gzip	41.8 kB	41.8 kB	✓
main-HASH.js gzip	8.17 kB	8.17 kB	✓
webpack-HASH.js gzip	1.19 kB	1.19 kB	✓
Overall change	68.1 kB	68.1 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
polyfills-HASH.js gzip	31.3 kB	31.3 kB	✓
Overall change	31.3 kB	31.3 kB	✓

Client Pages

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
_app-HASH.js gzip	791 B	791 B	✓
_error-HASH.js gzip	3.83 kB	3.83 kB	✓
amp-HASH.js gzip	531 B	531 B	✓
css-HASH.js gzip	333 B	333 B	✓
hooks-HASH.js gzip	910 B	910 B	✓
index-HASH.js gzip	230 B	230 B	✓
link-HASH.js gzip	1.64 kB	1.64 kB	✓
routerDirect..HASH.js gzip	297 B	297 B	✓
withRouter-HASH.js gzip	293 B	293 B	✓
e025d2764813..52f.css gzip	125 B	125 B	✓
Overall change	8.98 kB	8.98 kB	✓

Client Build Manifests

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
_buildManifest.js gzip	418 B	418 B	✓
Overall change	418 B	418 B	✓

Rendered Page Sizes

	vercel/next.js canary	azukaru/next.js x-add-response-cache	Change
index.html gzip	575 B	575 B	✓
link.html gzip	586 B	586 B	✓
withRouter.html gzip	568 B	568 B	✓
Overall change	1.73 kB	1.73 kB	✓

Commit: c7c6f45