Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the incremental cache storage and cache providers extensible. #22619

Closed

Conversation

MartinLG
Copy link

This will especially allow developers to store pages in object storages like S3 and shared cache like Redis. Thanks to that, all Next instances can share the same cache pool, therefore it increase the overall performances depending on the company infrastructure.

This commit basically changes nothing to the Next.js current features and behaviors. It just allow us to to use plugins to customize that behavior.

@ijjk
Copy link
Member

ijjk commented Feb 28, 2021

Stats from current PR

Default Server Mode (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
buildDuration 12.8s 12.9s ⚠️ +124ms
nodeModulesSize 42.7 MB 42.7 MB ⚠️ +3.76 kB
Page Load Tests Overall increase ✓
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
/ failed reqs 0 0
/ total time (seconds) 2.256 2.253 0
/ avg req/sec 1107.98 1109.82 +1.84
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.515 1.455 -0.06
/error-in-render avg req/sec 1650.24 1718.61 +68.37
Client Bundles (main, webpack, commons)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
677f882d2ed8..e7a0.js gzip 13.4 kB 13.4 kB
framework.HASH.js gzip 39 kB 39 kB
main-HASH.js gzip 6.65 kB 6.65 kB
webpack-HASH.js gzip 751 B 751 B
Overall change 59.7 kB 59.7 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB
Overall change 31.3 kB 31.3 kB
Client Pages
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_app-2a09aa2..4a98.js gzip 1.28 kB 1.28 kB
_error-8b758..aef6.js gzip 3.46 kB 3.46 kB
amp-e3e9bc99..932c.js gzip 536 B 536 B
hooks-5023ef..3670.js gzip 888 B 888 B
index-3a2b03..c677.js gzip 227 B 227 B
link-177412b..043e.js gzip 1.67 kB 1.67 kB
routerDirect..1a66.js gzip 303 B 303 B
withRouter-9..1b7d.js gzip 302 B 302 B
Overall change 8.66 kB 8.66 kB
Client Build Manifests
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_buildManifest.js gzip 347 B 347 B
Overall change 347 B 347 B
Rendered Page Sizes
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
index.html gzip 613 B 613 B
link.html gzip 620 B 620 B
withRouter.html gzip 608 B 608 B
Overall change 1.84 kB 1.84 kB

Serverless Mode
General Overall increase ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
buildDuration 14.3s 15.3s ⚠️ +1s
nodeModulesSize 42.7 MB 42.7 MB ⚠️ +3.76 kB
Client Bundles (main, webpack, commons)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
677f882d2ed8..e7a0.js gzip 13.4 kB 13.4 kB
framework.HASH.js gzip 39 kB 39 kB
main-HASH.js gzip 6.65 kB 6.65 kB
webpack-HASH.js gzip 751 B 751 B
Overall change 59.7 kB 59.7 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB
Overall change 31.3 kB 31.3 kB
Client Pages
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_app-2a09aa2..4a98.js gzip 1.28 kB 1.28 kB
_error-8b758..aef6.js gzip 3.46 kB 3.46 kB
amp-e3e9bc99..932c.js gzip 536 B 536 B
hooks-5023ef..3670.js gzip 888 B 888 B
index-3a2b03..c677.js gzip 227 B 227 B
link-177412b..043e.js gzip 1.67 kB 1.67 kB
routerDirect..1a66.js gzip 303 B 303 B
withRouter-9..1b7d.js gzip 302 B 302 B
Overall change 8.66 kB 8.66 kB
Client Build Manifests
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_buildManifest.js gzip 347 B 347 B
Overall change 347 B 347 B
Serverless bundles
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_error.js 1.01 MB 1.01 MB
404.html 2.67 kB 2.67 kB
500.html 2.65 kB 2.65 kB
amp.amp.html 10.4 kB 10.4 kB
amp.html 1.86 kB 1.86 kB
hooks.html 1.92 kB 1.92 kB
index.js 1.01 MB 1.01 MB
link.js 1.06 MB 1.06 MB
routerDirect.js 1.06 MB 1.06 MB
withRouter.js 1.06 MB 1.06 MB
Overall change 5.21 MB 5.21 MB

Webpack 5 Mode (Decrease detected ✓)
General Overall increase ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
buildDuration 15.3s 15.1s -192ms
nodeModulesSize 42.7 MB 42.7 MB ⚠️ +3.76 kB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
/ failed reqs 0 0
/ total time (seconds) 2.209 2.181 -0.03
/ avg req/sec 1131.9 1146.45 +14.55
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.486 1.513 ⚠️ +0.03
/error-in-render avg req/sec 1682.53 1652.18 ⚠️ -30.35
Client Bundles (main, webpack, commons)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
597-74632e79..1d47.js gzip 13.3 kB 13.3 kB
framework.HASH.js gzip 39.3 kB 39.3 kB
main-HASH.js gzip 6.59 kB 6.59 kB
webpack-HASH.js gzip 954 B 954 B
Overall change 60.1 kB 60.1 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB
Overall change 31.1 kB 31.1 kB
Client Pages
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_app-aedc815..1421.js gzip 1.26 kB 1.26 kB
_error-71ec2..1a96.js gzip 3.38 kB 3.38 kB
amp-33a09cb0..6745.js gzip 536 B 536 B
hooks-4e968a..f870.js gzip 902 B 902 B
index-5c6845..f75c.js gzip 230 B 230 B
link-99f0c6c..b84a.js gzip 1.65 kB 1.65 kB
routerDirect..bb56.js gzip 306 B 306 B
withRouter-7..2133.js gzip 302 B 302 B
Overall change 8.57 kB 8.57 kB
Client Build Manifests
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_buildManifest.js gzip 326 B 326 B
Overall change 326 B 326 B
Rendered Page Sizes
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
index.html gzip 586 B 586 B
link.html gzip 593 B 593 B
withRouter.html gzip 581 B 581 B
Overall change 1.76 kB 1.76 kB

Diffs

Diff for index.html
@@ -43,7 +43,7 @@
         "props": { "pageProps": {} },
         "page": "/",
         "query": {},
-        "buildId": "yWld_S5aVx4cvj8saAAm5",
+        "buildId": "tfdAi7ugfZucQpST5FhDR",
         "isFallback": false,
         "gip": true
       }
@@ -77,11 +77,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/yWld_S5aVx4cvj8saAAm5/_buildManifest.js"
+      src="/_next/static/tfdAi7ugfZucQpST5FhDR/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/yWld_S5aVx4cvj8saAAm5/_ssgManifest.js"
+      src="/_next/static/tfdAi7ugfZucQpST5FhDR/_ssgManifest.js"
       async=""
     ></script>
   </body>
Diff for link.html
@@ -48,7 +48,7 @@
         "props": { "pageProps": {} },
         "page": "/link",
         "query": {},
-        "buildId": "yWld_S5aVx4cvj8saAAm5",
+        "buildId": "tfdAi7ugfZucQpST5FhDR",
         "isFallback": false,
         "gip": true
       }
@@ -82,11 +82,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/yWld_S5aVx4cvj8saAAm5/_buildManifest.js"
+      src="/_next/static/tfdAi7ugfZucQpST5FhDR/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/yWld_S5aVx4cvj8saAAm5/_ssgManifest.js"
+      src="/_next/static/tfdAi7ugfZucQpST5FhDR/_ssgManifest.js"
       async=""
     ></script>
   </body>
Diff for withRouter.html
@@ -43,7 +43,7 @@
         "props": { "pageProps": {} },
         "page": "/withRouter",
         "query": {},
-        "buildId": "yWld_S5aVx4cvj8saAAm5",
+        "buildId": "tfdAi7ugfZucQpST5FhDR",
         "isFallback": false,
         "gip": true
       }
@@ -77,11 +77,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/yWld_S5aVx4cvj8saAAm5/_buildManifest.js"
+      src="/_next/static/tfdAi7ugfZucQpST5FhDR/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/yWld_S5aVx4cvj8saAAm5/_ssgManifest.js"
+      src="/_next/static/tfdAi7ugfZucQpST5FhDR/_ssgManifest.js"
       async=""
     ></script>
   </body>
Commit: d4f53eb

@ijjk
Copy link
Member

ijjk commented Mar 1, 2021

Stats from current PR

Default Server Mode
General Overall increase ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
buildDuration 14s 13.4s -570ms
nodeModulesSize 42.7 MB 42.7 MB ⚠️ +3.76 kB
Page Load Tests Overall increase ✓
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
/ failed reqs 0 0
/ total time (seconds) 2.502 2.457 -0.04
/ avg req/sec 999.27 1017.55 +18.28
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.715 1.734 ⚠️ +0.02
/error-in-render avg req/sec 1457.54 1441.72 ⚠️ -15.82
Client Bundles (main, webpack, commons)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
677f882d2ed8..e7a0.js gzip 13.4 kB 13.4 kB
framework.HASH.js gzip 39 kB 39 kB
main-HASH.js gzip 6.65 kB 6.65 kB
webpack-HASH.js gzip 751 B 751 B
Overall change 59.7 kB 59.7 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB
Overall change 31.3 kB 31.3 kB
Client Pages
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_app-2a09aa2..4a98.js gzip 1.28 kB 1.28 kB
_error-8b758..aef6.js gzip 3.46 kB 3.46 kB
amp-e3e9bc99..932c.js gzip 536 B 536 B
hooks-5023ef..3670.js gzip 888 B 888 B
index-3a2b03..c677.js gzip 227 B 227 B
link-177412b..043e.js gzip 1.67 kB 1.67 kB
routerDirect..1a66.js gzip 303 B 303 B
withRouter-9..1b7d.js gzip 302 B 302 B
Overall change 8.66 kB 8.66 kB
Client Build Manifests
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_buildManifest.js gzip 347 B 347 B
Overall change 347 B 347 B
Rendered Page Sizes
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
index.html gzip 613 B 613 B
link.html gzip 620 B 620 B
withRouter.html gzip 607 B 607 B
Overall change 1.84 kB 1.84 kB

Serverless Mode
General Overall increase ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
buildDuration 15.9s 15.8s -116ms
nodeModulesSize 42.7 MB 42.7 MB ⚠️ +3.76 kB
Client Bundles (main, webpack, commons)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
677f882d2ed8..e7a0.js gzip 13.4 kB 13.4 kB
framework.HASH.js gzip 39 kB 39 kB
main-HASH.js gzip 6.65 kB 6.65 kB
webpack-HASH.js gzip 751 B 751 B
Overall change 59.7 kB 59.7 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB
Overall change 31.3 kB 31.3 kB
Client Pages
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_app-2a09aa2..4a98.js gzip 1.28 kB 1.28 kB
_error-8b758..aef6.js gzip 3.46 kB 3.46 kB
amp-e3e9bc99..932c.js gzip 536 B 536 B
hooks-5023ef..3670.js gzip 888 B 888 B
index-3a2b03..c677.js gzip 227 B 227 B
link-177412b..043e.js gzip 1.67 kB 1.67 kB
routerDirect..1a66.js gzip 303 B 303 B
withRouter-9..1b7d.js gzip 302 B 302 B
Overall change 8.66 kB 8.66 kB
Client Build Manifests
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_buildManifest.js gzip 347 B 347 B
Overall change 347 B 347 B
Serverless bundles
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_error.js 1.01 MB 1.01 MB
404.html 2.67 kB 2.67 kB
500.html 2.65 kB 2.65 kB
amp.amp.html 10.4 kB 10.4 kB
amp.html 1.86 kB 1.86 kB
hooks.html 1.92 kB 1.92 kB
index.js 1.01 MB 1.01 MB
link.js 1.06 MB 1.06 MB
routerDirect.js 1.06 MB 1.06 MB
withRouter.js 1.06 MB 1.06 MB
Overall change 5.21 MB 5.21 MB

Webpack 5 Mode (Decrease detected ✓)
General Overall increase ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
buildDuration 15.6s 15.9s ⚠️ +286ms
nodeModulesSize 42.7 MB 42.7 MB ⚠️ +3.76 kB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
/ failed reqs 0 0
/ total time (seconds) 2.416 2.504 ⚠️ +0.09
/ avg req/sec 1034.89 998.49 ⚠️ -36.4
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.633 1.749 ⚠️ +0.12
/error-in-render avg req/sec 1530.68 1429.56 ⚠️ -101.12
Client Bundles (main, webpack, commons)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
597-74632e79..1d47.js gzip 13.3 kB 13.3 kB
framework.HASH.js gzip 39.3 kB 39.3 kB
main-HASH.js gzip 6.6 kB 6.6 kB
webpack-HASH.js gzip 954 B 954 B
Overall change 60.1 kB 60.1 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB
Overall change 31.1 kB 31.1 kB
Client Pages
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_app-aedc815..1421.js gzip 1.26 kB 1.26 kB
_error-71ec2..1a96.js gzip 3.38 kB 3.38 kB
amp-33a09cb0..6745.js gzip 536 B 536 B
hooks-4e968a..f870.js gzip 902 B 902 B
index-5c6845..f75c.js gzip 230 B 230 B
link-99f0c6c..b84a.js gzip 1.65 kB 1.65 kB
routerDirect..bb56.js gzip 306 B 306 B
withRouter-7..2133.js gzip 302 B 302 B
Overall change 8.57 kB 8.57 kB
Client Build Manifests
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_buildManifest.js gzip 326 B 326 B
Overall change 326 B 326 B
Rendered Page Sizes
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
index.html gzip 586 B 586 B
link.html gzip 594 B 594 B
withRouter.html gzip 581 B 581 B
Overall change 1.76 kB 1.76 kB

Diffs

Diff for index.html
@@ -43,7 +43,7 @@
         "props": { "pageProps": {} },
         "page": "/",
         "query": {},
-        "buildId": "HvAF7BNP9A-LFw0bAnJih",
+        "buildId": "nJvzziF4grWdkl0MNOk6K",
         "isFallback": false,
         "gip": true
       }
@@ -77,11 +77,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/HvAF7BNP9A-LFw0bAnJih/_buildManifest.js"
+      src="/_next/static/nJvzziF4grWdkl0MNOk6K/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/HvAF7BNP9A-LFw0bAnJih/_ssgManifest.js"
+      src="/_next/static/nJvzziF4grWdkl0MNOk6K/_ssgManifest.js"
       async=""
     ></script>
   </body>
Diff for link.html
@@ -48,7 +48,7 @@
         "props": { "pageProps": {} },
         "page": "/link",
         "query": {},
-        "buildId": "HvAF7BNP9A-LFw0bAnJih",
+        "buildId": "nJvzziF4grWdkl0MNOk6K",
         "isFallback": false,
         "gip": true
       }
@@ -82,11 +82,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/HvAF7BNP9A-LFw0bAnJih/_buildManifest.js"
+      src="/_next/static/nJvzziF4grWdkl0MNOk6K/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/HvAF7BNP9A-LFw0bAnJih/_ssgManifest.js"
+      src="/_next/static/nJvzziF4grWdkl0MNOk6K/_ssgManifest.js"
       async=""
     ></script>
   </body>
Diff for withRouter.html
@@ -43,7 +43,7 @@
         "props": { "pageProps": {} },
         "page": "/withRouter",
         "query": {},
-        "buildId": "HvAF7BNP9A-LFw0bAnJih",
+        "buildId": "nJvzziF4grWdkl0MNOk6K",
         "isFallback": false,
         "gip": true
       }
@@ -77,11 +77,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/HvAF7BNP9A-LFw0bAnJih/_buildManifest.js"
+      src="/_next/static/nJvzziF4grWdkl0MNOk6K/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/HvAF7BNP9A-LFw0bAnJih/_ssgManifest.js"
+      src="/_next/static/nJvzziF4grWdkl0MNOk6K/_ssgManifest.js"
       async=""
     ></script>
   </body>
Commit: c574f4a

@ijjk ijjk added the type: next label Mar 3, 2021
@MartinLG-LaFourche
Copy link

@timneutkens is there any special process to get this PR reviewed ?

Would you like me to provide a real life example of its usefulness ?

I noticed some Azure Pipeline tests failed, but they seem to be failing in many other branches, and they don't seem to be related to the code changes in this PR, should I do something about it ?

Thanks

This will especially allow developers to store pages in object storages like S3 and shared cache like Redis. Thanks to that, all Next instances can share the same cache pool, therefore it increase the overall performances depending on the company infrastructure.

This commit basically changes nothing to the Next.js current features and behaviors. It just allow us to to use plugins to customize that behavior.
@MartinLG MartinLG force-pushed the feature/extends-storage-and-cache branch from c574f4a to a49d0f5 Compare April 19, 2021 16:02
@MartinLG MartinLG requested a review from shuding as a code owner April 19, 2021 16:02
@ijjk
Copy link
Member

ijjk commented Apr 19, 2021

Stats from current PR

Default Server Mode (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
buildDuration 13.6s 14s ⚠️ +357ms
buildDurationCached 4.4s 4.2s -147ms
nodeModulesSize 46.3 MB 46.3 MB ⚠️ +3.76 kB
Page Load Tests Overall increase ✓
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
/ failed reqs 0 0
/ total time (seconds) 2.428 2.409 -0.02
/ avg req/sec 1029.49 1037.7 +8.21
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.233 1.219 -0.01
/error-in-render avg req/sec 2028.33 2051.67 +23.34
Client Bundles (main, webpack, commons)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
597-HASH.js gzip 13.3 kB 13.3 kB
778-HASH.js gzip 7.08 kB 7.08 kB
framework-HASH.js gzip 39.3 kB 39.3 kB
main-HASH.js gzip 151 B 151 B
webpack-HASH.js gzip 993 B 993 B
Overall change 60.8 kB 60.8 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB
Overall change 31.1 kB 31.1 kB
Client Pages
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_app-HASH.js gzip 1.3 kB 1.3 kB
_error-HASH.js gzip 3.68 kB 3.68 kB
amp-HASH.js gzip 558 B 558 B
hooks-HASH.js gzip 924 B 924 B
index-HASH.js gzip 243 B 243 B
link-HASH.js gzip 1.66 kB 1.66 kB
routerDirect..HASH.js gzip 336 B 336 B
withRouter-HASH.js gzip 334 B 334 B
Overall change 9.03 kB 9.03 kB
Client Build Manifests
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_buildManifest.js gzip 349 B 349 B
Overall change 349 B 349 B
Rendered Page Sizes
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
index.html gzip 609 B 609 B
link.html gzip 614 B 614 B
withRouter.html gzip 604 B 604 B
Overall change 1.83 kB 1.83 kB

Serverless Mode
General Overall increase ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
buildDuration 17s 17.1s ⚠️ +53ms
buildDurationCached 6.2s 6.2s ⚠️ +22ms
nodeModulesSize 46.3 MB 46.3 MB ⚠️ +3.76 kB
Client Bundles (main, webpack, commons)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
597-HASH.js gzip 13.3 kB 13.3 kB
778-HASH.js gzip 7.08 kB 7.08 kB
framework-HASH.js gzip 39.3 kB 39.3 kB
main-HASH.js gzip 151 B 151 B
webpack-HASH.js gzip 993 B 993 B
Overall change 60.8 kB 60.8 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB
Overall change 31.1 kB 31.1 kB
Client Pages
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_app-HASH.js gzip 1.3 kB 1.3 kB
_error-HASH.js gzip 3.68 kB 3.68 kB
amp-HASH.js gzip 558 B 558 B
hooks-HASH.js gzip 924 B 924 B
index-HASH.js gzip 243 B 243 B
link-HASH.js gzip 1.66 kB 1.66 kB
routerDirect..HASH.js gzip 336 B 336 B
withRouter-HASH.js gzip 334 B 334 B
Overall change 9.03 kB 9.03 kB
Client Build Manifests
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_buildManifest.js gzip 349 B 349 B
Overall change 349 B 349 B
Serverless bundles
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_error.js 1.39 MB 1.39 MB
404.html 2.76 kB 2.76 kB
500.html 2.75 kB 2.75 kB
amp.amp.html 10.7 kB 10.7 kB
amp.html 1.96 kB 1.96 kB
hooks.html 2.01 kB 2.01 kB
index.js 1.39 MB 1.39 MB
link.js 1.45 MB 1.45 MB
routerDirect.js 1.44 MB 1.44 MB
withRouter.js 1.44 MB 1.44 MB
Overall change 7.15 MB 7.15 MB

Webpack 4 Mode (Decrease detected ✓)
General Overall increase ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
buildDuration 11.9s 11.7s -205ms
buildDurationCached 4.7s 4.8s ⚠️ +17ms
nodeModulesSize 46.3 MB 46.3 MB ⚠️ +3.76 kB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
/ failed reqs 0 0
/ total time (seconds) 2.391 2.407 ⚠️ +0.02
/ avg req/sec 1045.49 1038.77 ⚠️ -6.72
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.257 1.272 ⚠️ +0.02
/error-in-render avg req/sec 1988.62 1965.52 ⚠️ -23.1
Client Bundles (main, webpack, commons)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
677f882d2ed8..HASH.js gzip 13.4 kB 13.4 kB
framework.HASH.js gzip 39 kB 39 kB
main-HASH.js gzip 7.16 kB 7.16 kB
webpack-HASH.js gzip 751 B 751 B
Overall change 60.2 kB 60.2 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB
Overall change 31.3 kB 31.3 kB
Client Pages
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_app-HASH.js gzip 1.28 kB 1.28 kB
_error-HASH.js gzip 3.71 kB 3.71 kB
amp-HASH.js gzip 536 B 536 B
hooks-HASH.js gzip 888 B 888 B
index-HASH.js gzip 227 B 227 B
link-HASH.js gzip 1.64 kB 1.64 kB
routerDirect..HASH.js gzip 303 B 303 B
withRouter-HASH.js gzip 302 B 302 B
Overall change 8.89 kB 8.89 kB
Client Build Manifests
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
_buildManifest.js gzip 371 B 371 B
Overall change 371 B 371 B
Rendered Page Sizes
vercel/next.js canary MartinLG/next.js feature/extends-storage-and-cache Change
index.html gzip 613 B 613 B
link.html gzip 619 B 619 B
withRouter.html gzip 607 B 607 B
Overall change 1.84 kB 1.84 kB
Commit: a49d0f5

}

async write(key: string, value: string | undefined) {
await promises.mkdir(key.substring(0, key.lastIndexOf('/')), {
Copy link

@reboxer reboxer Jul 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing this pull request and apparently this makes the directory to be an empty string and causing write() to fail each time it tries to save the files to disk.

Failed to update prerender files for /test [Error: ENOENT: no such file or directory, mkdir ''] {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'mkdir',
  path: ''
}

The file should require path and the line should be path.dirname(key) to achieve the same result as the current implementation.

Also, mkdir could be put in a different method and called before trying to write the HTML and JSON, as it will be called twice for each page.

@reboxer
Copy link

reboxer commented Jul 23, 2021

Hey @timneutkens, it's been quite some time since this pull request was opened and recently there have been a few other pull requests merged related to ISR, and i think this would be a great addition to next. I took some time to make a simple plugin (https://github.com/reboxer/next-storage-provider) using these additions to be able to use redis as an lru cache and store the rendered pages in S3, you can check it out in case you need an example.

@@ -31,6 +35,7 @@ export class IncrementalCache {
prerenderManifest: PrerenderManifest
cache: LRUCache<string, IncrementalCacheValue>
locales?: string[]
storageProvider: StorageProfiderInterface
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo

}: {
dev: boolean
max?: number
distDir: string
pagesDir: string
flushToDisk?: boolean
locales?: string[]
storageProvider?: StorageProfiderInterface
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of passing this in here, could it be an implementation detail of the lruCacheProvider?

In other words, could the default lruCacheProvider be built of both an in-memory LRUCache and a FilesystemCache fallback?

That would require moving some logic out of this file into something like a memory-cache-with-disk-fallback.ts. The benefit is that it would simplify this class's interface a bit.

@kodiakhq kodiakhq bot closed this in #37258 May 28, 2022
kodiakhq bot pushed a commit that referenced this pull request May 28, 2022
This refactors the `incremental-cache` and moves the file-system cache handling to it's own cache handler allowing it to be replaced by a custom cache handler (experimental). 

Closes: #22619

## Bug

- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`

## Documentation / Examples

- [ ] Make sure the linting passes by running `yarn lint`
- [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants