Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor(analysis): rust based page-static-info, deprecate js parse interface in next-swc #59300

Merged
merged 9 commits into from
Jan 22, 2024
Merged

refactor(analysis): rust based page-static-info, deprecate js parse interface in next-swc #59300

merged 9 commits into from
Jan 22, 2024

Conversation

kwonoj
Copy link
Contributor

@kwonoj kwonoj commented Dec 5, 2023 •
edited

Description

This PR refactors existing analysis/get-static-page-info, moves over most of parse / ast visiting logic into next-swc's rust codebase. By having this, turbopack can reuse same logic to extract info for the analysis. Also as a side effect, this removes JS side parse which is known to be inefficient due to serialization / deserialization.

The entrypoint getPageStaticInfo is still in the existing get-page-static-info, only for extracting / visiting logic is moved. There are some JS specific context to postprocess extracted information which would require additional effort to move into.

Closes PACK-2088

@ijjk ijjk added Turbopack Related to Turbopack with Next.js. created-by: Turbopack team PRs by the turbopack team type: next labels Dec 5, 2023
@kwonoj kwonoj changed the title refactor(analysis): port check_exports to napi [DONOTMERGE] refactor(analysis): port check_exports to napi Dec 5, 2023
@ijjk
Copy link
Member

ijjk commented Dec 5, 2023 •
edited

Stats from current PR

Default Build (Increase detected ⚠️)
General
vercel/next.js canary vercel/next.js refactor-parse-module Change
buildDuration 9.3s 11.6s ⚠️ +2.3s
buildDurationCached 5.7s 4.9s N/A
nodeModulesSize 200 MB 200 MB N/A
nextStartRea..uration (ms) 426ms 427ms N/A
Client Bundles (main, webpack)
vercel/next.js canary vercel/next.js refactor-parse-module Change
193.HASH.js gzip 181 B 182 B N/A
26-HASH.js gzip 28.9 kB 29 kB N/A
3f784ff6-HASH.js gzip 53.4 kB 53.4 kB ✓
framework-HASH.js gzip 45.2 kB 45.2 kB ✓
main-app-HASH.js gzip 216 B 244 B N/A
main-HASH.js gzip 31.8 kB 31.8 kB N/A
webpack-HASH.js gzip 1.7 kB 1.7 kB N/A
Overall change 98.6 kB 98.6 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js refactor-parse-module Change
polyfills-HASH.js gzip 31 kB 31 kB ✓
Overall change 31 kB 31 kB ✓
Client Pages
vercel/next.js canary vercel/next.js refactor-parse-module Change
_app-HASH.js gzip 194 B 195 B N/A
_error-HASH.js gzip 183 B 181 B N/A
amp-HASH.js gzip 504 B 502 B N/A
css-HASH.js gzip 321 B 321 B ✓
dynamic-HASH.js gzip 2.5 kB 2.5 kB N/A
edge-ssr-HASH.js gzip 255 B 253 B N/A
head-HASH.js gzip 350 B 349 B N/A
hooks-HASH.js gzip 369 B 369 B ✓
image-HASH.js gzip 4.18 kB 4.18 kB N/A
index-HASH.js gzip 255 B 256 B N/A
link-HASH.js gzip 2.61 kB 2.61 kB ✓
routerDirect..HASH.js gzip 312 B 311 B N/A
script-HASH.js gzip 385 B 383 B N/A
withRouter-HASH.js gzip 307 B 308 B N/A
1afbb74e6ecf..834.css gzip 106 B 106 B ✓
Overall change 3.4 kB 3.4 kB ✓
Client Build Manifests
vercel/next.js canary vercel/next.js refactor-parse-module Change
_buildManifest.js gzip 484 B 485 B N/A
Overall change 0 B 0 B ✓
Rendered Page Sizes
vercel/next.js canary vercel/next.js refactor-parse-module Change
index.html gzip 529 B 528 B N/A
link.html gzip 541 B 540 B N/A
withRouter.html gzip 524 B 523 B N/A
Overall change 0 B 0 B ✓
Edge SSR bundle Size Overall increase ⚠️
vercel/next.js canary vercel/next.js refactor-parse-module Change
page.js gzip 917 B 149 kB ⚠️ +148 kB
edge-ssr.js gzip N/A 94 kB N/A
Overall change 917 B 149 kB ⚠️ +148 kB
Middleware size Overall increase ⚠️
vercel/next.js canary vercel/next.js refactor-parse-module Change
middleware-b..fest.js gzip 624 B 625 B N/A
middleware-r..fest.js gzip 151 B 151 B ✓
middleware.js gzip 37.4 kB 37.5 kB ⚠️ +162 B
edge-runtime..pack.js gzip 1.72 kB 1.92 kB ⚠️ +208 B
Overall change 39.2 kB 39.6 kB ⚠️ +370 B
Next Runtimes
vercel/next.js canary vercel/next.js refactor-parse-module Change
app-page-exp...dev.js gzip 169 kB 169 kB ✓
app-page-exp..prod.js gzip 95.6 kB 95.6 kB ✓
app-page-tur..prod.js gzip 96.3 kB 96.3 kB ✓
app-page-tur..prod.js gzip 90.9 kB 90.9 kB ✓
app-page.run...dev.js gzip 142 kB 142 kB ✓
app-page.run..prod.js gzip 90.2 kB 90.2 kB ✓
app-route-ex...dev.js gzip 24.2 kB 24.2 kB ✓
app-route-ex..prod.js gzip 16.9 kB 16.9 kB ✓
app-route-tu..prod.js gzip 16.9 kB 16.9 kB ✓
app-route-tu..prod.js gzip 16.4 kB 16.4 kB ✓
app-route.ru...dev.js gzip 23.6 kB 23.6 kB ✓
app-route.ru..prod.js gzip 16.4 kB 16.4 kB ✓
pages-api-tu..prod.js gzip 9.41 kB 9.41 kB ✓
pages-api.ru...dev.js gzip 9.68 kB 9.68 kB ✓
pages-api.ru..prod.js gzip 9.4 kB 9.4 kB ✓
pages-turbo...prod.js gzip 22 kB 22 kB ✓
pages.runtim...dev.js gzip 22.6 kB 22.6 kB ✓
pages.runtim..prod.js gzip 22 kB 22 kB ✓
server.runti..prod.js gzip 49.7 kB 49.7 kB ✓
Overall change 944 kB 944 kB ✓
Diff details
Diff for page.js 
failed to diff
Diff for edge-runtime-webpack.js 
@@ -96,6 +96,21 @@
     /******/
   })();
   /******/
+  /******/ /* webpack/runtime/compat get default export */
+  /******/ (() => {
+    /******/ // getDefaultExport function for compatibility with non-harmony modules
+    /******/ __webpack_require__.n = (module) => {
+      /******/ var getter =
+        module && module.__esModule
+          ? /******/ () => module["default"]
+          : /******/ () => module;
+      /******/ __webpack_require__.d(getter, { a: getter });
+      /******/ return getter;
+      /******/
+    };
+    /******/
+  })();
+  /******/
   /******/ /* webpack/runtime/define property getters */
   /******/ (() => {
     /******/ // define getter functions for harmony exports
@@ -118,6 +133,15 @@
     /******/
   })();
   /******/
+  /******/ /* webpack/runtime/ensure chunk */
+  /******/ (() => {
+    /******/ // The chunk loading function for additional chunks
+    /******/ // Since all referenced chunks are already included
+    /******/ // in this file, this function is empty here.
+    /******/ __webpack_require__.e = () => Promise.resolve();
+    /******/
+  })();
+  /******/
   /******/ /* webpack/runtime/global */
   /******/ (() => {
     /******/ __webpack_require__.g = (function () {
Diff for middleware.js

Diff too large to display

Diff for edge-ssr.js

Diff too large to display

Diff for 26-HASH.js

Diff too large to display

Diff for main-app-HASH.js 
@@ -1,28 +1,40 @@
 (self["webpackChunk_N_E"] = self["webpackChunk_N_E"] || []).push([
   [744],
   {
-    /***/ 7224: /***/ function (
+    /***/ 1062: /***/ function (
       __unused_webpack_module,
       __unused_webpack_exports,
       __webpack_require__
     ) {
       Promise.resolve(/* import() eager */).then(
-        __webpack_require__.t.bind(__webpack_require__, 8438, 23)
+        __webpack_require__.t.bind(__webpack_require__, 7480, 23)
       );
       Promise.resolve(/* import() eager */).then(
-        __webpack_require__.t.bind(__webpack_require__, 6367, 23)
+        __webpack_require__.t.bind(__webpack_require__, 7394, 23)
       );
       Promise.resolve(/* import() eager */).then(
-        __webpack_require__.t.bind(__webpack_require__, 2267, 23)
+        __webpack_require__.t.bind(__webpack_require__, 6406, 23)
       );
       Promise.resolve(/* import() eager */).then(
-        __webpack_require__.t.bind(__webpack_require__, 7202, 23)
+        __webpack_require__.t.bind(__webpack_require__, 5828, 23)
       );
       Promise.resolve(/* import() eager */).then(
-        __webpack_require__.t.bind(__webpack_require__, 6516, 23)
+        __webpack_require__.t.bind(__webpack_require__, 7493, 23)
       );
       Promise.resolve(/* import() eager */).then(
-        __webpack_require__.t.bind(__webpack_require__, 9061, 23)
+        __webpack_require__.t.bind(__webpack_require__, 8398, 23)
+      );
+      Promise.resolve(/* import() eager */).then(
+        __webpack_require__.t.bind(__webpack_require__, 4320, 23)
+      );
+      Promise.resolve(/* import() eager */).then(
+        __webpack_require__.t.bind(__webpack_require__, 3311, 23)
+      );
+      Promise.resolve(/* import() eager */).then(
+        __webpack_require__.bind(__webpack_require__, 5908)
+      );
+      Promise.resolve(/* import() eager */).then(
+        __webpack_require__.t.bind(__webpack_require__, 7916, 23)
       );
 
       /***/
@@ -33,8 +45,8 @@
     /******/ var __webpack_exec__ = function (moduleId) {
       return __webpack_require__((__webpack_require__.s = moduleId));
     };
-    /******/ __webpack_require__.O(0, [904, 26], function () {
-      return __webpack_exec__(534), __webpack_exec__(7224);
+    /******/ __webpack_require__.O(0, [658, 724], function () {
+      return __webpack_exec__(1836), __webpack_exec__(1062);
     });
     /******/ var __webpack_exports__ = __webpack_require__.O();
     /******/ _N_E = __webpack_exports__;
Commit: 272ca3f

@ijjk
Copy link
Member

ijjk commented Dec 5, 2023 •
edited

Tests Passed

@kwonoj kwonoj force-pushed the refactor-parse-module branch 23 times, most recently from 497a9e0 to dfb8913 Compare December 8, 2023 00:20
@kwonoj kwonoj force-pushed the refactor-parse-module branch 5 times, most recently from 43ad2b0 to 030459b Compare January 11, 2024 23:52
@kwonoj kwonoj force-pushed the refactor-parse-module branch 2 times, most recently from f8e7a54 to 776135c Compare January 16, 2024 19:03
timneutkens
timneutkens reviewed Jan 16, 2024
View reviewed changes
packages/next/src/build/swc/index.ts
return astStr
analysis: {
isDynamicMetadataRoute: async (pageFilePath: string) => {
const fileContent = (await tryToReadFile(pageFilePath, true)) || ''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For swc-loader transforms we have the SWC-side read the file, seems we could do the same here in a follow-up PR.

@kwonoj kwonoj merged commit 9d5f62e into canary Jan 22, 2024
62 of 67 checks passed
@kwonoj kwonoj deleted the refactor-parse-module branch January 22, 2024 22:42
@sokra sokra mentioned this pull request Jan 23, 2024
Merged
sokra added a commit that referenced this pull request Jan 23, 2024
@sokra
Revert "refactor(analysis): rust based page-static-info, deprecate js…
66bb526 
… parse interface in next-swc (#59300)"

This reverts commit 9d5f62e.
sokra added a commit that referenced this pull request Jan 23, 2024
@sokra
Revert "refactor(analysis): rust based page-static-info, deprecate js…
3a7fea4 
… parse interface in next-swc" (#61021)

Reverts #59300

Breaks `app/page.mdx` files

Closes PACK-2279
@github-actions github-actions bot added the locked label Feb 6, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 6, 2024
@kwonoj kwonoj restored the refactor-parse-module branch February 8, 2024 18:09
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@timneutkens timneutkens timneutkens approved these changes

@ijjk ijjk Awaiting requested review from ijjk ijjk is a code owner

@shuding shuding Awaiting requested review from shuding shuding is a code owner

@huozhi huozhi Awaiting requested review from huozhi huozhi is a code owner

@ztanner ztanner Awaiting requested review from ztanner ztanner is a code owner

@feedthejim feedthejim Awaiting requested review from feedthejim feedthejim is a code owner

@wyattjoh wyattjoh Awaiting requested review from wyattjoh wyattjoh is a code owner

Assignees
No one assigned
Labels
created-by: Turbopack team PRs by the turbopack team locked Turbopack Related to Turbopack with Next.js. type: next
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@kwonoj @ijjk @timneutkens