Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial next swc package with first transform #27355

Merged
merged 31 commits into from
Jul 27, 2021

Conversation

padmaia
Copy link
Member

@padmaia padmaia commented Jul 20, 2021

This PR sets up a napi-rs package under packages/next/build/swc that uses swc's compiler with custom transforms written in rust. The transforms will be ported over from the current babel plugins, but the only transform currently implemented is the hook destructuring optimization.

There is also a new webpack loader next-swc-loader that will eventually replace babel-loader. First it will be enabled behind an experimental flag, but that will be done in a follow up.

I added a new workflow that runs on pull requests that touch the packages/next/build/swc folder and can also run manually. When it runs manually it builds the binaries for different platforms and commits them to packages/next/native. When it runs on pull requests, instead of committing it checks to see that the built binaries have already been committed to make sure they are up to date before merging. This means that only people working on the Rust code will have to deal with building it.

@ijjk ijjk added created-by: Next.js team PRs by the Next.js team. type: next labels Jul 20, 2021
@ijjk

This comment has been minimized.

@ijjk

This comment has been minimized.

@padmaia
Copy link
Member Author

padmaia commented Jul 20, 2021

There's quite a lot of copied code from swc and the swc loader. What are the necessary steps to make sure we comply with the open source licenses?

@ijjk

This comment has been minimized.

@padmaia padmaia requested review from styfle, ijjk and kdy1 July 20, 2021 19:34
@ijjk

This comment has been minimized.

@ijjk

This comment has been minimized.

Copy link
Member

@kdy1 kdy1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

serde_json = "1"
swc = "0.28"
swc_atoms = "0.2.6"
swc_common = { version = "0.10.23", features = ["tty-emitter", "sourcemap"] }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was a breaking change of swc_common recently, although library users would not be affected because implementations in swc repository implements all added methods.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not quite sure what you're suggesting here. I did update to the latest version if that helps.

@padmaia padmaia force-pushed the add/next-swc branch 2 times, most recently from c1aebfe to 7ac9a5b Compare July 21, 2021 16:43
@ijjk

This comment has been minimized.

@ijjk

This comment has been minimized.

@ijjk

This comment has been minimized.

@ijjk

This comment has been minimized.

@ijjk

This comment has been minimized.

Copy link
Member

@kdy1 kdy1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found some code readability issues, but mostly LGTM.

@ijjk

This comment has been minimized.

@ijjk

This comment has been minimized.

@timneutkens
Copy link
Member

Just local development on Next.js core. I haven't set the new loader up behind a flag so people downloading next as a package will not be affected at all yet.

Then that's totally fine 👍

@ijjk

This comment has been minimized.

@@ -36,6 +36,8 @@ Make sure you have `chromedriver` installed for your Chrome version. You can ins
- `chocolatey install chromedriver` on Windows
- Or manually download the version that matches your installed chrome version (if there's no match, download a version under it, but not above) from the [chromedriver repo](https://chromedriver.storage.googleapis.com/index.html) and add the binary to `<next-repo>/node_modules/.bin`

You may also have to [install Rust](https://www.rust-lang.org/tools/install) and build our native packages to see all tests pass locally. We check in binaries for the most common targets and those required for CI so that most people don't have to, but if you do not see a binary for your target in `packages/next/native`, you can build it by running `yarn --cwd packages/next build-native`. If you are working on the Rust code and you need to build the binaries for ci, you can manually trigger [the workflow](https://github.com/vercel/next.js/actions/workflows/build_native.yml) to build and commit with the "Run workflow" button.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: the manual workflow won't show up until it is merged to the default branch, but you can see what it will look like on my fork: https://github.com/padmaia/next.js/actions/workflows/build_native.yml

@ijjk
Copy link
Member

ijjk commented Jul 26, 2021

Stats from current PR

Default Build (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary padmaia/next.js add/next-swc Change
buildDuration 13.6s 13.8s ⚠️ +158ms
buildDurationCached 3.1s 3.1s ⚠️ +9ms
nodeModulesSize 50.3 MB 50.4 MB ⚠️ +33 kB
Page Load Tests Overall increase ✓
vercel/next.js canary padmaia/next.js add/next-swc Change
/ failed reqs 0 0
/ total time (seconds) 2.259 2.278 ⚠️ +0.02
/ avg req/sec 1106.65 1097.23 ⚠️ -9.42
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.3 1.286 -0.01
/error-in-render avg req/sec 1922.38 1943.89 +21.51
Client Bundles (main, webpack, commons)
vercel/next.js canary padmaia/next.js add/next-swc Change
359.HASH.js gzip 2.96 kB 2.96 kB
745.HASH.js gzip 180 B 180 B
framework-HASH.js gzip 42.2 kB 42.2 kB
main-HASH.js gzip 21 kB 21 kB
webpack-HASH.js gzip 1.53 kB 1.53 kB
Overall change 67.9 kB 67.9 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary padmaia/next.js add/next-swc Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB
Overall change 31.1 kB 31.1 kB
Client Pages
vercel/next.js canary padmaia/next.js add/next-swc Change
_app-HASH.js gzip 803 B 803 B
_error-HASH.js gzip 3.06 kB 3.06 kB
amp-HASH.js gzip 554 B 554 B
css-HASH.js gzip 329 B 329 B
dynamic-HASH.js gzip 2.52 kB 2.52 kB
head-HASH.js gzip 2.28 kB 2.28 kB
hooks-HASH.js gzip 903 B 903 B
image-HASH.js gzip 5.63 kB 5.63 kB
index-HASH.js gzip 261 B 261 B
link-HASH.js gzip 1.66 kB 1.66 kB
routerDirect..HASH.js gzip 319 B 319 B
script-HASH.js gzip 387 B 387 B
withRouter-HASH.js gzip 320 B 320 B
bb14e60e810b..30f.css gzip 125 B 125 B
Overall change 19.1 kB 19.1 kB
Client Build Manifests
vercel/next.js canary padmaia/next.js add/next-swc Change
_buildManifest.js gzip 489 B 489 B
Overall change 489 B 489 B
Rendered Page Sizes
vercel/next.js canary padmaia/next.js add/next-swc Change
index.html gzip 530 B 530 B
link.html gzip 544 B 544 B
withRouter.html gzip 524 B 524 B
Overall change 1.6 kB 1.6 kB

Webpack 4 Mode (Decrease detected ✓)
General Overall increase ⚠️
vercel/next.js canary padmaia/next.js add/next-swc Change
buildDuration 10.8s 11.1s ⚠️ +307ms
buildDurationCached 4.2s 4.1s -51ms
nodeModulesSize 50.3 MB 50.4 MB ⚠️ +33 kB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary padmaia/next.js add/next-swc Change
/ failed reqs 0 0
/ total time (seconds) 2.265 2.284 ⚠️ +0.02
/ avg req/sec 1103.73 1094.79 ⚠️ -8.94
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.277 1.323 ⚠️ +0.05
/error-in-render avg req/sec 1958.17 1890.26 ⚠️ -67.91
Client Bundles (main, webpack, commons)
vercel/next.js canary padmaia/next.js add/next-swc Change
17.HASH.js gzip 2.98 kB 2.98 kB
18.HASH.js gzip 185 B 185 B
677f882d2ed8..HASH.js gzip 13.8 kB 13.8 kB
framework.HASH.js gzip 41.9 kB 41.9 kB
main-HASH.js gzip 8.4 kB 8.4 kB
webpack-HASH.js gzip 1.22 kB 1.22 kB
Overall change 68.5 kB 68.5 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary padmaia/next.js add/next-swc Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB
Overall change 31.3 kB 31.3 kB
Client Pages
vercel/next.js canary padmaia/next.js add/next-swc Change
_app-HASH.js gzip 791 B 791 B
_error-HASH.js gzip 3.76 kB 3.76 kB
amp-HASH.js gzip 552 B 552 B
css-HASH.js gzip 333 B 333 B
dynamic-HASH.js gzip 2.71 kB 2.71 kB
head-HASH.js gzip 2.97 kB 2.97 kB
hooks-HASH.js gzip 911 B 911 B
index-HASH.js gzip 231 B 231 B
link-HASH.js gzip 1.64 kB 1.64 kB
routerDirect..HASH.js gzip 298 B 298 B
script-HASH.js gzip 3.02 kB 3.02 kB
withRouter-HASH.js gzip 294 B 294 B
e025d2764813..52f.css gzip 125 B 125 B
Overall change 17.6 kB 17.6 kB
Client Build Manifests
vercel/next.js canary padmaia/next.js add/next-swc Change
_buildManifest.js gzip 500 B 500 B
Overall change 500 B 500 B
Rendered Page Sizes
vercel/next.js canary padmaia/next.js add/next-swc Change
index.html gzip 575 B 575 B
link.html gzip 589 B 589 B
withRouter.html gzip 569 B 569 B
Overall change 1.73 kB 1.73 kB
Commit: 60dde3b

@padmaia
Copy link
Member Author

padmaia commented Jul 26, 2021

@timneutkens Added the license info 👍

Copy link
Member

@timneutkens timneutkens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@ijjk
Copy link
Member

ijjk commented Jul 26, 2021

Stats from current PR

Default Build (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary padmaia/next.js add/next-swc Change
buildDuration 17.7s 18s ⚠️ +284ms
buildDurationCached 4.1s 4s -126ms
nodeModulesSize 50.3 MB 50.4 MB ⚠️ +34.1 kB
Page Load Tests Overall increase ✓
vercel/next.js canary padmaia/next.js add/next-swc Change
/ failed reqs 0 0
/ total time (seconds) 3.089 3.036 -0.05
/ avg req/sec 809.4 823.38 +13.98
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.725 1.654 -0.07
/error-in-render avg req/sec 1449.49 1511.83 +62.34
Client Bundles (main, webpack, commons)
vercel/next.js canary padmaia/next.js add/next-swc Change
359.HASH.js gzip 2.96 kB 2.96 kB
745.HASH.js gzip 180 B 180 B
framework-HASH.js gzip 42.2 kB 42.2 kB
main-HASH.js gzip 21 kB 21 kB
webpack-HASH.js gzip 1.53 kB 1.53 kB
Overall change 67.9 kB 67.9 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary padmaia/next.js add/next-swc Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB
Overall change 31.1 kB 31.1 kB
Client Pages
vercel/next.js canary padmaia/next.js add/next-swc Change
_app-HASH.js gzip 803 B 803 B
_error-HASH.js gzip 3.06 kB 3.06 kB
amp-HASH.js gzip 554 B 554 B
css-HASH.js gzip 329 B 329 B
dynamic-HASH.js gzip 2.52 kB 2.52 kB
head-HASH.js gzip 2.28 kB 2.28 kB
hooks-HASH.js gzip 903 B 903 B
image-HASH.js gzip 5.63 kB 5.63 kB
index-HASH.js gzip 261 B 261 B
link-HASH.js gzip 1.66 kB 1.66 kB
routerDirect..HASH.js gzip 319 B 319 B
script-HASH.js gzip 387 B 387 B
withRouter-HASH.js gzip 320 B 320 B
bb14e60e810b..30f.css gzip 125 B 125 B
Overall change 19.1 kB 19.1 kB
Client Build Manifests
vercel/next.js canary padmaia/next.js add/next-swc Change
_buildManifest.js gzip 489 B 489 B
Overall change 489 B 489 B
Rendered Page Sizes
vercel/next.js canary padmaia/next.js add/next-swc Change
index.html gzip 531 B 531 B
link.html gzip 544 B 544 B
withRouter.html gzip 525 B 525 B
Overall change 1.6 kB 1.6 kB

Webpack 4 Mode (Decrease detected ✓)
General Overall increase ⚠️
vercel/next.js canary padmaia/next.js add/next-swc Change
buildDuration 14.1s 14.1s -4ms
buildDurationCached 5.4s 5.3s -120ms
nodeModulesSize 50.3 MB 50.4 MB ⚠️ +34.1 kB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary padmaia/next.js add/next-swc Change
/ failed reqs 0 0
/ total time (seconds) 3.024 3.146 ⚠️ +0.12
/ avg req/sec 826.77 794.58 ⚠️ -32.19
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.641 1.693 ⚠️ +0.05
/error-in-render avg req/sec 1523.85 1476.6 ⚠️ -47.25
Client Bundles (main, webpack, commons)
vercel/next.js canary padmaia/next.js add/next-swc Change
17.HASH.js gzip 2.98 kB 2.98 kB
18.HASH.js gzip 185 B 185 B
677f882d2ed8..HASH.js gzip 13.8 kB 13.8 kB
framework.HASH.js gzip 41.9 kB 41.9 kB
main-HASH.js gzip 8.4 kB 8.4 kB
webpack-HASH.js gzip 1.22 kB 1.22 kB
Overall change 68.5 kB 68.5 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary padmaia/next.js add/next-swc Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB
Overall change 31.3 kB 31.3 kB
Client Pages
vercel/next.js canary padmaia/next.js add/next-swc Change
_app-HASH.js gzip 791 B 791 B
_error-HASH.js gzip 3.76 kB 3.76 kB
amp-HASH.js gzip 552 B 552 B
css-HASH.js gzip 333 B 333 B
dynamic-HASH.js gzip 2.71 kB 2.71 kB
head-HASH.js gzip 2.97 kB 2.97 kB
hooks-HASH.js gzip 911 B 911 B
index-HASH.js gzip 231 B 231 B
link-HASH.js gzip 1.64 kB 1.64 kB
routerDirect..HASH.js gzip 298 B 298 B
script-HASH.js gzip 3.02 kB 3.02 kB
withRouter-HASH.js gzip 294 B 294 B
e025d2764813..52f.css gzip 125 B 125 B
Overall change 17.6 kB 17.6 kB
Client Build Manifests
vercel/next.js canary padmaia/next.js add/next-swc Change
_buildManifest.js gzip 500 B 500 B
Overall change 500 B 500 B
Rendered Page Sizes
vercel/next.js canary padmaia/next.js add/next-swc Change
index.html gzip 577 B 577 B
link.html gzip 588 B 588 B
withRouter.html gzip 569 B 569 B
Overall change 1.73 kB 1.73 kB
Commit: 7f7e547

@timneutkens timneutkens requested a review from kdy1 July 26, 2021 20:48
Copy link
Member

@kdy1 kdy1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed rust codes and there was no issue

@ijjk
Copy link
Member

ijjk commented Jul 27, 2021

Stats from current PR

Default Build (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary padmaia/next.js add/next-swc Change
buildDuration 18.5s 19s ⚠️ +509ms
buildDurationCached 4.3s 4.1s -224ms
nodeModulesSize 50.3 MB 50.4 MB ⚠️ +34.1 kB
Page Load Tests Overall increase ✓
vercel/next.js canary padmaia/next.js add/next-swc Change
/ failed reqs 0 0
/ total time (seconds) 3.129 3.103 -0.03
/ avg req/sec 799.08 805.58 +6.5
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 2.094 1.934 -0.16
/error-in-render avg req/sec 1193.73 1292.9 +99.17
Client Bundles (main, webpack, commons)
vercel/next.js canary padmaia/next.js add/next-swc Change
359.HASH.js gzip 2.96 kB 2.96 kB
745.HASH.js gzip 180 B 180 B
framework-HASH.js gzip 42.2 kB 42.2 kB
main-HASH.js gzip 21 kB 21 kB
webpack-HASH.js gzip 1.53 kB 1.53 kB
Overall change 67.9 kB 67.9 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary padmaia/next.js add/next-swc Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB
Overall change 31.1 kB 31.1 kB
Client Pages
vercel/next.js canary padmaia/next.js add/next-swc Change
_app-HASH.js gzip 803 B 803 B
_error-HASH.js gzip 3.06 kB 3.06 kB
amp-HASH.js gzip 554 B 554 B
css-HASH.js gzip 329 B 329 B
dynamic-HASH.js gzip 2.52 kB 2.52 kB
head-HASH.js gzip 2.28 kB 2.28 kB
hooks-HASH.js gzip 903 B 903 B
image-HASH.js gzip 5.63 kB 5.63 kB
index-HASH.js gzip 261 B 261 B
link-HASH.js gzip 1.66 kB 1.66 kB
routerDirect..HASH.js gzip 319 B 319 B
script-HASH.js gzip 387 B 387 B
withRouter-HASH.js gzip 320 B 320 B
bb14e60e810b..30f.css gzip 125 B 125 B
Overall change 19.1 kB 19.1 kB
Client Build Manifests
vercel/next.js canary padmaia/next.js add/next-swc Change
_buildManifest.js gzip 489 B 489 B
Overall change 489 B 489 B
Rendered Page Sizes
vercel/next.js canary padmaia/next.js add/next-swc Change
index.html gzip 531 B 531 B
link.html gzip 544 B 544 B
withRouter.html gzip 525 B 525 B
Overall change 1.6 kB 1.6 kB

Webpack 4 Mode (Decrease detected ✓)
General Overall increase ⚠️
vercel/next.js canary padmaia/next.js add/next-swc Change
buildDuration 14.7s 14.7s ⚠️ +38ms
buildDurationCached 5.7s 5.5s -216ms
nodeModulesSize 50.3 MB 50.4 MB ⚠️ +34.1 kB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary padmaia/next.js add/next-swc Change
/ failed reqs 0 0
/ total time (seconds) 3.171 3.253 ⚠️ +0.08
/ avg req/sec 788.46 768.45 ⚠️ -20.01
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 2.006 2.203 ⚠️ +0.2
/error-in-render avg req/sec 1246.17 1134.84 ⚠️ -111.33
Client Bundles (main, webpack, commons)
vercel/next.js canary padmaia/next.js add/next-swc Change
17.HASH.js gzip 2.98 kB 2.98 kB
18.HASH.js gzip 185 B 185 B
677f882d2ed8..HASH.js gzip 13.8 kB 13.8 kB
framework.HASH.js gzip 41.9 kB 41.9 kB
main-HASH.js gzip 8.4 kB 8.4 kB
webpack-HASH.js gzip 1.22 kB 1.22 kB
Overall change 68.5 kB 68.5 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary padmaia/next.js add/next-swc Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB
Overall change 31.3 kB 31.3 kB
Client Pages
vercel/next.js canary padmaia/next.js add/next-swc Change
_app-HASH.js gzip 791 B 791 B
_error-HASH.js gzip 3.76 kB 3.76 kB
amp-HASH.js gzip 552 B 552 B
css-HASH.js gzip 333 B 333 B
dynamic-HASH.js gzip 2.71 kB 2.71 kB
head-HASH.js gzip 2.97 kB 2.97 kB
hooks-HASH.js gzip 911 B 911 B
index-HASH.js gzip 231 B 231 B
link-HASH.js gzip 1.64 kB 1.64 kB
routerDirect..HASH.js gzip 298 B 298 B
script-HASH.js gzip 3.02 kB 3.02 kB
withRouter-HASH.js gzip 294 B 294 B
e025d2764813..52f.css gzip 125 B 125 B
Overall change 17.6 kB 17.6 kB
Client Build Manifests
vercel/next.js canary padmaia/next.js add/next-swc Change
_buildManifest.js gzip 500 B 500 B
Overall change 500 B 500 B
Rendered Page Sizes
vercel/next.js canary padmaia/next.js add/next-swc Change
index.html gzip 577 B 577 B
link.html gzip 588 B 588 B
withRouter.html gzip 569 B 569 B
Overall change 1.73 kB 1.73 kB
Commit: cf1d081

@ijjk
Copy link
Member

ijjk commented Jul 27, 2021

Failing test suites

Commit: cf1d081

test/integration/config-devtool-dev/test/index.test.js

  • devtool set in development mode in next config > should warn and revert when a devtool is set in development mode
Expand output

● devtool set in development mode in next config › should warn and revert when a devtool is set in development mode

thrown: "Exceeded timeout of 30000 ms for a test.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

  17 |
  18 | describe('devtool set in development mode in next config', () => {
> 19 |   it('should warn and revert when a devtool is set in development mode', async () => {
     |   ^
  20 |     let stderr = ''
  21 |
  22 |     const appPort = await findPort()

  at integration/config-devtool-dev/test/index.test.js:19:3
  at Object.<anonymous> (integration/config-devtool-dev/test/index.test.js:18:1)

@timneutkens timneutkens merged commit 4172a4c into vercel:canary Jul 27, 2021
@padmaia padmaia deleted the add/next-swc branch July 27, 2021 16:21
flybayer pushed a commit to blitz-js/next.js that referenced this pull request Aug 19, 2021
* Set up next-swc package with no custom transforms

* Add hook_optimizer transform

* Fix lint

* Build next-swc on CI

* Add toolchain in rust action

* Build binaries in manual workflow

* Commit from GitHub Actions (Build next-swc native binaries)

* Update dependencies

* Update swc and use stable rust

* Build next-swc binaries

* Test compiled code

* Dedupe @node-rs/helper

* Add workflow to check next-swc binaries

* Fix check native

* PR feedback

* PR feedback

* Pr feedback

* Build next-swc binaries

* Combine native workflows

* Add docs for contributors on less common platforms

* PR feedback

* Compare JsWord instead of converting to string

* Fix workflow formatting

* Add docs for building binaries for CI

* Build next-swc binaries

* Fix workflow if syntax

* Add license info to copied code

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
@vercel vercel locked as resolved and limited conversation to collaborators Jan 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants