Migrate from Webpacker to ViteJS #24981

renchap · 2023-05-14T21:31:12Z

Warning: this is a big work in progress with a lot of things not working.

Is it based on my #24906 PR, so ignore Intl-related commits for now.

With this change, the frontend toolchain is managed by Vite, a fast Javascript bundler based on esbuild. It brings a lot of improvements, including support for all modern JS features, much more efficient code-splitting, far less required packages (from 1669 to 1139 at my last count) and a much improved performance.

Todo:

github-actions · 2023-05-15T13:00:46Z

This pull request has merge conflicts that must be resolved before it can be merged.

ThisIsMissEm

Might be worth adding the following to .gitattributes:

*.tsx.snap linguist-generated
*.jsx.snap linguist-generated

This will make the snapshots be collapsed by default when review the PRs.

vite.config.ts

ThisIsMissEm

This is looking good, left comments on a few things I noticed when looking through it, though I know it's not fully ready yet.

.github/dependabot.yml

ThisIsMissEm · 2023-05-15T17:52:11Z

Gemfile.lock

+
+RUBY VERSION
+   ruby 3.2.2p53
+
+BUNDLED WITH
+   2.4.12


I think this is normally not committed? (though I personally think it should be)

app/javascript/entrypoints/application.js

app/javascript/mastodon/features/emoji/emoji_compressed.js

app/javascript/mastodon/load_locale.js

app/views/admin/accounts/index.html.haml

app/views/auth/sessions/two_factor.html.haml

app/views/settings/two_factor_authentication/webauthn_credentials/new.html.haml

app/views/shared/_web_app.html.haml

renchap · 2023-05-19T22:13:25Z

Might be worth adding the following to .gitattributes:
*.tsx.snap linguist-generated
*.jsx.snap linguist-generated
This will make the snapshots be collapsed by default when review the PRs.

I am not sure this is a good idea, you might want to review any diffs in snapshots to ensure the snapshot did not get updated to something not expected?

github-actions · 2023-05-20T09:16:33Z

This pull request has resolved merge conflicts and is ready for review.

renchap · 2023-05-20T09:49:18Z

With the removal of Webpacker, Jest (and therefore most of Babel), we go from 1670 node packages to less than 1000 🎉

Slowly moving forward with migrating all the special usecases

Locale files are now loaded properly, ignoring the ones we dont need
Images should load properly (more testing is needed here)
The share page load correctly
Output files are compressed using gzip and brotli, like the current Webpack setup
I added some custom code to include the parent's folder name when possible in the output filename, otherwise there were a lot of unhelpful index-smthg.js

Some assets were using media/… paths, I am not sure why, they are now treated like the other assets.

I still need to ensure every other entrypoint is loading correctly.

The locale file should be manually pre-loaded as we know the locale server-side, should should not be hard to add.

I added a way to visualise the output size through a RollUp plugin:

It shows that the emoji data files are bundled in the history.js chunk, which is weird. I need to detangle all this emoji-picker code, and ensure it is loaded only when needed using a dynamic import. It should not be this hard, using <Suspense> I think.

But the emoji-related code is a real mess, and it would be very welcome to update to a recent and supported Emoji-mart, and remove all this weird emoji handling code.

github-actions · 2023-05-22T12:03:38Z

This pull request has merge conflicts that must be resolved before it can be merged.

github-actions · 2023-06-01T10:10:55Z

This pull request has resolved merge conflicts and is ready for review.

github-actions · 2023-06-01T13:06:52Z

This pull request has merge conflicts that must be resolved before it can be merged.

mjankowski · 2023-08-23T19:43:24Z

I was looking around at blockers to updating Node and stumbled across this one ... I wonder if there's a more incremental way to do things here? Migrating one piece at a time instead of bigger replacement?

The only thing that comes to mind here would be to try to isolate the various areas (admin web pages, user settings pages, the main react app, etc) and/or also isolate the assets, stylesheets, images, fonts, etc -- and try to go piece by piece there instead of all at one.

I can open a POC PR along those lines if you think that might nudge this forward.

renchap · 2023-08-23T20:14:52Z

We could try to migrate entrypack per entrypack, but it is quite limited because of the shared code between them, which may need to be adapted from Vite, or handle both Vite & Webpack, which can be complex.

I stopped my work on this because it will not be mergeable until 4.2 is released, but I still intent to have it all migrated.

I could extract the migration from jest to vitest, it might make sense and reduce the overall diff a bit.

github-actions · 2023-09-22T21:02:45Z

This pull request has resolved merge conflicts and is ready for review.

github-actions · 2023-09-25T09:15:18Z

This pull request has merge conflicts that must be resolved before it can be merged.

github-actions · 2024-04-15T10:04:53Z

This pull request has merge conflicts that must be resolved before it can be merged.

github-actions · 2024-04-22T17:03:17Z

This pull request has resolved merge conflicts and is ready for review.

renchap · 2024-04-23T08:35:20Z

Some more updates:

the service worker has been converted to work with Vite. It should behave the same, with Workbox as a preloader/cache layer (is this really needed? Unsure, but its there)
babel-plugin-preval is no longer used to execute a JS file at compilation time and bundle its result in the service worker (for only packing the needed locales), I switched it to a custom vite plugin which does the same thing though a virtual module. Note: this could be used to remove preval entirely later for the 2 remaining emoji-related files
app/javascript/packs is renamed to /app/javascript/entrypoints in a first commit while still using webpacker, which we should be able to extract in a separate PR
the production output is kept in public/packs, so there is no need to change nginx configuration or any specific server/CDN config

github-actions · 2024-04-23T17:04:41Z

This pull request has merge conflicts that must be resolved before it can be merged.

github-actions · 2024-04-23T23:02:42Z

This pull request has resolved merge conflicts and is ready for review.

github-actions · 2024-04-26T15:05:37Z

This pull request has merge conflicts that must be resolved before it can be merged.

github-actions · 2024-04-29T09:09:01Z

This pull request has resolved merge conflicts and is ready for review.

github-actions · 2024-04-29T16:05:04Z

This pull request has merge conflicts that must be resolved before it can be merged.

renchap mentioned this pull request May 14, 2023

Replace to shakapacker from webpacker #18369

Closed

github-actions bot added the rebase needed 🚧 label May 15, 2023

renchap force-pushed the vite branch from 1f87f68 to b3dd313 Compare May 15, 2023 13:08

ThisIsMissEm reviewed May 15, 2023

View reviewed changes

vite.config.ts Outdated Show resolved Hide resolved

ThisIsMissEm reviewed May 15, 2023

View reviewed changes

renchap mentioned this pull request May 19, 2023

Use same babel plugins across environments #24854

Closed

renchap force-pushed the vite branch from b3dd313 to a2cb8ce Compare May 20, 2023 09:16

github-actions bot removed the rebase needed 🚧 label May 20, 2023

github-actions bot added the rebase needed 🚧 label May 22, 2023

renchap force-pushed the vite branch from a2cb8ce to e67de5c Compare June 1, 2023 10:10

github-actions bot removed the rebase needed 🚧 label Jun 1, 2023

github-actions bot added the rebase needed 🚧 label Jun 1, 2023

renchap mentioned this pull request Jul 1, 2023

Update project to use Node.js v18.x #20968

Closed

renchap force-pushed the vite branch from e67de5c to 4823d19 Compare September 22, 2023 20:20

github-actions bot removed the rebase needed 🚧 label Sep 22, 2023

renchap force-pushed the vite branch from 4823d19 to 5a5328e Compare September 22, 2023 21:32

github-actions bot added the rebase needed 🚧 label Sep 25, 2023

renchap force-pushed the vite branch from 5a5328e to 26f7fc8 Compare November 9, 2023 22:08

mjankowski mentioned this pull request Apr 12, 2024

Replace jest with test:js in package scripts #29938

Closed

github-actions bot added the rebase needed 🚧 label Apr 15, 2024

renchap force-pushed the vite branch 2 times, most recently from 19dd515 to 5b13aff Compare April 22, 2024 16:13

github-actions bot removed the rebase needed 🚧 label Apr 22, 2024

renchap force-pushed the vite branch 4 times, most recently from b56cbc8 to 8e7cb56 Compare April 23, 2024 08:30

renchap force-pushed the vite branch from 8e7cb56 to c41c2af Compare April 23, 2024 08:38

mjankowski mentioned this pull request Apr 23, 2024

Move JS source from packs to entrypoints #30037

Merged

github-actions bot added the rebase needed 🚧 label Apr 23, 2024

renchap force-pushed the vite branch from c41c2af to a9d0bf2 Compare April 23, 2024 22:12

github-actions bot removed the rebase needed 🚧 label Apr 23, 2024

github-actions bot added the rebase needed 🚧 label Apr 26, 2024

renchap added 4 commits April 29, 2024 10:58

Migrate from Jest to Vitest

fe1c378

Replace Webpacker usage with Vite

5aadd56

Remove everything webpack-related

2cdf08f

Convert the service worker to work with Vite

4842bd0

renchap force-pushed the vite branch from a9d0bf2 to 4842bd0 Compare April 29, 2024 09:01

github-actions bot removed the rebase needed 🚧 label Apr 29, 2024

github-actions bot added the rebase needed 🚧 label Apr 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate from Webpacker to ViteJS #24981

Migrate from Webpacker to ViteJS #24981

renchap commented May 14, 2023 •

edited

github-actions bot commented May 15, 2023

ThisIsMissEm left a comment

ThisIsMissEm left a comment

ThisIsMissEm May 15, 2023

renchap commented May 19, 2023

github-actions bot commented May 20, 2023

renchap commented May 20, 2023

github-actions bot commented May 22, 2023

github-actions bot commented Jun 1, 2023

github-actions bot commented Jun 1, 2023

mjankowski commented Aug 23, 2023

renchap commented Aug 23, 2023

github-actions bot commented Sep 22, 2023

github-actions bot commented Sep 25, 2023

github-actions bot commented Apr 15, 2024

github-actions bot commented Apr 22, 2024

renchap commented Apr 23, 2024

github-actions bot commented Apr 23, 2024

github-actions bot commented Apr 23, 2024

github-actions bot commented Apr 26, 2024

github-actions bot commented Apr 29, 2024

github-actions bot commented Apr 29, 2024

Migrate from Webpacker to ViteJS #24981

Are you sure you want to change the base?

Migrate from Webpacker to ViteJS #24981

Conversation

renchap commented May 14, 2023 • edited

github-actions bot commented May 15, 2023

ThisIsMissEm left a comment

Choose a reason for hiding this comment

ThisIsMissEm left a comment

Choose a reason for hiding this comment

ThisIsMissEm May 15, 2023

Choose a reason for hiding this comment

renchap commented May 19, 2023

github-actions bot commented May 20, 2023

renchap commented May 20, 2023

github-actions bot commented May 22, 2023

github-actions bot commented Jun 1, 2023

github-actions bot commented Jun 1, 2023

mjankowski commented Aug 23, 2023

renchap commented Aug 23, 2023

github-actions bot commented Sep 22, 2023

github-actions bot commented Sep 25, 2023

github-actions bot commented Apr 15, 2024

github-actions bot commented Apr 22, 2024

renchap commented Apr 23, 2024

github-actions bot commented Apr 23, 2024

github-actions bot commented Apr 23, 2024

github-actions bot commented Apr 26, 2024

github-actions bot commented Apr 29, 2024

github-actions bot commented Apr 29, 2024

renchap commented May 14, 2023 •

edited