fix(build): fix tree shaking and bundle size #1343

thekip · 2023-01-17T10:14:45Z

Guys, I came with bad news. During working with size-limit i noticed that we don't test for the size real entries written in package.json such as build/index.js build/esm/inde.js

Instead, size-limit shamelessly set up to test a productiondirectly. Assuming that user's bundlers will also consume a production bundle somehow.

I decided to setup size-limit to the real entry, and it showed completely different results. The bundle size increased almost twice.
Firstly, i thought this is a problem with size-limit itself, maybe it doesn't replace process.env.NODE_ENV === "production" or doesn't apply treeshaking.
But after hours of digging in the size-limit sourcecode and webpack / esbuild documentation, i've found that size-limit pointing to a real problem with current ESM builds.

The problem

Current ESM entries written as:

import {
  I18nProvider as devI18nProvider, Trans as devTrans, useLingui as devuseLingui, withI18n as devwithI18n
} from "./react.development.js"

import {
  I18nProvider as I18nProviderProd, Trans as TransProd, useLingui as useLinguiProd, withI18n as withI18nProd
} from "./react.production.min.js"

export const I18nProvider = process.env.NODE_ENV === "production" ? I18nProviderProd : devI18nProvider;
export const Trans = process.env.NODE_ENV === "production" ? TransProd : devTrans;
export const useLingui = process.env.NODE_ENV === "production" ? useLinguiProd : devuseLingui;
export const withI18n = process.env.NODE_ENV === "production" ? withI18nProd : devwithI18n;

Assuming the bundler will replace process.env.NODE_ENV === "production" to the true and then only production branch would be retained after tree-shaking.
Unfortunately, bundlers don't work like that. They do replace ENV statement but not tree-shake a "conditional" import. Bundlers are not smart enough and they are very conservative. This code is just not safe to be tree-shaked from theirs point of view.

So our current ESM builds (which are consumed by all bundlers and shipped to the browser) contains both development and production builds together with bundled dependencies (this is another issue)

Proofs:
https://esbuild.github.io/api/#conditionally-injecting-a-file

Try it yourself

cd ./packages/core
esbuild ./build/esm/index.js --bundle --outdir=esbuild --minify --tree-shaking
# inspect output file, both bundles would be in the result as well as message-parser dep

Solution

According to the doc to have this imports to be treeshaked each file should be in its own folder with package.json with {sideEffect:false}

I didn't go that way, because I think that producing "production" and "development" bundles should be up to the user's bundler.

It had a sense when library produced as UMD module which you can directly use in browser consuming from CDN.
But for NPM library it's useless because it's going to be bundled and minified by the user's bundler.

So I changed the build process to produce only one version of the bundle, with conditions and left the rest of the job for the consumers bundler.

This make a build process so much simpler and so much closer to the modern build practices.

Issue with dependencies

Most of the dependencies were not externalized. Mean they were bundled together with library code.
Especially babel/core and @format-message/parser. That happened because externals check in rollup options had a logical issue.
I fixed it and wrote more robust check. Everethyng from node_modules should be externalized + packages from monorepo.

What was changed & updated

Remove building production/minified build + dev build in favor of one tree-shakable build for both ESM and CJS
Normalize all entries in package.json files. Added exports field where it was not presented
Update Rollup + Plugins to v3
Updated babel config to support node v14 instead of v8
Rollup typings in one "bundled" typings file according to best practices on the market.

In prior APF versions each entry point would have a src directory next to the .d.ts entry point and this directory contained individual d.ts files matching the structure of the original source code. While this distribution format is still allowed and supported, it is highly discouraged because it confuses tools like IDEs that then offer incorrect autocompletion, and allows users to depend on deep-import paths which are typically not considered to be public API of a library or a package.

Leverage Rollup's multioutput feature to produce cjs/esm instead of 2 separate builds processes. This speedup build.
Remove size reporting after rollup build in favor of more real-life reporting by "size-limit", it actually became not compatible with multioutput and I did not want to fix it.
Moved jest.configs to the root of repo, for easier integration with IDEs
Cleaned not used dependecies from monorepo package.json

I also think to change babel to esbuild, but this is not decided yet.

vercel · 2023-01-17T10:14:50Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
js-lingui	✅ Ready (Inspect)	Visit Preview	Jan 17, 2023 at 4:51PM (UTC)

thekip · 2023-01-17T11:33:50Z

scripts/build/bundles.ts

-// Based on deep-freeze by substack (public domain)
-function deepFreeze(o) {
-  Object.freeze(o)
-  Object.getOwnPropertyNames(o).forEach(function(prop) {
-    if (
-      o[prop] !== null &&
-      (typeof o[prop] === "object" || typeof o[prop] === "function") &&
-      !Object.isFrozen(o[prop])
-    ) {
-      deepFreeze(o[prop])
-    }
-  })
-  return o
-}
-
-// Don't accidentally mutate config as part of the build
-deepFreeze(bundles)


Deleted in favor of Typescript static analysis.

thekip · 2023-01-17T13:43:32Z

Possibly related #1263

thekip · 2023-01-17T16:35:14Z

packages/core/src/i18n.ts

@@ -1,7 +1,7 @@
 import { interpolate, UNICODE_REGEX } from "./context"
 import { isString, isFunction } from "./essentials"
 import { date, number } from "./formats"
-import { compileMessage } from "./compile"
+import { compileMessage } from "@lingui/core/compile"


That's very important change. If we would access this with relative path, tree shaker would not be able to eliminate this import.

Because of package import, tree shaker first looks into package's pakage.json, see {sideEffects: false} and then remove it from the bundle.

thekip · 2023-01-17T16:42:44Z

packages/core/src/index.ts

 export {
  setupI18n,
  I18n,
+} from "./i18n"
+
+export type {
  AllMessages,
  MessageDescriptor,
  Messages,


Without typescript plugin, rollup treat type only exports as regular, and obviously fail. Here we do a little help for him and manually mark them as type only.

thekip · 2023-01-17T16:43:46Z

packages/loader/package.json

@@ -41,7 +41,8 @@
    "loader-utils": "^2.0.0"
  },
  "devDependencies": {
-    "memory-fs": "^0.5.0"
+    "memory-fs": "^0.5.0",
+    "webpack": "^4.44.2"


That was defined on monorepo root but actually used here, i fixed it.

scripts/build/rollup.ts

thekip · 2023-01-17T16:45:47Z

scripts/build/rollup.ts

+import ora from "ora";
+import chalk from "chalk";
+
+const codeFrame = require("@babel/code-frame")


this package doesn't have typings

thekip · 2023-01-17T16:51:44Z

scripts/build/babel.config.js

-          node: 8,
-          browsers: "> 1%, last 2 versions"
+          node: 14,
+          browsers: "> 1%, last 2 versions, not dead"


not dead is recommended by browserlist itself. Otherwise, query last 2 versions will pull internet explorer and other dead browsers.

thekip · 2023-01-17T16:54:00Z

scripts/build/rollup.ts

+import babelConfig from './babel.config';
+
+import ora from "ora";
+import chalk from "chalk";


also i tried to update chalk to the latest version, but unfortunately it distributed as ESM package and you can not import ESM package from cjs code. So we stick with this version for a while.

andrii-bodnar · 2023-01-18T10:05:34Z

@thekip thank you! 🚀

thekip requested a review from Martin005 January 17, 2023 10:15

vercel bot deployed to Preview January 17, 2023 10:16 View deployment

Martin005 changed the title ~~fix(build) fix tree shaking and bundle size~~ fix(build): fix tree shaking and bundle size Jan 17, 2023

Martin005 requested a review from andrii-bodnar January 17, 2023 10:26

vercel bot deployed to Preview January 17, 2023 11:14 View deployment

thekip force-pushed the fix/tree-shaking branch from bb9ff1e to d6f1a0c Compare January 17, 2023 11:25

thekip marked this pull request as ready for review January 17, 2023 11:26

vercel bot deployed to Preview January 17, 2023 11:27 View deployment

thekip commented Jan 17, 2023

View reviewed changes

thekip force-pushed the fix/tree-shaking branch from d6f1a0c to 122e0bc Compare January 17, 2023 16:21

vercel bot deployed to Preview January 17, 2023 16:22 View deployment

thekip force-pushed the fix/tree-shaking branch from 122e0bc to ec0df9d Compare January 17, 2023 16:29

vercel bot deployed to Preview January 17, 2023 16:30 View deployment

thekip force-pushed the fix/tree-shaking branch from ec0df9d to b1029ec Compare January 17, 2023 16:31

vercel bot deployed to Preview January 17, 2023 16:33 View deployment

thekip commented Jan 17, 2023

View reviewed changes

Martin005 reviewed Jan 17, 2023

View reviewed changes

scripts/build/rollup.ts Outdated Show resolved Hide resolved

thekip commented Jan 17, 2023

View reviewed changes

fix(build) fix tree shaking and bundle size

03b85f6

thekip force-pushed the fix/tree-shaking branch from b1029ec to 03b85f6 Compare January 17, 2023 16:50

vercel bot deployed to Preview January 17, 2023 16:51 View deployment

thekip commented Jan 17, 2023

View reviewed changes

Martin005 approved these changes Jan 17, 2023

View reviewed changes

andrii-bodnar approved these changes Jan 18, 2023

View reviewed changes

andrii-bodnar merged commit 567905c into lingui:main Jan 18, 2023

andrii-bodnar mentioned this pull request Jan 18, 2023

Cannot find module '@lingui/core/compile' #1347

Closed

thekip deleted the fix/tree-shaking branch January 22, 2023 12:22

thekip mentioned this pull request Jan 23, 2023

Major bundlesize increase with @lingui/core 3.14.0 #1263

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(build): fix tree shaking and bundle size #1343

fix(build): fix tree shaking and bundle size #1343

thekip commented Jan 17, 2023 •

edited

vercel bot commented Jan 17, 2023 •

edited

thekip Jan 17, 2023

thekip commented Jan 17, 2023

thekip Jan 17, 2023

thekip Jan 17, 2023

thekip Jan 17, 2023

thekip Jan 17, 2023

thekip Jan 17, 2023

thekip Jan 17, 2023

andrii-bodnar commented Jan 18, 2023

fix(build): fix tree shaking and bundle size #1343

fix(build): fix tree shaking and bundle size #1343

Conversation

thekip commented Jan 17, 2023 • edited

The problem

Solution

Issue with dependencies

What was changed & updated

vercel bot commented Jan 17, 2023 • edited

thekip Jan 17, 2023

Choose a reason for hiding this comment

thekip commented Jan 17, 2023

thekip Jan 17, 2023

Choose a reason for hiding this comment

thekip Jan 17, 2023

Choose a reason for hiding this comment

thekip Jan 17, 2023

Choose a reason for hiding this comment

thekip Jan 17, 2023

Choose a reason for hiding this comment

thekip Jan 17, 2023

Choose a reason for hiding this comment

thekip Jan 17, 2023

Choose a reason for hiding this comment

andrii-bodnar commented Jan 18, 2023

thekip commented Jan 17, 2023 •

edited

vercel bot commented Jan 17, 2023 •

edited