misc(build): minify bundles with terser #9605

connorjclark · 2019-08-23T22:23:48Z

removes usage of babel for minifying, replace with terser
emits source maps for each bundle
adds a more detailed comment at the top of each bundle

I encourage reviewers to yarn build-lr and view the map on https://sokra.github.io/source-map-visualization/

dist/lightrider/lighthouse-lr-bundle.js 9.60 MB -> 8.94 MB (7%)
dist/lighthouse-dt-bundle.js 2.10 MB -> 1.62 MB (23%)

build/build-bundle.js

connorjclark · 2019-08-23T22:27:02Z

Note: the viewer needs additional changes to its build script to emit a map. This only emits a map for LR, the extension, and DT.

connorjclark · 2019-08-23T23:56:03Z

some changes....

there's an extra space in the fn declaration for modules:

connorjclark · 2019-08-24T00:02:42Z

not obvious why the devtools bundle got so much bigger (+ 504KB on disk).

Here's the visual bundle:

Can't do the same for "base"/master, because this tool requires a source map :)

connorjclark · 2019-08-24T00:14:30Z

got a visual bundle from the previous process of browserify -> babel (ccf966d).

as you can see, "unmapped" went up a lot.

connorjclark · 2019-08-24T00:21:30Z

the answer is in here somewhere... (diff b/wn dt bundles) https://gist.github.com/connorjclark/c80ca014042950ff0dd15ee23fcb6f42

~~Must be related to the babel version bump.~~ I think it's related to "minifying" with a transform during browserify, instead of processing the output of browserify.

connorjclark · 2019-08-24T00:32:40Z

yarn.lock

@@ -5412,6 +5228,14 @@ module-deps@^6.0.0:
    through2 "^2.0.0"
    xtend "^4.0.0"

+mold-source-map@~0.4.0:


this is an interesting package

connorjclark · 2019-08-26T23:06:44Z

The culprit is the browserify builtins. They don't get processed by the babelify transform - so comments, whitespace, all remain.

connorjclark · 2019-08-27T00:04:44Z

Here where we're at:

We do browserify -> babel. Bundle is 1.6M. Babel's job is just to reduce the size of the script (via retain_lines and compact). Great when their were two passes. However, emitting source maps with this two-pass compilation created off-by-one issues.

Moving babel into browserify via babelify is cool, but it missed out on minimizing the builtins that browserify injects. This adds ~0.3M to the devtools bundle.

We could completely remove the line retaining minimization, and just use Terser. This save ~0.4MB. Source maps are perfect in this two-pass compilation. Terser handles input maps better than babel, apparently.

I asked @patrickhulce about the utility of "retain lines" in the bundle output. He said it originated from wanting "readable" diffs when rolling to DevTools. I wonder if that is a behavior we should bother keeping?

patrickhulce · 2019-08-27T00:20:11Z

He said it originated from wanting "readable" diffs when rolling to DevTools

yeah that and the moderately improved stack trace is a nice feature of retainLines, but I'm not sure how much we need either at this point. I can't remember the last time we got a legit bug report where the bundled stack trace was useful. If we had a mechanism (script?) that could easily translate a stack trace using our sourcemap, saving 0.5MB sounds like a steal to me.

connorjclark · 2019-08-27T00:30:24Z

If we had a mechanism (script?) that could easily translate a stack trace using our sourcemap, saving 0.5MB sounds like a steal to me.

covered! https://www.npmjs.com/package/stack-beautifier

npx stack-beautifier -t <file_with_stack.txt> <app.js.map>

brendankenny · 2019-08-27T00:44:29Z

I like getting rid of yet another minifier in our deps :)

Did we figure out the serving thing, though? My main thing is probably the occasional stepping through LH in devtools and the current formatting does make that actually viable without a source map.

brendankenny · 2019-08-27T00:46:34Z

Agreed with Patrick that it's pretty rare I find myself actually doing that, but then again that means what use are source maps? :)

connorjclark · 2019-08-27T16:15:42Z

Did we figure out the serving thing, though? My main thing is probably the occasional stepping through LH in devtools and the current formatting does make that actually viable without a source map.

For devtools / Lightrider, you can attach source maps from a URL, which you would get by checking out the correct sha, building, and starting a local server in dist ... I think that's the best solution we have.

But don't forget the viewer - it's trivial to use maps there.

connorjclark · 2019-08-28T23:29:39Z

Worth noting: we have no tests that utilize the output of our bundled code (ala how React tests their lib). We might do good to run the smoke tests on the bundled code (#9501 references how).

paulirish

unfortunately, building the LR bundle now takes ~60 seconds. but there's not much we can do about that.

a few small things, but is good to get a green

paulirish · 2020-09-16T22:52:12Z

lighthouse-core/gather/gatherers/seo/tap-targets.js

@@ -308,7 +310,6 @@ class TapTargets extends Gatherer {
   */
  afterPass(passContext) {
    const expression = `(function() {
-      const tapTargetsSelector = "${tapTargetsSelector}";


i looked through the other uses of evalAsync for a similar situation and i think we're good.

only one that caught my eye was L116 of the start-url gatherer

lighthouse/lighthouse-core/gather/gatherers/start-url.js

Line 116 in 039b6c6

.evaluateAsync(`window.location = '${startUrl}'`)

but looking at the minified source, i think we're ok

ya?

in minified land this was that taptarget section:

yea code inside template strings' ${...} are safe

build/build-bundle.js

paulirish · 2020-09-16T23:09:14Z

package.json

@@ -1,7 +1,7 @@
 {
  "name": "lighthouse",
  "version": "6.3.0",
-  "description": "Lighthouse",
+  "description": "Automated auditing, performance metrics, and best practices for the web.",


package.json

paulirish · 2020-09-16T23:11:33Z

package.json

@@ -108,10 +108,9 @@
    "@wardpeet/brfs": "2.1.0-0",
    "angular": "^1.7.4",
    "archiver": "^3.0.0",
-    "babel-core": "^6.26.0",


brendankenny · 2020-09-17T00:04:10Z

package.json

    "browserify": "^16.2.3",
+    "browserify-banner": "^1.0.15",
+    "bundlesize": "^0.18.0",


what now

:)

brendankenny · 2020-09-17T00:06:34Z

build/build-bundle.js

-    assert.ok(!minified.includes('\nrequire='), 'contained unexpected browserify require stub');
+  // Add the banner and modify globals for DevTools if necessary.
+  // This will mess up the source map for the first line, but we don't ship
+  // the map to DevTools, so that's fine.


what does that leave that gets a source map?

I just removed this comment since we aren't shipping any maps in CDT or LR. I put the default map generation back to false. can set to true whenever we want to get a bundle viz.

build/build-bundle.js

Co-authored-by: Brendan Kenny <bckenny@gmail.com>

connorjclark added 2 commits August 23, 2019 15:17

initial commit

96da5ca

remove direct babel deps

1fa3d23

googlebot added the cla: yes label Aug 23, 2019

connorjclark commented Aug 23, 2019

View reviewed changes

build/build-bundle.js Outdated Show resolved Hide resolved

banner -> footer

7bc9ec2

vercel bot deployed to staging August 23, 2019 23:02 View deployment

connorjclark added 2 commits August 23, 2019 16:47

bump lr size

83d7db7

Merge branch 'master' into bundle-sourcemaps-2

3bd9fd5

vercel bot deployed to staging August 23, 2019 23:51 View deployment

connorjclark commented Aug 24, 2019

View reviewed changes

terser

ab20f97

vercel bot deployed to staging August 27, 2019 00:23 View deployment

connorjclark added 3 commits October 4, 2019 15:23

Merge remote-tracking branch 'origin/master' into bundle-sourcemaps-2

8ae4dd8

update

cad37a0

fix

0a38795

vercel bot deployed to staging October 4, 2019 22:46 View deployment

connorjclark changed the title ~~misc: emit source map for bundles~~ misc: minify bundles with terser, emit source map Oct 4, 2019

connorjclark added the waiting4reviewer label Sep 15, 2020

fix tap target

d657e76

vercel bot deployed to Preview September 15, 2020 21:45 View deployment

fix type check

e0b7eb9

vercel bot deployed to Preview September 16, 2020 00:10 View deployment

no need for mangle options now

abf8648

vercel bot deployed to Preview September 16, 2020 22:43 View deployment

paulirish approved these changes Sep 16, 2020

View reviewed changes

author

df900dd

vercel bot deployed to Preview September 16, 2020 23:39 View deployment

brendankenny reviewed Sep 17, 2020

View reviewed changes

connorjclark mentioned this pull request Sep 17, 2020

core: change how axe source is imported to reduce duplication #11442

Closed

update map generation

3a79dbb

vercel bot deployed to Preview September 17, 2020 00:38 View deployment

oops

dbc02fd

vercel bot deployed to Preview September 17, 2020 00:40 View deployment

Update build/build-bundle.js

0e828d2

Co-authored-by: Brendan Kenny <bckenny@gmail.com>

vercel bot deployed to Preview September 17, 2020 00:40 View deployment

rm unused package

bb49b42

vercel bot deployed to Preview September 17, 2020 00:41 View deployment

connorjclark changed the title ~~misc(build): minify bundles with terser, emit source map~~ misc(build): minify bundles with terser Sep 17, 2020

fix ts check

6140f42

vercel bot deployed to Preview September 17, 2020 21:22 View deployment

connorjclark added the land-when-ci-is-green label Sep 17, 2020

comment

8c39a2e

vercel bot deployed to Preview September 17, 2020 21:31 View deployment

fix

198a623

vercel bot deployed to Preview September 17, 2020 22:10 View deployment

connorjclark merged commit d41591d into master Sep 17, 2020

connorjclark deleted the bundle-sourcemaps-2 branch September 17, 2020 22:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

misc(build): minify bundles with terser #9605

misc(build): minify bundles with terser #9605

connorjclark commented Aug 23, 2019 •

edited

Loading

connorjclark commented Aug 23, 2019 •

edited

Loading

connorjclark commented Aug 23, 2019 •

edited

Loading

connorjclark commented Aug 24, 2019 •

edited

Loading

connorjclark commented Aug 24, 2019 •

edited

Loading

connorjclark commented Aug 24, 2019 •

edited

Loading

connorjclark Aug 24, 2019

connorjclark commented Aug 26, 2019

connorjclark commented Aug 27, 2019 •

edited by patrickhulce

Loading

patrickhulce commented Aug 27, 2019

connorjclark commented Aug 27, 2019

brendankenny commented Aug 27, 2019

brendankenny commented Aug 27, 2019

connorjclark commented Aug 27, 2019 •

edited

Loading

connorjclark commented Aug 28, 2019

paulirish left a comment

paulirish Sep 16, 2020

connorjclark Sep 16, 2020 •

edited

Loading

paulirish Sep 16, 2020

paulirish Sep 16, 2020

brendankenny Sep 17, 2020

brendankenny Sep 17, 2020

connorjclark Sep 17, 2020

misc(build): minify bundles with terser #9605

misc(build): minify bundles with terser #9605

Conversation

connorjclark commented Aug 23, 2019 • edited Loading

connorjclark commented Aug 23, 2019 • edited Loading

connorjclark commented Aug 23, 2019 • edited Loading

connorjclark commented Aug 24, 2019 • edited Loading

connorjclark commented Aug 24, 2019 • edited Loading

connorjclark commented Aug 24, 2019 • edited Loading

connorjclark Aug 24, 2019

Choose a reason for hiding this comment

connorjclark commented Aug 26, 2019

connorjclark commented Aug 27, 2019 • edited by patrickhulce Loading

patrickhulce commented Aug 27, 2019

connorjclark commented Aug 27, 2019

brendankenny commented Aug 27, 2019

brendankenny commented Aug 27, 2019

connorjclark commented Aug 27, 2019 • edited Loading

connorjclark commented Aug 28, 2019

paulirish left a comment

Choose a reason for hiding this comment

paulirish Sep 16, 2020

Choose a reason for hiding this comment

connorjclark Sep 16, 2020 • edited Loading

Choose a reason for hiding this comment

paulirish Sep 16, 2020

Choose a reason for hiding this comment

paulirish Sep 16, 2020

Choose a reason for hiding this comment

brendankenny Sep 17, 2020

Choose a reason for hiding this comment

brendankenny Sep 17, 2020

Choose a reason for hiding this comment

connorjclark Sep 17, 2020

Choose a reason for hiding this comment

connorjclark commented Aug 23, 2019 •

edited

Loading

connorjclark commented Aug 23, 2019 •

edited

Loading

connorjclark commented Aug 23, 2019 •

edited

Loading

connorjclark commented Aug 24, 2019 •

edited

Loading

connorjclark commented Aug 24, 2019 •

edited

Loading

connorjclark commented Aug 24, 2019 •

edited

Loading

connorjclark commented Aug 27, 2019 •

edited by patrickhulce

Loading

connorjclark commented Aug 27, 2019 •

edited

Loading

connorjclark Sep 16, 2020 •

edited

Loading