Switch to lodash + lodash-webpack-plugin ... #5616

benvinegar · 2017-06-24T01:46:01Z

Removes underscore in favor of lodash
Uses lodash-webpack-plugin + babel-lodash to minimize filesize impact
Saves ~110kb unminified between removing underscore (~50kb) and using lodash-webpack-plugin to generate a partial lodash build (another ~60kb)

A big reason we did this was because we have some dependencies which were requiring lodash anyways, which meant we were bundling both lodash and underscore together 🤐.

I experimented with a bunch of dependency changes:

Dependencies	Filesize (unminified)
underscore + full lodash (before)	263kb
underscore + optimized lodash	164kb
optimized lodash only (this PR)	154kb

Interestingly, removing underscore doesn't result in as much benefit as you'd expect (vs optimizing the lodash build alone), because those functions just end up being transferred to the lodash build.

Note: need to make corresponding changes in getsentry repo, since it uses a global alias to sentry's exported underscore global.

cc @dcramer

#nochanges

benvinegar · 2017-06-24T01:48:04Z

src/sentry/static/sentry/app/views/organizationStats/index.jsx

@@ -1,4 +1,4 @@
-import $ from 'jquery';
+import _ from 'lodash';


This wasn't really necessary, it got pulled with some other changes I was exploring (using jQuery's slim build which doesn't have $.extend) ... I can revert.

benvinegar · 2017-06-24T01:48:53Z

src/sentry/static/sentry/app/components/events/interfaces/breadcrumbs/crumbTable.jsx

-        );
-      })
-      .value();
+    let tuples = _.map(kvData, (val, key) => [val, key]);


The lodash webpack bundle doesn't support chaining, and lodash's .map doesn't play as nice w/ empty objects so I made this more defensive.

jdalton · 2017-06-24T03:56:40Z

webpack.config.js

@@ -136,6 +136,10 @@ var config = {
    ]
  },
  plugins: [
+    new LodashModuleReplacementPlugin({
+      shorthands: true,


If you can avoid the shorthands feature it will save you a considerable amount. It's one of the heaviest features.

jdalton · 2017-06-24T03:57:47Z

You can upgrade to webpack v3 and use new webpack.optimize.ModuleConcatenationPlugin() for the production build to optimize the build a bit more.

benvinegar · 2017-06-24T06:00:05Z

You can upgrade to webpack v3 and use new webpack.optimize.ModuleConcatenationPlugin() for the production build to optimize the build a bit more.

Yeah, I'll probably do that separately, and maybe after a few patch releases.

And thanks for your feedback on this PR.

jdalton · 2017-06-24T06:29:21Z

Sold!

I'm curious what the minified bundle numbers are now that you dropped the "shorthands" feature set.

Yeah, I'll probably do that separately, and maybe after a few patch releases.

And thanks for your feedback on this PR.

Cool. When you do keep in mind the ModuleConcatenationPlugin that enables scope hoisting (think rollup) works on ES6 modules. So to get the benefit with your deps, like Lodash, you'll want to use the lodash-es module instead. You can even use webpack to alias lodash requires as lodash-es too.

benvinegar · 2017-06-24T06:33:29Z

So to get the benefit with your deps, like Lodash, you'll want to use the lodash-es module instead.
You can even use webpack to alias lodash requires as lodash-es too.

Good to know.

I'm curious what the minified bundle numbers are now that you dropped the "shorthands" feature set.

I did re-run the numbers, and it turned out that by adding collections, the number jumped up (I didn't update the table), and after removing shorthands, it went back down to the same figure: ~154kb. I ran it again just now to check.

jdalton · 2017-06-24T06:36:32Z

~154kb. I ran it again just now to check.

That's unminified right? Lodash has inline documentation in its unminified form so that is a falsely inflated view. You can minify as well as gzip to get a more prod-centric size.

benvinegar · 2017-06-24T06:37:49Z

That's unminified right? Lodash has inline documentation in its unminified form so that is a falsely inflated view. If you minify as well as gzip to get a more prod-centric size.

Also good to know. Yeah, I'm excited for the results – just too impatient to wait for webpack -p to finish 😅

dcramer · 2017-06-24T16:31:07Z

src/sentry/static/sentry/app/views/organizationStats/index.jsx

@@ -92,7 +92,7 @@ const OrganizationStats = React.createClass({
      });
    });

-    _.each(this.state.rawProjectData, statName => {
+    $.each(this.state.rawProjectData, statName => {


couldn't se just use .forEach at this point?

couldn't se just use .forEach at this point?

If it's an array yes. If you're only working with arrays then you also don't need the "collections" feature set.

MaxBittker

we need audit the other uses of underscore that didn't show up in this diff for problems like pick/pickBy (which I only saw because of the mapValues change). glad we're doing this though

MaxBittker · 2017-07-11T20:17:22Z

src/sentry/static/sentry/app/components/events/interfaces/breadcrumbs/crumbTable.jsx

-      })
-      .value();
+    return _.map(this.props.kvData, (val, key) => [val, key]).map(([val, key]) => {
+      return (


return _.map(this.props.kvData, (val, key) => [val, key]).map(([val, key]) => {
is equivalent to
return _.map(this.props.kvData, (val, key) => {

Laugh, true

MaxBittker · 2017-07-11T20:20:42Z

src/sentry/static/sentry/app/components/forms/form.jsx

@@ -1,5 +1,6 @@
 import React from 'react';
-import underscore from 'underscore';
+import lodash from 'lodash';


use _ here for consistency?

If _ can be used safely in the file without conflicts, yes.

MaxBittker · 2017-07-11T20:21:02Z

src/sentry/static/sentry/app/components/internalStatChart.jsx

@@ -1,6 +1,6 @@
 /*eslint getsentry/jsx-needs-il8n:0*/
 import React from 'react';
-import underscore from 'underscore';
+import underscore from 'lodash';


here too, _

See comment above. If that can't be done, rename to lodash.

MaxBittker · 2017-07-11T20:21:17Z

src/sentry/static/sentry/app/components/stackedBarChart.jsx

@@ -1,5 +1,5 @@
 import moment from 'moment';
-import underscore from 'underscore';
+import underscore from 'lodash';


and this one, _

MaxBittker · 2017-07-11T20:21:30Z

src/sentry/static/sentry/app/index.js

@@ -44,7 +44,7 @@ import ReactDOM from 'react-dom';
 import {renderToStaticMarkup} from 'react-dom/server';
 import Reflux from 'reflux';
 import * as Router from 'react-router';
-import underscore from 'underscore';
+import underscore from 'lodash';


this one too _

this one will break the bundle's external API

MaxBittker · 2017-07-11T20:24:43Z

src/sentry/static/sentry/app/plugins/components/settings.jsx

@@ -1,5 +1,5 @@
 import React from 'react';
-import underscore from 'underscore';
+import underscore from 'lodash';


'_' in this file too IMO

MaxBittker · 2017-07-11T20:26:17Z

src/sentry/static/sentry/app/views/asyncView.jsx

@@ -1,6 +1,6 @@
 import DocumentTitle from 'react-document-title';
 import React from 'react';
-import underscore from 'underscore';
+import lodash from 'lodash';


_ if you please

MaxBittker · 2017-07-11T20:33:00Z

src/sentry/static/sentry/app/utils/deviceNameMapper.jsx

@@ -1,5 +1,5 @@
 import iOSDeviceList from 'ios-device-list';
-import {isString} from 'underscore';
+import {isString} from 'lodash';


do we get static analysis benefits from importing like this?

In this case, not really since we're using this webpack plugin.

MaxBittker · 2017-07-11T20:37:00Z

src/sentry/static/sentry/app/views/installWizard.jsx

@@ -163,7 +163,7 @@ const InstallWizard = React.createClass({
    let loadingIndicator = IndicatorStore.add(t('Saving changes..'));

    // We only want to send back the values which weren't disabled
-    let data = _.mapObject(
+    let data = _.mapValues(
      _.pick(options, option => !option.field.disabled),


pick doesn't doesn't work like this in lodash, we need to do _.pickBy for it to accept a predicate.

If so, please change (and good catch). Let's also test this page (I definitely did not, sigh).

MaxBittker · 2017-07-11T20:37:25Z

src/sentry/static/sentry/app/views/projectKeyDetails.jsx

@@ -1,6 +1,6 @@
 import React from 'react';
 import DocumentTitle from 'react-document-title';
-import underscore from 'underscore';
+import underscore from 'lodash';


just commenting '_' on these because i feel obligated now

I would also say it makes it less confusing for new people to know where to google documentation

MaxBittker · 2017-07-12T00:01:56Z

https://github.com/lodash/lodash/wiki/Migrating was super useful 👍

MaxBittker

my concerns are addressed in my commits

MaxBittker · 2017-07-13T23:44:36Z

Do not merge yet. at least comments are broken

MaxBittker · 2017-07-25T01:15:33Z

update: react-mentions relies on some features we're disabling in LodashModuleReplacementPlugin

MaxBittker · 2017-07-25T01:15:45Z

figuring out what those are in order to proceed

MaxBittker · 2017-07-25T21:30:14Z

this is deploying to staging right now for good measure, but I think it's safe to merge. @benvinegar @billyvg

billyvg · 2017-07-25T21:39:02Z

We should probably add an acceptance test for the activity tab, since our suite did not catch these errors

MaxBittker · 2017-07-25T21:45:37Z

you're right billy, also not the first time this page has hidden a regression now that I think. wrote one off the cuff so I'll see what percy shows

benvinegar · 2017-07-25T22:35:55Z

webpack.config.js

+    new LodashModuleReplacementPlugin({
+      collections: true,
+      currying: true,
+      flattening: true


We talked about having a comment about mentioning currying and flattening are enabled to support react-mentions. Can we add that?

* Removes underscore in favor of lodash * Uses lodash-webpack-plugin + babel-lodash to minimize filesize impact * Saves ~100kb unminified between removing underscore (~50kb) and using lodash-webpack-plugin over full lodash dep (another ~50kb) Note: need to make corresponding changes in getsentry repo, since it uses a global alias to sentry's exported `underscore` global.

This reverts commit a9d890f4d8d3a338d22b49f2ead9b8c888277700.

This reverts commit cda5a13f099f3703b91f1d2b750e13b782e6ffdd.

This reverts commit a5c1080eb2f285d3e22cc6e7a954241d2464ceb7.

MaxBittker · 2017-07-26T22:46:59Z

I'm not sure how you were measuring the bundle sizes, webpack -p gave me:

  app.js          946 kB       0  [emitted]  [big]  app
  vendor.js     960 kB      14  [emitted]  [big]  vendor

MaxBittker · 2017-07-26T22:50:58Z

 254K Jul 26 15:42 vendor.js.gz
 217K Jul 26 15:42 app.js.gz

Undoes the work introduced in #5616. This plugin adds around 40-50 seconds to webpack build times and has caused confusion (see #13834). Also adds an eslint rule to prevent importing `lodash` bundle: getsentry/eslint-config-sentry#46 | -- | With Plugin | Without Plugin | Difference | lodash-es | lodash-es + react-mentions | | -- | ----------- | -------------- | ---------- | --------- | - | | app.js | 739051 | 739120 | 0% | | | | app.js.gzip | 184017 | 184040 | 0% | | 186949 | | vendor.js | 2076311 | 2090876 | +0.7% | 2089352 | | vendor.js.gizp | 375718 | 380446 | + 1.2% | 380630 | 380476 | | build time (w/ stats) | 316s | 466s | + 47% | | | build time (env=prod) | 335s | 198s | n/a | | Currently, there is no sizable difference between `lodash-es` and `lodash` -- we should revisit this in the future.

benvinegar requested review from MaxBittker and macqueen June 24, 2017 01:46

benvinegar commented Jun 24, 2017

View reviewed changes

jdalton reviewed Jun 24, 2017

View reviewed changes

dcramer reviewed Jun 24, 2017

View reviewed changes

benvinegar mentioned this pull request Jun 26, 2017

Code split select heavy views (stream, issue details, etc) #5623

Closed

benvinegar force-pushed the lodash-bundler branch from 651ebf8 to ffd134e Compare July 11, 2017 19:40

benvinegar requested a review from billyvg July 11, 2017 20:09

MaxBittker suggested changes Jul 11, 2017

View reviewed changes

MaxBittker approved these changes Jul 12, 2017

View reviewed changes

MaxBittker force-pushed the lodash-bundler branch from 43dde40 to a9d890f Compare July 24, 2017 22:55

MaxBittker force-pushed the lodash-bundler branch from ea7a9a5 to 7273a99 Compare July 25, 2017 21:12

benvinegar commented Jul 25, 2017

View reviewed changes

benvinegar added 3 commits July 26, 2017 13:57

Enable collections in LodashModuleReplacementPlugin

41616f9

Remove shorthands option from LodashModuleReplacementPlugin

1c271e9

benvinegar and others added 15 commits July 26, 2017 13:57

Revert back to $.extend in orgStats view

cd2531a

Fix underscore refs

7f1d4f7

audit lodash compat

4080949

remove lodash from bundle

f120c5a

use omit correctly

9983a3b

wip

3c41e8f

Revert "remove lodash from bundle"

37ee476

This reverts commit a9d890f4d8d3a338d22b49f2ead9b8c888277700.

Revert "use omit correctly"

b1d0948

This reverts commit cda5a13f099f3703b91f1d2b750e13b782e6ffdd.

correct lodash plugin featureset

fe64bd8

Revert "Revert "remove lodash from bundle""

2ffa3d5

This reverts commit a5c1080eb2f285d3e22cc6e7a954241d2464ceb7.

fix a react warning

17bea7d

fix react proptypes warning

7115b89

add acceptance test for activity page

7be78c2

comment to explain react-mentions lodash dep

6eeea5e

update snap

584bb73

MaxBittker force-pushed the lodash-bundler branch from 56d6ceb to 584bb73 Compare July 26, 2017 20:57

MaxBittker merged commit 2ac81ef into master Jul 27, 2017

benvinegar deleted the lodash-bundler branch February 27, 2018 20:01

billyvg mentioned this pull request Nov 11, 2019

build(webpack): Remove lodash-webpack-plugin #15521

Merged

github-actions bot locked and limited conversation to collaborators Dec 22, 2020

		@@ -1,4 +1,4 @@
		import $ from 'jquery';
		import _ from 'lodash';

Switch to lodash + lodash-webpack-plugin ... #5616

Switch to lodash + lodash-webpack-plugin ... #5616

Conversation

benvinegar commented Jun 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdalton commented Jun 24, 2017

benvinegar commented Jun 24, 2017 • edited Loading

jdalton commented Jun 24, 2017 • edited Loading

benvinegar commented Jun 24, 2017

jdalton commented Jun 24, 2017 • edited Loading

benvinegar commented Jun 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaxBittker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaxBittker commented Jul 12, 2017

MaxBittker left a comment

Choose a reason for hiding this comment

MaxBittker commented Jul 13, 2017

MaxBittker commented Jul 25, 2017

MaxBittker commented Jul 25, 2017

MaxBittker commented Jul 25, 2017

billyvg commented Jul 25, 2017

MaxBittker commented Jul 25, 2017

Choose a reason for hiding this comment

MaxBittker commented Jul 26, 2017

MaxBittker commented Jul 26, 2017

benvinegar commented Jun 24, 2017 •

edited

Loading

benvinegar commented Jun 24, 2017 •

edited

Loading

jdalton commented Jun 24, 2017 •

edited

Loading

jdalton commented Jun 24, 2017 •

edited

Loading

benvinegar commented Jun 24, 2017 •

edited

Loading