Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(dependencies): stopping (and preventing) full lodash library import... now using only method level imports. #26710

Merged
merged 8 commits into from
Jan 23, 2024

Conversation

rusackas
Copy link
Member

SUMMARY

There were a few spots where the entire lodash library was being loaded, bloating our bundles for no good reason. Now there's an eslint rule to prevent this, and the code has been adjusted to use/allow only method level imports.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

Copy link

codecov bot commented Jan 22, 2024

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (5eb4e82) 69.48% compared to head (adf9cef) 69.48%.
Report is 2 commits behind head on master.

Files Patch % Lines
...gins/plugin-chart-echarts/src/utils/treeBuilder.ts 66.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##           master   #26710   +/-   ##
=======================================
  Coverage   69.48%   69.48%           
=======================================
  Files        1894     1894           
  Lines       74151    74151           
  Branches     8243     8243           
=======================================
  Hits        51527    51527           
  Misses      20555    20555           
  Partials     2069     2069           
Flag Coverage Δ
hive 53.80% <ø> (ø)
mysql 77.98% <ø> (ø)
postgres 78.08% <ø> (ø)
presto 53.75% <ø> (ø)
python 83.02% <ø> (ø)
sqlite 77.66% <ø> (ø)
unit 56.39% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kgabryje
Copy link
Member

According to lodash docs, babel-plugin-lodash (which we are using) makes imports like import { x } from "lodash" tree-shakable just like import x from lodash/x. Do you know which module appears to import the whole lodash without tree shaking?

@kgabryje
Copy link
Member

I see that some files are using import _ from 'lodash'; which is definitely not treeshakable. Maybe it'd be enough to fix just those few cases?

@michael-s-molina
Copy link
Member

+1 to @kgabryje's comments.

@rusackas
Copy link
Member Author

It's always a slippery slope...

Indeed, there were a few places importing _ which loads the whole darn thing. I fixed those, and this was a nice, small PR.

Then, I thought "gee, we should prevent this from happening, is there a lint rule?" and there is! So installed it, and found a catch... while you can block the full import, it forces consistency between module and method imports... it doesn't let you mix and match.

I chose method because (a) it was a simpler, smaller migration, and (b) this produces a reliably small bundle, even in an environment where we might not trust our tree-shaking fully (and I'm not sure if I do).

@michael-s-molina
Copy link
Member

@rusackas Is it possible to replace the occurrences of _ with a named import { map } and use member instead of method?

@rusackas
Copy link
Member Author

Yes, it's possible... we already have 36 method-style imports in Superset, so we have to force consistency one way or the other. Method is the default/preference of the lodash community, and is faster to compile since no treeshaking is required, and provides no opportunity for surprises.

Replacing the instances of _ is trivial either way, but going with method seems to have more going for it, other than taking up multiple lines. if everyone wants module, I'll go the other direction, but the PR will still be a big one.

@michael-s-molina
Copy link
Member

I think the main reason for using named imports/exports is because is the default syntax in Superset.

@kgabryje
Copy link
Member

@rusackas Is there any chance that that eslint plugin could flag only default imports from "lodash", but not default imports from "lodash/feature_name" and named imports from "lodash"? I think that would be perfect for us

@rusackas
Copy link
Member Author

Nope, it doesn't support that. I can revise this PR to do module imports if this pattern bothers people, or close this one and open a new PR... whichever is easier. It's not a priority, so that would happen on some other rainy day (I opened this on a Sunday night on a whim)

@michael-s-molina
Copy link
Member

michael-s-molina commented Jan 22, 2024

I can revise this PR to do module imports if this pattern bothers people

Do you mean member imports? The reason I'm asking is because we don't need the whole module.

@kgabryje
Copy link
Member

Nope, it doesn't support that. I can revise this PR to do module imports if this pattern bothers people, or close this one and open a new PR... whichever is easier. It's not a priority, so that would happen on some other rainy day (I opened this on a Sunday night on a whim)

Oh that's a shame... In such case, I think it's more important to prevent default imports from lodash than to stick to named imports style - especially since if we let just 1 bad import slip past code reviews, it will include the whole library in our bundle.

@pull-request-size pull-request-size bot added size/S and removed size/L labels Jan 22, 2024
@pull-request-size pull-request-size bot added size/M and removed size/S labels Jan 22, 2024
@rusackas
Copy link
Member Author

OK, everything is now converted to member rather than method :) Let's hope it all builds correctly now, and our bundles get smaller!

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NICE! 👏 I had no idea we had these still lingering around. LGTM! 👍

Copy link
Member

@michael-s-molina michael-s-molina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for working on reducing bundle size!

@@ -20,7 +20,14 @@
/* eslint-disable sort-keys */

const Generator = require('yeoman-generator');
const _ = require('lodash');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we do const { kebabCase, startCase, camelCase, upperFirst } = require('lodash'); instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just saw this after merging..... but... yes I suppose we can. I'll open a smaller/less contentious PR on some rainy day

@rusackas rusackas merged commit 1d4b8b6 into master Jan 23, 2024
53 checks passed
rusackas added a commit that referenced this pull request Jan 24, 2024
eschutho pushed a commit to preset-io/superset that referenced this pull request Jan 31, 2024
sfirke pushed a commit to sfirke/superset that referenced this pull request Mar 22, 2024
@rusackas rusackas deleted the no-full-lodash branch April 16, 2024 16:51
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 4.0.0 labels Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/M 🚢 4.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants