Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Unnecessary queries when changing filter values #16994

Conversation

michael-s-molina
Copy link
Member

@michael-s-molina michael-s-molina commented Oct 6, 2021

SUMMARY

The native filters were firing unnecessary queries when their values changed. This was happening because an useEffect hook uses the formData as a dependency and the url_params property changes every time a value changes. To avoid that, I removed the url_params from the comparison but we should definitely improve our useEffect dependencies to use more granular information instead of big objects that require deep comparison. Added a TODO in the code for future refactoring.

I also changed the dependency in the calculation of the cascade filters tree to avoid unnecessary runs.

@rusackas @villebro @graceguo-supercat @junlincc

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Screen.Recording.2021-10-06.at.3.43.23.PM.mov
Screen.Recording.2021-10-06.at.3.29.56.PM.mov

TESTING INSTRUCTIONS

Check the videos for instructions.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@codecov
Copy link

codecov bot commented Oct 6, 2021

Codecov Report

Merging #16994 (70df9ab) into master (c577191) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #16994      +/-   ##
==========================================
- Coverage   76.92%   76.92%   -0.01%     
==========================================
  Files        1030     1030              
  Lines       55022    55025       +3     
  Branches     7465     7466       +1     
==========================================
+ Hits        42328    42329       +1     
- Misses      12440    12442       +2     
  Partials      254      254              
Flag Coverage Δ
javascript 70.90% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ilters/FilterBar/FilterControls/FilterControls.tsx 80.00% <ø> (ø)
...veFilters/FilterBar/FilterControls/FilterValue.tsx 68.51% <100.00%> (+0.29%) ⬆️
...frontend/src/SqlLab/components/ResultSet/index.tsx 61.48% <0.00%> (-0.46%) ⬇️
...set-frontend/src/views/CRUD/welcome/ChartTable.tsx 73.17% <0.00%> (ø)
...frontend/src/views/CRUD/welcome/DashboardTable.tsx 64.89% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c577191...70df9ab. Read the comment docs.

@@ -63,7 +63,8 @@ const FilterControls: FC<FilterControlsProps> = ({
dataMask: dataMaskSelected[filter.id],
}));
return buildCascadeFiltersTree(filtersWithValue);
}, [filterValues, dataMaskSelected]);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [JSON.stringify(filterValues), dataMaskSelected]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we wrap filterValues in line 51 in useMemo instead? Like

const filterValues = useMemo(() => Object.values<Filter>(filters), [filters]);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we wrap filterValues in line 51 in useMemo instead? Like

const filterValues = useMemo(() => Object.values<Filter>(filters), [filters]);

That's a good idea - however, won't we need to use stringify filter in the deps array? Seems like the object reference may change without the contents actually changing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, now I see that useFilters hooks returns Object.entries which creates a new object on each run... Yeah, that means that it probably needs JSON.stringify, but in general I'm not sure if the pattern used in useFilters is good

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could utilise a comparison function in useSelector (in useFilters hook)? https://react-redux.js.org/api/hooks#useselector

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, that might as well be a part of some refactor PR. I'm fine with using JSON.stringify here as getting rid of those redundant queries has much higher priority 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, now I see that useFilters hooks returns Object.entries which creates a new object on each run... Yeah, that means that it probably needs JSON.stringify, but in general I'm not sure if the pattern used in useFilters is good

I agree, it's definitely an antipattern. However, until we have a guideline for avoiding them I'd suggest going with JSON.stringify for now, leaving a // TODO: here and then cleaning these stringifys up in follow-up refactor PRs once we've settled on a good pattern for being able to use strict equality in the deps arrays.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. Let's use JSON.stringify for now and plan to deal with all occurrences of it in a specific refactor effort. Meanwhile, we should avoid using objects that require deep comparison as useEffect dependencies.

if (
!isRefreshing &&
(!areObjectsEqual(formData, newFormData) ||
!areObjectsEqual(ownState, filterOwnState) ||
(!isEqualWith(formData, newFormData, customizer) ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could enhance areObjectsEqual to accept customizer function?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could enhance areObjectsEqual to accept customizer function?

Agreed, let's do that by adding customizer to opts in areObjectsEqual.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I agree. This is the implementation of areObjectsEqual:

export function areObjectsEqual(
  obj1: any,
  obj2: any,
  opts = { ignoreUndefined: false },
) {
  let comp1 = obj1;
  let comp2 = obj2;
  if (opts.ignoreUndefined) {
    comp1 = omitBy(obj1, isUndefined);
    comp2 = omitBy(obj2, isUndefined);
  }
  return isEqual(comp1, comp2);
}

The first thing is that is very inefficient because it creates new objects (omitBy) only for the purpose of removing the properties that are undefined. The second thing is that this seems to be a customizer implementation and not a new version of isEqual. I think the best approach would be to have pre-defined customizers to use with Lodash isEqualWith. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point of areObjectsEqual IMO is to serve as a minimum-effort wrapper around isEqual that can be centrally updated if the lodash API were to change (I believe I may have implemented that ignoreUndefined some time ago based on some docs I found). I'm all for making this more efficient - if this can be implemented in a simpler fashion let's do it 👍

@AAfghahi
Copy link
Member

🏷 v1.4

eschutho pushed a commit that referenced this pull request Nov 22, 2021
QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 28, 2021
@mistercrunch mistercrunch added 🍒 1.4.0 🍒 1.4.1 🍒 1.4.2 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/S v1.4 🍒 1.4.0 🍒 1.4.1 🍒 1.4.2 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants