feat(perf): huge filtering execution speed improvements #694
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The problem was that previous code was parsing the search values at the same time as processing the filtering check (on every single row) and that is totally unnecessary, the search value once provided will never change when comparing to each cell. This PR is rewriting the Filter Service to process the search values only once (before looping through all cells). Parsing Dates are the biggest performance killer because we take dates, that could be in any format, and we parse (convert) them to MomentJS object and that is expensive on 50,000 or more rows (see below for perf logs before/after).
Avg Gain on a set of 50k rows
below is a quick test with perf logs performing date filtering before & after code change with 50k rows and 8 columns grid
note: the date are all in ISO format (yyyy-mm-dd), there might be an even bigger perf boost with non-ISO format
The tests were performed on Example 14 and then clicking on "50k rows" button
each tests were made after clearing cache & refresh page
BEFORE
with a single date
searching: 1471.196044921875 ms
searching: 1438.469970703125 ms
searching: 1363.123046875 ms
avg: ~1424.26 ms
with a date range
searching: 2695.779052734375 ms
searching: 2751.52490234375 ms
searching: 2727.10888671875 ms
avg: ~2724.80 ms
AFTER
with a single date
searching: 246.010009765625 ms
searching: 236.799072265625 ms
searching: 217.127197265625 ms
avg: ~233.31 ms
with a date range
searching: 591.408935546875 ms
searching: 695.953857421875 ms
searching: 585.989990234375 ms
avg: ~624.45 ms