[v7] Accessor derivation slow with 100k+ rows

[gargroh]

When i provide react-table v7 with good amount of rows(below case is with 100K rows and 15 columns, then getBy, makePathArray are taking long time to execute.

Is there anything we can do to optimize it ?
I can see before calling getBy we are splitting by ., then in makePathArray we are doing map(d => String(d).replace('.', '_')), this thing seems to be redundant(or can be applicable at some other place also), is this good to remove ?

[tannerlinsley]

I think that needs to stay, but we can definitely start by extracting out the regex expressions to variables so they aren’t created for every row. Tanner Linsley

…

On Dec 5, 2019, 6:39 AM -0500, gargroh ***@***.***>, wrote: When i provide react-table v7 with good amount of rows(below case is with 100K rows and 15 columns, then getBy, makePathArray are taking long time to execute. Is there anything we can do to optimize it ? I can see before calling getBy we are splitting by ., then in makePathArray we are doing map(d => String(d).replace('.', '_')) is this good to remove ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[Asuskf]

'react-table' does not contain a default export (imported as 'ReactTable').

[tannerlinsley]

@Asuskf, please update to the latest version of react-table@latest and try again. If the issue persist, please open a new issue with more information, thanks.

[tannerlinsley]

I just pushed a new RC with the regex's pulled out into vars. Let's see if that speeds it up at all?

[gargroh]

Ok, will check and get on this.

[gargroh]

Following is the time taken (its weird) for 100K records stats are -

Before(using inline regex) -
getAccessedRows: 10793.928955078125ms
getAccessedRows: 10435.14306640625ms

After(using stored regex) -
getAccessedRows: 11396.470947265625ms
getAccessedRows: 11333.337890625ms

Also we can see the same at https://jsperf.com/regexp-indexof-perf/24, it can be improve by ~10 to 15%.

How about using memoization, as accessor will be same throughout column, so instead of calculating for every cell, can it be calculated once for each column and use it for cells ?

[tannerlinsley]

In the latest version, the table now caches the pathObjects used in column accessors via the getBy function. I would love to see if and how this improves your speeds.

Another piece to note is that the debug and timing code was removed in the latest version. You will need to edit the source to get those exact timing logs back. Here's the location of those loggers for reference: https://github.com/tannerlinsley/react-table/blob/176986a07ad8b25907fff435e1246962dba18e47/src/hooks/useTable.js#L174-L231

[tannerlinsley]

Any updates here?

[gargroh]

Checking brb

[gargroh]

Woah, that improved performance of getAccessedRows by ~5x, time taken for 100K rows :-

Before -
getAccessedRows: 9396.649169921875ms
getAccessedRows: 10379.017333984375ms

After -
getAccessedRows: 2384.264892578125ms
getAccessedRows: 2155.550048828125ms

PS. - I am using 7.0.0-rc.1 version and only replaced old getBy method with new one(present in master)

[tannerlinsley]

Wow that’s awesome! Tanner Linsley

…

On Dec 8, 2019, 11:32 PM -0700, gargroh ***@***.***>, wrote: Woah, that improved performance of getAccessedRows by ~5x, time taken for 100K rows :- Before - getAccessedRows: 9396.649169921875ms getAccessedRows: 10379.017333984375ms After - getAccessedRows: 2384.264892578125ms getAccessedRows: 2155.550048828125ms PS. - I am using 7.0.0-rc.1 version and only replaced old getBy method with new one(present in master) — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[tannerlinsley]

I would say this is a big win out of the gate. I would still like to see some more profiling to see what else we could optimize, but I'd like to close this issue for now. If you're still not satisfied with the speed, feel free to comment with more info and we can open it back up. :)