Search Locking at 10K records

[oppman29]

Hi David,

Unfortunately found an issues not as easy as the scaling to fix.. I would love to give back again!
But so far everything I've tried hasn't worked.

Here is the issue if you load up 10,000 or more rows in a columns where they are vary similar.
For example all begin with 1012XXXX and say 1013XXXX

After the they are displayed when the user goes to search and wants to type a number like 10121212 as soon as they type the first 1,, it tries to load and check all 10K nodes and this happens again after they type the next number and so on..

I have been trying ( without success) to limit the amount of values that are displayed back in the FilterList kinda like what excel does,,,
Excel only shows the first 1000 and shows a too many message ( but as continue to type it adds more matches)
or if not put some kind of delay so the end user has enough time to type the complete number.

My first idea was to limited the number of vals from menustrip GetValuesForFilter passed to BuildNodes(vals); but that came to byte me later because it prevents the number typed being found .

also try to modify the Menustrip.Show by limiting the TreeNodeItemSelector in the foreach but no go..

let me know if you fully understand the issue or i can explain further,,,
It happens with any amount of records but its not noticeable until you hit at least 8- 10K rows,,,

[davidegironi]

Hello @oppman29 , that's about performance. I've already get a question on that. Maybe in the weekend I can take a view on it. I'll get you back.

[oppman29]

Thanks David, what I did in the mean time and work but it's not elegant at alll... On method buildnodes after you add the empty node, I create a new list with the distinct values from vals and if the count > 1000 I clear all the nodes and just create a node that says please use custom filter and disable checkTextfilter and checkFilterListControlHost and button_filter....

…

________________________________ From: Davide Gironi ***@***.***> Sent: Wednesday, March 17, 2021 3:11:39 PM To: davidegironi/advanceddatagridview ***@***.***> Cc: oppman29 ***@***.***>; Mention ***@***.***> Subject: Re: [davidegironi/advanceddatagridview] Search Locking at 10K records (#58) Hello @oppman29<https://github.com/oppman29> , that's about performance. I've already get a question on that. Maybe in the weekend I can take a view on it. I'll get you back. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#58 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIFCPILTWXRI2HC75OR2PFLTED5GXANCNFSM4ZHYHUZA>.

[davidegironi]

Hello @oppman29,
Check this out: https://filebin.net/791cp6y1chcdikwe
MenuStrip.cs is changed, you can find the changes on top. I have to expose the changes to parent components, comment it out and make a little bit of order, but now, just for a check, you can set _limitCheckListNodes. Let me know if it's good for you.

[oppman29]

Thanks for the update David. I noticed the following behaviors. 1-Create a column that just contains numbers 0-10000 ...when the filter pops up unselect 3 values( I did 0,1,2) ... The count will be 998... So far so good... Now go back to the same column and unselect a few more values ( 3, 4) .. Now the count is completely off with 94 rows only. Looking at the filter string the first time has all 998 values the second time only has the subset of the first 100. Before you had all the values so it would show the 996. 2- same setup 0,10000 if you type 998 in the search box, nothing comes up.. Probably because there is only 0-99 that exist in the filter with check boxes. You have to use custom filter to find it. Thank you again for taking the time to update the code. I will report back if I find something else.

…

________________________________ From: Davide Gironi ***@***.***> Sent: Sunday, March 21, 2021 10:52:18 AM To: davidegironi/advanceddatagridview ***@***.***> Cc: oppman29 ***@***.***>; Mention ***@***.***> Subject: Re: [davidegironi/advanceddatagridview] Search Locking at 10K records (#58) Hello @oppman29<https://github.com/oppman29>, Check this out: https://filebin.net/791cp6y1chcdikwe MenuStrip.cs is changed, you can find the changes on top. I have to expose the changes to parent components, comment it out and make a little bit of order, but now, just for a check, you can set _limitCheckListNodes. Let me know if it's good for you. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#58 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIFCPIP67PKNMWRXIG5TELTTEYB2FANCNFSM4ZHYHUZA>.

[davidegironi]

Hello @oppman29 ,
I know the behaviour of point 2. I'm trying to solve this, but it's a big change in the main logic of the MenuStrip. I suppose I've to refactor the full logic, cause it's all based on nodes of CheckList now. Unluckily I've not enought time now to rebuild this.
I post here the latest MenuStrip.cs I've write that has a few bug expecially when rebuilding the filter. As soon as I've time, I will take a look at this.
MenuStrip_lotdata.zip

[davidegironi]

Hello @oppman29
Take a look here, logic is rewritten. I need your help to check ALL the filtering and sorting capabilities before publishing.
Note: this is a pre/publish version that already contains the SDK (net5.0 and net40 multitarget) mod #49

Download here: https://file.io/tN7wEwMRZ2Gc

[davidegironi]

Hello @oppman29
Just to inform you that I'm going to release in a few days a new version with the fix above, and the memory leak cured find it here: #49 (comment), let me know if you find problems)