#3202 - Large speed increases to SelectBox and SelectFilter2

[kylemacfarlane]

I have an admin page with two filter_horizontal widgets on it. One has almost 800 options and the other 11,500. As you can imagine it's pretty slow but also it sometimes copies the wrong selections if you have a lot of them, plus it crashes on page load in IE and Opera. The scripts in 1.4 make it even slower by using a slow jQuery selector to check if anything is selected when deciding what icons to use.

The tweaks to redisplay from #3202 will cut a second or so off of every action but the code still calls redisplay way too often which means doing anything takes 2-5 seconds. Ticket #9102 attempted to avoid redisplay but its method of hiding <option> tags doesn't work in Chrome.

I've come up with the following which will generally transfer options instantly on my 11,500 list. It starts to slow down if you're moving stuff from side to side in bulk but it's still faster than what's currently in Django. It works excellently in Chrome, Firefox, and Safari, but Opera and IE are still slow but no longer crash.

1. Included the tweak from the current SelectBox.patch on ticket Fixed #23453 -- project_template sys.dont_write_bytecode = True #3202 to speed up redisplay.
2. Don't run redisplay on page load or moving options between columns. Instead just move the actual DOM node and this is super fast.
3. However, when moving large numbers of options then keeping them in alphabetical order is often slower than redisplaying, so redisplay instead. I currently set it to redisplay on any movement of more than 100 options but that number is just a guess and involved little testing. The number really depends on the amount of options being moved, how many options are in the destination column, how fast the user's browser is, etc.
4. Only redisplay filter results 250ms after you've stopped typing. This prevents the major lag on the first 1-3 letters when you try to filter something on medium to large lists.
5. Store a separate key map for quicker access to the cache and less iterating.
6. Check against selectedIndex to see if any options are selected instead of going through every option again with jQuery just to display icons.
7. Every browser except Firefox will lock up for 2-3 minutes if you have too many selected="selected" (even in a plain HTML file). So while you can use this javascript to select 10,000 options in 2 seconds your tab might crash after the form submit.
8. Bonus: fixed the keyboard shortcuts and enter no longer bubbles up and submits the form.

[charettes]

Why are you cloning here? Can't you just append the fragment?

[kylemacfarlane]

1. I made some more speed improvements and it can now do about 4000 single node moves before a full redisplay is faster. Plus redisplays are faster too.
2. Increased the delay on the typing setTimeout because too often it fired early. I then made it so that hitting enter sets the delay to 0 and immediately filters.
3. The add new option popup no longer calls redisplay.
4. I made large option moves maintain your scroll position so that you aren't lost in the middle of nowhere afterwards.

I think there's potential for usability improvements that can be done in relation to tabindex etc but I'm not sure how people would want the form to actually behave.

[kylemacfarlane]

The cache ended up having race issues now that it was working quickly, but I realised that it wasn't even needed anymore anyway.

I also changed the keyboard functionality to something better I believe.

1. When you tab in you will first land on the green plus icon if it's available. Hitting enter will cause the popup and you can add a new object using only the keyboard.
2. The second tab will land in the filter field. In this field you can obviously filter and use enter to to bypass the typing delay. The arrow keys no longer do anything here.
3. The third tab lands you in the left column where you can use the up and down arrows to select options. The wrapping from top to bottom and vice versa was removed because it interfered with the default browser abilities such as using shift to select multiple options. You can use home and end to get to the top and bottom anyway. Once you've selected some options you can move them over with either enter, space, or the left/right arrow keys. In Webkit, Ctrl + A works in here by default but other browsers can use shift + home and end anyway.
4. The fourth tab will land you in the right column which behaves the same.
5. The fifth tab will leave the widget and enter the next field.

[jphalip]

Thanks a lot for your work. Do you think you could reproduce the same performance improvements, but also rewrite the widget with jQuery? If so, then I think it'd be a better approach to kill these two birds with one stone. See this ticket for reference: https://code.djangoproject.com/ticket/15220

[dbrgn]

Fixing this issue would be hugely appreciated :) I'm currently working with a project that displays 13k items in a filter_horizontal widget and the page takes 20-30 seconds to load.

[timgraham]

Closing this in light of @jphalip's comment above.