Navigation through directory with large number of files continuously slows down as you advance through listing

[salacpavel]

With large number of files in a single directory, auto scrolling in files becomes continuosly slower and slower, as you advance through files listing from beginning to the end.
Steps to reproduce:
Create a folder on your USB flashdrive and put some 500+ floppy image files into it. Plug it to Gotek and navigate to that directory. Press and hold "right" button, so that Gotek starts srolling filenames on display at full speed. After a while, you will notice that auto scrolling speed drops down. It becomes very apparent after some 300 files.
If you (like me in this case) have a single directory with all 5000 games sorted by name from A to Z and you somehow manage to navigate to files at the end of listing, Gotek becomes so irresponsive, that it takes him about 10 seconds just to advance in the listing by one.

[salacpavel]

On a sidenote: while it is certainly not very wise to have such a large number of files in a single directory, it indicates there is some issue in a firmware code.

[keirf]

This can no doubt be improved but it's not a priority. For now (and probably in any case makes it easier to navigate 1000s of files) is to use a tool to auto-arrange into smaller subfolder. Like, for example: https://github.com/discoHR/auto-organize-files-recursively

I will leave this issue open to track any progress or other ideas.

Actually to be fair the slowing down as you progress is definitely fixable... post v1.0 ;)

[keirf]

Okay here is a firmware which implements a buffer cache at the mass-storage layer. It speeds up navigation a lot. Could be faster still, as navigating to item N in a folder still always iterates from item 1 upwards.

But perhaps this is fast enough up to say 2-300 items in a folder?
And does anyone really want more items than that in a folder?

I am interested in feedback on these questions, and people's results from testing this firmware.
Thanks!

ff_60_1.zip

[keirf]

These improvements and others are rolled into release v2.5a. Please test!

I have some other improvements for image navigation speed waiting in the wings. They are more invasive, and need more testing, so will appear in a future release. They should close out this ticket entirely.

[keirf]

Okay closing this now. New default behaviour is to sort full folders in RAM, hence they are fast to navigate (except for small latency to do the load into RAM). Large folders get truncated by default. If large folders are a must-have and must work faster, that needs more work and a new ticket. However new behaviour should be good for ~500 items per folder.

[keirf]

This is new as of v2.7a by the way.