optimise fs reads

[jackkav]

changelog(Improvements): Added some optimizations to filesystem read frequency in order to improve Insomnia's performance

motivation: sidebar rendering speed is very slow since moving app state from redux to remix, this is due to the lifespan changing dramatically, previously the redux state was a copy of the db created at app start and listened for all changes, potentially causing rerenders after a db write where useSelector was used but reads were always to the redux in memory store. Currently the read are always to the database and happen at remix loaders which run at app start and also on every action. This means the current lifespan is very short so forces re-computation.

ideas:

• optimise filesystems reads so they are batched together and not happening in a recursive loop between computations.
• recreate the client side state cache synced with the database ideally without redux.

highlight

• only recursively access folders from fs
• other fs reads happen inline and not during recursion which saves on hdd read frequency

todo:

• remove timers

recording is taken after typing in the body causing a remix action and the loader to reload
basline with 100 requests and one folder

after batching fs reads rather than recursing

future work

• if the tree is built in main then it could offer gains by making far fewer ipc roundtrips
• moving filter out of the loader and into the render component will make filter instant

notes:

• there is also a perf issue in the sidebar gridlist component blocking layout effects