Thoughts about a faster "fasder"

This is an idea for improving the performance of fasder even more. It addresses the increasing latency when your .fasd data files grow pretty large.

The prompt hook that runs with every shell command invokes the mapfile builtin to reload the fasder data from the file, set up an array, and run the data through the populate...() function every time. This is an almost-intractable performance hit. (Timing experiments indicate the first two steps are the real issue.) One possible alternative is to cache the data in shell memory, thus persisting it across separate calls to fasder so we need only access the data files occasionally.

We would store a copy of the data in an environment variable that we would populate once during shell initialization, by adding a command to the "fasder --init" handler that would load the data the long, hard way. After that, every time the user enters a command, the prompt hook would scan the data from the shell environment, manipulate it, and perform the flush-to-file only once every so often.

Depending on usage patterns, the fasder data can grow large enough that storing it in an environment variable might present some systems issues. See for example this discussion and also this one. Fortunately the data does not grow quickly, but if issues do crop up in the long run we see several possible means for the user to address this:

• Manually identify unwanted entries and "fasder --delete" them
• Run a "fasder --clean" action which would remove entries referring to non-existent or duplicate objects
• Reduce the configured value of FASD_MAX so that old entries will fall off the radar more quickly

Hence the overall idea seems feasible. OTOH, fasder is fast enough as-is for my taste, so I regard this as simply a nice-to-have.

UPDATE

---

This idea cannot be implemented exactly as stated since we cannot alter the shell environment from within a running script. However, it is feasible to update the script-local copy of the environment with the cumulative changes to the fasder data every time the script runs. We'd simply need to load the changes every time (probably with a call to mapfile), apply them to the "constant" data and save them again after we've processed the most recent shell command.

One argument in favor of this is that the changes will usually be much smaller than the "constant" data whenever there's enough of the latter to even worry about it. The new call to mapfile which replaces the older one would face a much smaller burden, hence a time savings is likely. However, this is offset by the need for an extra file write ach time in order to re-save the cumulative changes. I don't think the effort required is justified.