Implement "fuzzy filtering"

[djc]

I touched the config just now for the first time because I was trying to find a project-recent command. I ended up enabling workspaces = true because that seems closest to what I want. However, I was kind of expecting atuin to do something like this by default, and also to be more flexible? Like, it seems like a cool default would be to rank commands using some score that combines all of the filter modes (host, directory/workspace, session). Like commands from the same host might get a small boost, commands from the directory get a bigger boost, commands from the same session get a decent boost and recency gets some logarithmic decaying boost. Feels more flexible and potentially a good default compared to having to select a filter mode by hand.

[ellie]

This is pretty much what I'd like to achieve with #1455! Thanks for raising

[djc]

Will follow along there!

[ellie]

oh, and thank you for your open source contributions! much appreciated 😊

[djc]

Thanks!

Looking at #1455, it seems to encompass much more than this, to the point that it might make sense to address this first? Without knowing anything about the interenals, it feels like making ranking better without the user having to learn/type in a query language, and avoiding making the indexing story more complex would be (relatively) low-hanging fruit.

[ellie]

without the user having to learn/type in a query language

There wouldn't be a query language to learn for normal use. Only when you wish to be more specific than you can with any search modes we have right now.

Ranking that is based upon a whole bunch of columns would really require one of three things

1. A big complicated ball of sql
2. Sorting results in Rust
3. A real search index

(1) is something I'd like to stay very far away from, as it does not tend to perform well or be that maintainable. Our fuzzy query is already far more complex than I'd like. Queries need to complete in a couple of ms over 100k+ rows, on modest hardware. I've explored the various FTS virtual tables available in SQLite, but they're not as flexible as I'd like. Really, we'd need a custom extension for ranking.

(2) is maybe something we could do, but would be pretty limited as we only have access to a small number of results from the database

(3) is where I'd like to go, and makes this sort of sorting very easy. There is upfront effort in integrating a search index, but it will make this sort of scoring, highlighting, filtering searches, etc really easy to do. I'd rather do that than implement all of the features of a search index in a hacky way on top of SQL

The closest thing we have right now is the skim search mode. I refer to this indirectly in #1455 (it loads all history into memory). You may wish to try it though, sounds like it's closer to what you want

[djc]

Fair, thanks for the additional context!

[atuin-bot]

This issue has been mentioned on Atuin Community. There might be relevant details there:

https://forum.atuin.sh/t/a-regex-search-mode/169/14