Skip to content

Levenshtein distance score with custom score function (POC for #998) - #1244

Draft
Restioson wants to merge 8 commits into
quickwit-oss:mainfrom
Restioson:fst-state
Draft

Restioson wants to merge 8 commits into
quickwit-oss:mainfrom
Restioson:fst-state

Conversation

@Restioson

Copy link
Copy Markdown

Hi, I've tried to implement the ability to use a custom score function (Fn(u8) -> f32) for fuzzy queries, which take the DFA distance as the input and return the score for the term. This is a proof of concept for #998, and I am opening this as a draft so that people can more easily see the diff and comment on it. If the solution is good, it could perhaps be merged into @lerouxrgd's branch first, and then #998 could be merged, or whatever people see fit.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❗ No coverage uploaded for pull request base (main@8e7fe06). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1244   +/-   ##
=======================================
  Coverage        ?   94.10%           
=======================================
  Files           ?      206           
  Lines           ?    35072           
  Branches        ?        0           
=======================================
  Hits            ?    33003           
  Misses          ?     2069           
  Partials        ?        0           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e7fe06...47f8723. Read the comment docs.

@Restioson

Copy link
Copy Markdown
Author

I'll run rustfmt on this in a moment


/// A weight struct for Fuzzy Term and Regex Queries
pub struct AutomatonWeight<A> {
pub struct AutomatonWeight<A, F> {

@fulmicoton fulmicoton Jan 3, 2022 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach is not good...

The automaton weight is for regex and fuzzy queries, as mentionned above.
F has no meaning for regex queries.

Let's try something nicer.
We can have F match F: Fn(A::State) -> Score .

For better usability, we can also offer a default to the Const function that returns one.

The tricky part then, is that we want the user to pass an array over the distance. Not a function of the state.
Well, that's a job for the FuzzyQuery itself.

The FuzzyQuery has access to the automaton. It can therefore build a function that picks a state, computes the distance, and return a score.

@Restioson Restioson Jan 3, 2022 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F has no meaning for regex queries.

In this case, it can default to some no-op function like |_| 1.0

We can have F match F: Fn(A::State) -> Score .

I don't really understand this because there would still be an F for a regex query then, wouldn't there?

For better usability, we can also offer a default to the Const function that returns one.

If F is a Fn then we cannot provide struct AutomatonWeight<A, F = <some default>> because we can't name |_| 1.0, nor implement it for a struct DefaultScorer; because that's unstable. Otherwise, a scorer trait could be added, which can then have a named default implementation, and be blanket implemented for Fn(A::State) -> Score.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also don't quite understand what you mean by "we want the user to pass an array over the distance, not a function of the state", if you are also suggesting Fn(A::State) -> Score. Apologies if I am misinterpreting this

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand this because there would still be an F for a regex query then, wouldn't there?

Ah yes... :)
But there is one big notable difference.

The role of F can then be explained without necessary talking about the fuzzy term query.

 because we can't name |_| 1.0

nor implement it for a struct DefaultScorer; because that's unstable

Which feature is unstable exactly?

I also don't quite understand what you mean by "we want the user to pass an array over the distance, not a function of the state", if you are also suggesting Fn(A::State) -> Score. Apologies if I am misinterpreting this

We can keep the function.
The user would give the FuzzyTermQuery an array over the distance rather than a function. It's tiny anyway.

user gives [f32; max_distance + 1]
FuzzyTermQuery takes that an transforms into a Fn(u32) -> f32,
The implementation take a state, uses the automaton to get a distance, and do a lookup in the user array.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which feature is unstable exactly?}

Implementing a Fn trait is unstable: rust-lang/rust#29625. This can be mitigated by making trait AutomatonScorer { fn score(A::State) -> f32 } or similar.

The user would give the FuzzyTermQuery an array over the distance rather than a function. It's tiny anyway.

So, in FuzzyTermQuery the user provides a function, and then at the AutomatonWeight boundary it is converted to a suitable F?

@fulmicoton fulmicoton left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comment inline.

@Restioson

Restioson commented Jan 3, 2022 •

Copy link
Copy Markdown
Author

Another question: is there any way to factor in the length of the suffix of the term (when using the query as the prefix) to the score? If possible it seems like something that should be exposed.

A::State: Clone,
F: Fn(u8) -> f32,
{
if TypeId::of::<DfaWrapper>() == automaton.type_id() && TypeId::of::<u32>() == state.type_id() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this it the code we want to get rid of.

@fulmicoton

fulmicoton commented Jan 3, 2022 •

Copy link
Copy Markdown
Collaborator

Another question: is there any way to factor in the length of the suffix of the term (when using the query as the prefix) to the score? If possible it seems like something that should be exposed.

Great question...
Factoring in the length of the overall term should be relatively easy.

Factoring in the length of the suffix is harder, but we can probably cook something if we dig deep in the levenshtein automata crate. I suggest we use the length - querylength as a first approximation. That should probably do the job.

One case where that will give us different result is

-A: target: elephant, query: eleh
vs
-B: target: elephant, query: elepj

In this case, B will get a better score. But A could be the deletion of a character bringing us to the actual same suffix size, same levenshtein distance.

@Restioson

Copy link
Copy Markdown
Author

I suggest we use the length - querylength as a first approximation. That should probably do the job.

This sounds good to me. How can we expose this to the user in such a way that they can factor it in?

@fulmicoton

Copy link
Copy Markdown
Collaborator

@Restioson a funciton is good I think.

Fn(lev_distance: u8, approximate_suffix_length: u32) -> f32

We can cache the result for
lev_distance = 0,1,2,3,>=3
and
approximate_suffix_length = 0..10 or something like that.

@Restioson

Copy link
Copy Markdown
Author

As far as caching goes, would it maybe be cleaner to let the user do their own caching? They could always wrap it with a memoisation function if they wanted to, perhaps? I'm just wondering if caching might be extraneous in a situation of just reading off a predefined const value table.

@ChillFish8

Copy link
Copy Markdown
Collaborator

As far as caching goes, would it maybe be cleaner to let the user do their own caching? They could always wrap it with a memoisation function if they wanted to, perhaps? I'm just wondering if caching might be extraneous in a situation of just reading off a predefined const value table.

I agree that caching should be left to the user, I imagine in most cases anyway the calculations are going to be pretty simple and easily optimised by the compiler.

@fulmicoton
fulmicoton force-pushed the main branch 2 times, most recently from e550a98 to 84e0c75 Compare September 2, 2022 03:01

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants