Skip to content
This repository has been archived by the owner on Apr 4, 2023. It is now read-only.

Matching query terms policy #598

Merged
merged 5 commits into from
Aug 22, 2022
Merged

Matching query terms policy #598

merged 5 commits into from
Aug 22, 2022

Conversation

ManyTheFish
Copy link
Member

@ManyTheFish ManyTheFish commented Jul 21, 2022

Summary

Implement several optional words strategy.

Content

Replace optional_words boolean with an enum containing several term matching strategies:

pub enum TermsMatchingStrategy {
    // remove last word first
    Last,
    // remove first word first
    First,
    // remove more frequent word first
    Frequency,
    // remove smallest word first
    Size,
    // only one of the word is mandatory
    Any,
    // all words are mandatory
    All,
}

All strategies implemented during the prototype are kept, but only Last and All will be published by Meilisearch in the v0.29.0 release.

Related

spec: meilisearch/specifications#173
prototype discussion: meilisearch/meilisearch#2639 (reply in thread)

@ManyTheFish ManyTheFish force-pushed the matching-query-terms-policy branch 6 times, most recently from fc1da78 to 019cd1e Compare July 25, 2022 13:50
@ManyTheFish ManyTheFish added the API breaking The related changes break the milli API label Aug 22, 2022
@ManyTheFish ManyTheFish changed the title Prototype: Matching query terms policy Matching query terms policy Aug 22, 2022
@ManyTheFish ManyTheFish marked this pull request as ready for review August 22, 2022 12:56
Copy link
Member

@Kerollmops Kerollmops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! (please bors)
bors merge

@bors
Copy link
Contributor

bors bot commented Aug 22, 2022

@bors bors bot merged commit 18886dc into main Aug 22, 2022
@bors bors bot deleted the matching-query-terms-policy branch August 22, 2022 16:06
bors bot added a commit to meilisearch/meilisearch that referenced this pull request Aug 23, 2022
2636: Upgrade milli to v0.33.0 r=Kerollmops a=ManyTheFish

# Summary
- Update milli to v0.33.0
- Classify the new InvalidLmdbOpenOptions error as an Internal error
- Update filter error check in tests
- Introduce Terms Matching Policies

fixes #2479
fixes #2484
fixes #2486
fixes #2516
fixes #2578
fixes #2580
fixes #2583
fixes #2600
fixes #2640
fixes #2672
fixes #2679
fixes #2686

# Terms Matching Policies
This PR allows end users to customize matching term policies

## Todo

- [x] Update the API to return the number of pages and allow users to directly choose a page instead of computing an offset
- [x] Change generation of the query tree depending on the chosen settings meilisearch/milli#598

## Small Documentation

### Default search query

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "doctor of tokio" }'
```

**result**:
```json
{
  "hits":[...],
  "estimatedTotalHits":32,
  "query":"doctor of tokio",
  "limit":20,
  "offset":0,
  "processingTimeMs":7
}
```

The default behavior doesn't change with the current Meilisearch behavior:
If we don't have enough documents to fit the requested limit, we remove the query words from the last to the first typed word.

## Search query with `optionalWords` parameter

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "doctor of tokio", "matchingStrategy": "all"}'
```

**result**:
```json
{
  "hits":[...],
  "estimatedTotalHits":1,
  "query":"doctor of tokio",
  "limit":20,
  "offset":0,
  "processingTimeMs":7
}
```

### allowed `matchingStrategy` values

#### `last`
The default behavior, If we don't have enough documents to fit the requested limit, we remove the query words from the last to the first typed word.

#### `all`
No word will be removed, If we don't have enough documents to fit the requested limit, we return the number of documents we found.

### In charge of the feature

Core: `@ManyTheFish` & `@curquiza`  
Docs: TBD
Integration: `@bidoubiwa` 


Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Many the fish <many@meilisearch.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
API breaking The related changes break the milli API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants