Introduce the filter syntax for CONTAINS, STARTS and ENDS WITH

[Kerollmops]

This PR introduces three filters. These filters give more control over the facet string values, with can be powerful in certain situations. Keep in mind that there are performance issues with both the CONTAINS and ENDS WITH filters. It is algorithmically impossible to fix the CONTAINS one.

Related to meilisearch/product#544.

Mini Spec

The CONTAINS filter

⚠️ Performance issues can occur when the total number of facet values to filter are high
📝 Can't be improved

• CONTAINS matches fields that EXISTS and contains a specific string inside
• NOT CONTAINS matches fields that NOT EXISTS OR do not contain a specific string

1. {"name":"A", "company": "Airbus 2000"}
2. {"name":"B", "company": "Tesla Inc"}
3. {"name":"C", "company": "Meilisearch"}
4. {"name":"D"}

company CONTAINS 'meilisearch' would match 3
company NOT CONTAINS "meilisearch" or NOT company CONTAINS meilisearch would match 1,2,4
company CONTAINS a would match 1,2,3
company NOT CONTAINS microsoft or NOT company CONTAINS microsoft would match 1,2,3,4

common query : (company CONTAINS meilisearch) OR (company NOT EXISTS) would match 3,4

The STARTS WITH filter

📝 Can be sped up by using the levels/layers structures

• STARTS WITH matches fields that EXISTS and starts with a specific string inside
• NOT STARTS WITH matches fields that NOT EXISTS OR do not starts with a specific string

1. {"name":"A", "company": "Airbus 2000"}
2. {"name":"B", "company": "Tesla Inc"}
3. {"name":"C", "company": "Meilisearch"}
4. {"name":"D"}

company STARTS WITH 'meili' would match 3
company NOT STARTS WITH "meili" or NOT company STARTS WITH meili would match 1,2,4
company STARTS WITH a would match 1,2,3
company NOT STARTS WITH micro or NOT company STARTS WITH micro would match 1,2,3,4

common query : (company STARTS WITH meili) OR (company NOT EXISTS) would match 3,4

The ENDS WITH filter

⚠️ Performance issues can occur when the total number of facet values to filter are high
📝 Can be improved by duplicating the entire database and swap the strings chars

• ENDS WITH matches fields that EXISTS and ends with a specific string inside
• NOT ENDS WITH matches fields that NOT EXISTS OR do not ends with a specific string

1. {"name":"A", "company": "Airbus 2000"}
2. {"name":"B", "company": "Tesla Inc"}
3. {"name":"C", "company": "Meilisearch"}
4. {"name":"D"}

company ENDS WITH 'search' would match 3
company NOT ENDS WITH "search" or NOT company ENDS WITH search would match 1,2,4
company ENDS WITH a would match 1,2,3
company NOT ENDS WITH soft or NOT company ENDS WITH soft would match 1,2,3,4

common query : (company ENDS WITH search) OR (company NOT EXISTS) would match 3,4

Todo List

• Document that it only works on strings
  • Define what happens to the casing/normalization
  • Document performance issues
  • Create a setting to enable it
• Add tests
• Create a prototype and announce it Experimental feature:`CONTAINS` and Prefix / Suffix filter operators product#544
  • Modify the title
  • Create a mini-spec like for Filter on impossible to match values (null, empty) product#539 (comment)

[GelistirmeKF]

This issue is so important for us. When will it be released?

Thanks,
KF

[curquiza]

Hello @GelistirmeKF

Thanks for letting us know. What is your use case exactly? You need this feature, but can you describe more about what you need?
Please feel free to tell us about your use case and the details of it in this related product discussion and not in the PR. Our product team will be able to manage your request 😊

[GelistirmeKF]

Hi @curquiza,

I mentioned it in this meilisearch/product#544 and but didn't get any response.

Thanks in advance,
KF

[2567910]

When will this be available?

[curquiza]

Hello everyone

This feature is not planned yet. We encountered performance issues when we created this prototype
I always recommend referring to the related discussion to talk about your needs and expectations
This PR discussion is for technical details 😊

Thank you in advance

[james2001]

Hi

Any news about this feature?

[curquiza]

Hello @james2001

It's not planned at the moment
We encountered performance issues when we created this prototype and it's not in our priority for the moment.
I always recommend referring to the meilisearch/product#544 to talk about your needs and expectations, and asking some updates about the feature.

As a reminder, this PR discussion is for technical details 😊