Remove the lowercase_expanded_terms and locale options from (simple_)query_string.
#19057
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ple_)query_string`. This pull request uses the `MultiTermAwareComponent` interface in order to figure out how to deal with queries that match partial strings. This provides a better out-of-the-box experience and allows to remove the `lowercase_expanded_terms` and `locale` (which was only used for lowercasing) options. Things are expected to work well for custom analyzers. However, built-in analyzers make it challenging to know which components should be kept for multi-term analysis. The way it is implemented today is thet there is a default implementation that returns a lowercasing analyzer, which should be fine for most language analyzers for european languages. I did not want to go crazy with configuring the correct multi-term analyzer for those until we have a way to test that we are sync'ed with what happens in Lucene like we do for testing which factories need to implement `MultiTermAwareComponent`. In the future we could consider removing `analyze_wildcards` as well, but the query parser currently has the ability to tokenize it and generate a term query for the n-1 first tokens and a wildcard query on the last token. I suspect some users are relying on this behaviour so I think this should be explored in a separate change. Closes elastic#9978
|
I have been working on https://issues.apache.org/jira/browse/LUCENE-7355 on the Lucene side, that would help simplify this PR considerably. So I am stalling it until LUCENE-7355 is resolved. |
|
Superseded by #20208 |
clintongormley
added
:Search/Search
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request uses the
MultiTermAwareComponentinterface in order tofigure out how to deal with queries that match partial strings. This provides a
better out-of-the-box experience and allows to remove the
lowercase_expanded_termsandlocale(which was only used for lowercasing)options.
Things are expected to work well for custom analyzers. However, built-in
analyzers make it challenging to know which components should be kept for
multi-term analysis. The way it is implemented today is thet there is a default
implementation that returns a lowercasing analyzer, which should be fine for
most language analyzers for european languages. I did not want to go crazy
with configuring the correct multi-term analyzer for those until we have a way
to test that we are sync'ed with what happens in Lucene like we do for testing
which factories need to implement
MultiTermAwareComponent.In the future we could consider removing
analyze_wildcardsas well, but thequery parser currently has the ability to tokenize it and generate a term query
for the n-1 first tokens and a wildcard query on the last token. I suspect some
users are relying on this behaviour so I think this should be explored in a
separate change.
Closes #9978