Don't honor lowercase expanded terms in regex query generated from query string query

[mattweber]

On the mailing list a user was using a query string query against an unanalyzed field that contained values such as:

5.2
5.3-SNAPSHOT

He wanted to return all documents that were not a snapshot version so they used a query string like -version:/.*SNAPSHOT/. This did not work. Turns out that lowercase_expanded_terms was turning the regex into -version:/.*snapshot/ which doesn't match the unanalyzed field due to case.

He was using Kibana which does not currently expose the lowercaseExpandedTerms option, so the user really had no way to fix this without changing the mapping and reindexing or patching kibana.

Should the lowercase_expanded_terms option even be honored in a regex query? I would think no.

I did not see a way to set a case insensitive flag on the lucene regex query and the (?i) flag from java regex did not work.

[clintongormley]

Or at least we shouldn't lowercase expanded terms when we're querying a not_analyzed field

[amitelad7]

+1 running into this issue as well

[clintongormley]

We should add a (default) third setting for the lowercase_expanded_terms of auto. That will only lowercase if the field is analyzed, but not if it is not analyzed.

[NickMeves]

+1

Was about to open a new issue then I stumbled across this. Running into this alot lately with both Kibana and Graylog.

Issue is with not_analyzed field using regex for us as well.

[khadrin]

👍

[pickypg]

I wonder why we even need this setting. The analysis process should take care of lowercasing, if it's desirable for the fields in question, which is not a query-global decision, and it should otherwise not touch the entry.

[pickypg]

Highly related to #9978

[clintongormley]

Closing in favour of #9978