Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit the number of expanded fields it query_string and simple_query_string #26541

Merged
merged 2 commits into from Sep 8, 2017

Conversation

dakrone
Copy link
Member

@dakrone dakrone commented Sep 7, 2017

This limits the number of automatically expanded fields for the "all fields"
mode ("default_field": "*") for the query_string and simple_query_string
queries to 1024 fields.

Resolves #25105

…string

This limits the number of automatically expanded fields for the "all fields"
mode (`"default_field": "*"`) for the `query_string` and `simple_query_string`
queries to 1024 fields.

Resolves elastic#25105
@dakrone dakrone added :Search/Search Search-related issues that do not fall into other categories resiliency v7.0.0 labels Sep 7, 2017
Copy link
Contributor

@jimczi jimczi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Can you add a small warning in the query_string and simple_query_string docs regarding this limit ?

@Bargs
Copy link
Contributor

Bargs commented Oct 30, 2017

@dakrone is there a limit for multi_match queries as well?

jimczi added a commit that referenced this pull request Oct 30, 2017
…s that can be queried at once in the multi_match query.
@jimczi
Copy link
Contributor

jimczi commented Oct 30, 2017

@Bargs Yes I changed the docs in 792641a
The limit is applied to query_string, simple_query_string and multi_match queries.

jasontedor added a commit that referenced this pull request Oct 30, 2017
* master:
  Refactor internal engine
  [Docs] #26541: add warning regarding the limit on the number of fields that can be queried at once in the multi_match query.
  [Docs] Fix note in bucket_selector
  [Docs] Fix indentation of examples (#27168)
  [Docs] Clarify `span_not` query behavior for non-overlapping matches (#27150)
  [Docs] Remove first person "I" from getting started (#27155)
@Bargs
Copy link
Contributor

Bargs commented Oct 30, 2017

@jimczi so metricbeat creates more than 1024 fields, which is causing ES to return errors when Kibana sends search requests that expand to all fields. Initially we thought setting the default_field in the index pattern settings would be a good solution since metricbeat can handle that without asking the user to do anything. But multi_match doesn't use default_field.

Any ideas how we can handle this gracefully for multi_match?

@dakrone
Copy link
Member Author

dakrone commented Oct 30, 2017

Any ideas how we can handle this gracefully for multi_match?

Can you grab the existing value of default_field and use that for the fields parameter in the multi_match query?

@simianhacker
Copy link
Member

@dakrone They would need to figure out which index to grab it from based on the pattern.

@Bargs
Copy link
Contributor

Bargs commented Nov 1, 2017

And the default_field could potentially be different for different indices that match that index pattern :/

@dakrone dakrone deleted the limit-field-expansion branch December 13, 2017 20:36
cbuescher pushed a commit to cbuescher/elasticsearch that referenced this pull request Oct 26, 2018
Starting with Elasticsearch 7.0, we limit the number of automatically expanded
fields for the "all fields" mode ("default_field": "*") for the query_string and
simple_query_string queries to 1024 fields (see elastic#26541). This change adds a
deprecation warning to the QueryParserHelper where in the next major version we
will throw an error to warn users of the upcoming change.

Relates to elastic#26541
cbuescher pushed a commit to cbuescher/elasticsearch that referenced this pull request Oct 26, 2018
Adds a note to warn users about the limit introduced in elastic#26541.
cbuescher pushed a commit to cbuescher/elasticsearch that referenced this pull request Oct 26, 2018
Adds a note to warn users about the limit introduced in elastic#26541.
cbuescher pushed a commit that referenced this pull request Oct 29, 2018
Starting with Elasticsearch 7.0, we limit the number of automatically expanded
fields for the "all fields" mode ("default_field": "*") for the query_string and
simple_query_string queries to 1024 fields (see #26541). This change adds a
deprecation warning to the QueryParserHelper where in the next major version we
will throw an error to warn users of the upcoming change.

Relates to #26541
cbuescher pushed a commit that referenced this pull request Oct 29, 2018
Adds a note to warn users about the limit introduced in #26541.
kcm pushed a commit that referenced this pull request Oct 30, 2018
Adds a note to warn users about the limit introduced in #26541.
cbuescher pushed a commit to cbuescher/elasticsearch that referenced this pull request Nov 6, 2018
Currently we introduced a hard limit of 1024 to the number of fields a query can
be expanded to in elastic#26541. Instead of using a hard limit, we should make this
configurable. This change removes the hard limit check and uses the existing
`max_clause_count` setting instead.

Closes elastic#34778
cbuescher pushed a commit that referenced this pull request Nov 8, 2018
Currently we introduced a hard limit of 1024 to the number of fields a query can
be expanded to in #26541. Instead of using a hard limit, we should make this
configurable. This change removes the hard limit check and uses the existing
`max_clause_count` setting instead.

Closes #34778
@jimczi jimczi added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>breaking resiliency :Search/Search Search-related issues that do not fall into other categories v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants