New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a limit for graph phrase query expansion #34031

Merged
merged 3 commits into from Sep 25, 2018

Conversation

Projects
None yet
4 participants
@jimczi
Member

jimczi commented Sep 25, 2018

Today query parsers throw TooManyClauses exception when a query creates
too many clauses. However graph phrase queries do not respect this limit.
This change adds a protection against crazy expansions that can happen when
building a graph phrase query. This is a temporary copy of the fix available
in https://issues.apache.org/jira/browse/LUCENE-8479 but not merged yet.
This logic will be removed when we integrate the Lucene patch in a future
release.

Add a limit for graph phrase query expansion
Today query parsers throw TooManyClauses exception when a query creates
too many clauses. However graph phrase queries do not respect this limit.
This change adds a protection against crazy expansions that can happen when
building a graph phrase query. This is a temporary copy of the fix available
in https://issues.apache.org/jira/browse/LUCENE-8479 but not merged yet.
This logic will be removed when we integrate the Lucene patch in a future
release.
@elasticmachine

This comment has been minimized.

elasticmachine commented Sep 25, 2018

Pinging @elastic/es-search-aggs

@romseygeek

LGTM

Term[] terms = graph.getTerms(field, start);
assert terms.length > 0;
if (terms.length >= BooleanQuery.getMaxClauseCount()) {
throw new BooleanQuery.TooManyClauses();

This comment has been minimized.

@cbuescher

cbuescher Sep 25, 2018

Member

It seems the former check is triggered by the test, but this code path isn't. I haven't checked all the test setup logic, but would it be difficult to include a case that also triggers this code path?

This comment has been minimized.

@jimczi

jimczi Sep 25, 2018

Member

I pushed 5bc8a40 to test both paths

@cbuescher

@jimczi LGTM, I left two minor questions mostly for my own education, would be nice to hear your opinion those

TokenStream ts = it.next();
SpanQuery q = createSpanQuery(ts, field);
if (q != null) {
if (queries.size() >= BooleanQuery.getMaxClauseCount()) {

This comment has been minimized.

@cbuescher

cbuescher Sep 25, 2018

Member

Just for my own education, and it is certainly super minor: when reading this part I was wondering if it would make sense to get the maxClauseCount limit once at the beginning of this method since its unlikely to change to avoid method calls in each iteration). Maybe Java does some clever optimizations to avoid this though and the effect most likely negligible.

@jimczi jimczi merged commit 0f878ef into elastic:master Sep 25, 2018

5 checks passed

CLA Commit author is a member of Elasticsearch
Details
elasticsearch-ci Build finished.
Details
elasticsearch-ci/java11 Build finished.
Details
elasticsearch-ci/oss-distro-docs Build finished.
Details
elasticsearch-ci/packaging-sample Build finished.
Details

@jimczi jimczi deleted the jimczi:max_graph_phrase_expansion branch Sep 25, 2018

jimczi added a commit that referenced this pull request Sep 25, 2018

Add a limit for graph phrase query expansion (#34031)
Today query parsers throw TooManyClauses exception when a query creates
too many clauses. However graph phrase queries do not respect this limit.
This change adds a protection against crazy expansions that can happen when
building a graph phrase query. This is a temporary copy of the fix available
in https://issues.apache.org/jira/browse/LUCENE-8479 but not merged yet.
This logic will be removed when we integrate the Lucene patch in a future
release.

@jimczi jimczi added the v6.4.2 label Sep 25, 2018

jimczi added a commit that referenced this pull request Sep 25, 2018

Add a limit for graph phrase query expansion (#34031)
Today query parsers throw TooManyClauses exception when a query creates
too many clauses. However graph phrase queries do not respect this limit.
This change adds a protection against crazy expansions that can happen when
building a graph phrase query. This is a temporary copy of the fix available
in https://issues.apache.org/jira/browse/LUCENE-8479 but not merged yet.
This logic will be removed when we integrate the Lucene patch in a future
release.

jimczi added a commit to jimczi/elasticsearch that referenced this pull request Sep 25, 2018

pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Sep 26, 2018

Add a limit for graph phrase query expansion (elastic#34031)
Today query parsers throw TooManyClauses exception when a query creates
too many clauses. However graph phrase queries do not respect this limit.
This change adds a protection against crazy expansions that can happen when
building a graph phrase query. This is a temporary copy of the fix available
in https://issues.apache.org/jira/browse/LUCENE-8479 but not merged yet.
This logic will be removed when we integrate the Lucene patch in a future
release.

anuptripathi4u added a commit to anuptripathi4u/elasticsearch that referenced this pull request Sep 28, 2018

Add a limit for graph phrase query expansion (elastic#34031)
Today query parsers throw TooManyClauses exception when a query creates
too many clauses. However graph phrase queries do not respect this limit.
This change adds a protection against crazy expansions that can happen when
building a graph phrase query. This is a temporary copy of the fix available
in https://issues.apache.org/jira/browse/LUCENE-8479 but not merged yet.
This logic will be removed when we integrate the Lucene patch in a future
release.

jimczi added a commit that referenced this pull request Oct 19, 2018

kcm added a commit that referenced this pull request Oct 30, 2018

Add a limit for graph phrase query expansion (#34031)
Today query parsers throw TooManyClauses exception when a query creates
too many clauses. However graph phrase queries do not respect this limit.
This change adds a protection against crazy expansions that can happen when
building a graph phrase query. This is a temporary copy of the fix available
in https://issues.apache.org/jira/browse/LUCENE-8479 but not merged yet.
This logic will be removed when we integrate the Lucene patch in a future
release.

@Mpdreamz Mpdreamz referenced this pull request Dec 13, 2018

Open

[meta] 6.5.0 Release #3457

1 of 92 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment