SQL: Fix wrong appliance of StackOverflow limit for IN #36724

matriv · 2018-12-17T16:42:15Z

Fix grammar so that each element inside the list of values for IN
is a valueExpression and not a more generic expression. Introduce a
mapping for context names as some rules in the grammar are exited with
a different rule from the one they entered.This helps so that the decrement
of depth counts in the Parser's CircuitBreakerListener works correctly.

For the list of values for IN, don't count the
PrimaryExpressionContext as this is not visited on exitRule() due to
the peculiarity in our gramamr with the predicate and predicated.

Fixes: #36592

elasticmachine · 2018-12-17T16:42:17Z

Pinging @elastic/es-search

Fix grammar so that each element inside the list of values for IN is a `valueExpression` and not a more generic `expression`. Also change some names in the grammar so that they match the primary rule name plus "Default". This helps so that the decrement of depth counts in the Parser's `CircuitBreakerListener` works correctly. For the list of values for `IN` don't count the `PrimaryExpressionContext` as this is not visited on exit due to the peculiarity in our gramamr with the `predicate` and `predicated`. Fixes: elastic#36592

costin

LGTM

astefan

Left few minor comments.

astefan · 2018-12-18T10:03:29Z

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/parser/SqlParser.java

+
+            // Skip PrimaryExpressionContext for IN as it's not visited on exit due to
+            // the grammar's peculiarity rule with "predicated" and "predicate".
+            // Also skip the Identifiers as they are "cheap".
            if (ctx.getClass() != SqlBaseParser.UnquoteIdentifierContext.class &&


Can you import static all these classes to make the code a bit less crowded?

astefan · 2018-12-18T10:24:22Z

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/parser/SqlParser.java

+
+            // Avoid having negative numbers
+            if (depthCounts.containsKey(classNameToDecrement)) {
+                depthCounts.putOrAdd(classNameToDecrement, (short) 0, (short) -1);


Is the scenario of negative counter even possible?

Yes, because currently some rules are not encountered on enter but only on exit.
I'll open an issue soon to track this and investigate more.

astefan · 2018-12-18T10:32:30Z

x-pack/plugin/sql/src/test/java/org/elasticsearch/xpack/sql/parser/SqlParserTests.java

@@ -254,6 +261,40 @@ public void testLimitToPreventStackOverflowFromLargeComplexSubselectTree() {
            e.getMessage());
    }

+    public void testLimitStackOverflowForInAndLiteralsIsNotApplied() {
+        new SqlParser().createStatement("SELECT * FROM t WHERE a IN(" +


Only the parser being invoked is enough here? Can you check that at least that a minimal plan is created correctly given the query? Also, there is already a method that creates the statement given a query String: parseStatement(String).

this test is only to check that we don't hit the limit, but I can add more assertions.

astefan · 2018-12-18T10:40:03Z

run the gradle build tests 1

astefan

LGTM

matriv · 2018-12-18T11:46:53Z

run the gradle build tests 2

matriv · 2018-12-18T12:31:38Z

run the gradle build tests 2

matriv · 2018-12-18T12:31:44Z

run the gradle build tests 1

matriv · 2018-12-18T13:12:40Z

run the gradle build tests 2

Fix grammar so that each element inside the list of values for IN is a valueExpression and not a more generic expression. Introduce a mapping for context names as some rules in the grammar are exited with a different rule from the one they entered.This helps so that the decrement of depth counts in the Parser's CircuitBreakerListener works correctly. For the list of values for IN, don't count the PrimaryExpressionContext as this is not visited on exitRule() due to the peculiarity in our gramamr with the predicate and predicated. Fixes: #36592

matriv · 2018-12-18T15:22:14Z

Backported to 6.x with 7d2e2cf

Fix grammar so that each element inside the list of values for IN is a valueExpression and not a more generic expression. Introduce a mapping for context names as some rules in the grammar are exited with a different rule from the one they entered.This helps so that the decrement of depth counts in the Parser's CircuitBreakerListener works correctly. For the list of values for IN, don't count the PrimaryExpressionContext as this is not visited on exitRule() due to the peculiarity in our gramamr with the predicate and predicated. Fixes: #36592

matriv · 2018-12-18T15:46:13Z

Backported to 6.5 with 7fa3154

* elastic/master: (31 commits) enable bwc tests and switch transport serialization version to 6.6.0 for CAS features [DOCs] Adds ml-cpp PRs to alpha release notes (elastic#36790) Synchronize WriteReplicaResult callbacks (elastic#36770) Add CcrRestoreSourceService to track sessions (elastic#36578) [Painless] Add tests for boxed return types (elastic#36747) Internal: Remove originalSettings from Node (elastic#36569) [ILM][DOCS] Update ILM API authorization docs (elastic#36749) Core: Deprecate use of scientific notation in epoch time parsing (elastic#36691) [ML] Merge the Jindex master feature branch (elastic#36702) Tests: Mute SnapshotDisruptionIT.testDisruptionOnSnapshotInitialization Update versions in SearchSortValues transport serialization Update version in SearchHits transport serialization [Geo] Integrate Lucene's LatLonShape (BKD Backed GeoShapes) as default `geo_shape` indexing approach (elastic#36751) [Docs] Fix error in Common Grams Token Filter (elastic#36774) Fix rollup search statistics (elastic#36674) SQL: Fix wrong appliance of StackOverflow limit for IN (elastic#36724) [TEST] Added more logging Invalidate Token API enhancements - HLRC (elastic#36362) Deprecate types in index API (elastic#36575) Disable bwc tests until elastic#36555 backport is complete (elastic#36737) ...

* elastic/master: (61 commits) enable bwc tests and switch transport serialization version to 6.6.0 for CAS features [DOCs] Adds ml-cpp PRs to alpha release notes (elastic#36790) Synchronize WriteReplicaResult callbacks (elastic#36770) Add CcrRestoreSourceService to track sessions (elastic#36578) [Painless] Add tests for boxed return types (elastic#36747) Internal: Remove originalSettings from Node (elastic#36569) [ILM][DOCS] Update ILM API authorization docs (elastic#36749) Core: Deprecate use of scientific notation in epoch time parsing (elastic#36691) [ML] Merge the Jindex master feature branch (elastic#36702) Tests: Mute SnapshotDisruptionIT.testDisruptionOnSnapshotInitialization Update versions in SearchSortValues transport serialization Update version in SearchHits transport serialization [Geo] Integrate Lucene's LatLonShape (BKD Backed GeoShapes) as default `geo_shape` indexing approach (elastic#36751) [Docs] Fix error in Common Grams Token Filter (elastic#36774) Fix rollup search statistics (elastic#36674) SQL: Fix wrong appliance of StackOverflow limit for IN (elastic#36724) [TEST] Added more logging Invalidate Token API enhancements - HLRC (elastic#36362) Deprecate types in index API (elastic#36575) Disable bwc tests until elastic#36555 backport is complete (elastic#36737) ...

matriv added >bug v7.0.0 :Analytics/SQL SQL querying v6.6.0 v6.5.5 labels Dec 17, 2018

matriv requested review from costin and astefan December 17, 2018 16:42

matriv added >enhancement and removed >bug labels Dec 17, 2018

matriv force-pushed the mt/fix-36592 branch 2 times, most recently from 00bb58e to 73a1e1c Compare December 18, 2018 09:26

matriv force-pushed the mt/fix-36592 branch from 73a1e1c to 3b4053d Compare December 18, 2018 09:28

Merge remote-tracking branch 'upstream/master' into mt/fix-36592

de576de

costin approved these changes Dec 18, 2018

View reviewed changes

astefan reviewed Dec 18, 2018

View reviewed changes

matriv and others added 2 commits December 18, 2018 13:17

Address comments

450cc18

Merge remote-tracking branch 'upstream/master' into mt/fix-36592

3ef248d

astefan approved these changes Dec 18, 2018

View reviewed changes

matriv merged commit 74502d7 into elastic:master Dec 18, 2018

matriv deleted the mt/fix-36592 branch December 18, 2018 15:07

colings86 added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SQL: Fix wrong appliance of StackOverflow limit for IN #36724

SQL: Fix wrong appliance of StackOverflow limit for IN #36724

matriv commented Dec 17, 2018 •

edited

elasticmachine commented Dec 17, 2018

costin left a comment

astefan left a comment

astefan Dec 18, 2018

astefan Dec 18, 2018

matriv Dec 18, 2018

astefan Dec 18, 2018

matriv Dec 18, 2018

astefan commented Dec 18, 2018

astefan left a comment

matriv commented Dec 18, 2018

matriv commented Dec 18, 2018

matriv commented Dec 18, 2018

matriv commented Dec 18, 2018

matriv commented Dec 18, 2018

matriv commented Dec 18, 2018

SQL: Fix wrong appliance of StackOverflow limit for IN #36724

SQL: Fix wrong appliance of StackOverflow limit for IN #36724

Conversation

matriv commented Dec 17, 2018 • edited

elasticmachine commented Dec 17, 2018

costin left a comment

Choose a reason for hiding this comment

astefan left a comment

Choose a reason for hiding this comment

astefan Dec 18, 2018

Choose a reason for hiding this comment

astefan Dec 18, 2018

Choose a reason for hiding this comment

matriv Dec 18, 2018

Choose a reason for hiding this comment

astefan Dec 18, 2018

Choose a reason for hiding this comment

matriv Dec 18, 2018

Choose a reason for hiding this comment

astefan commented Dec 18, 2018

astefan left a comment

Choose a reason for hiding this comment

matriv commented Dec 18, 2018

matriv commented Dec 18, 2018

matriv commented Dec 18, 2018

matriv commented Dec 18, 2018

matriv commented Dec 18, 2018

matriv commented Dec 18, 2018

matriv commented Dec 17, 2018 •

edited