sql: avoid stack overflow on deeply nested ASTs #9117
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I still need to check that this actually fixes the issue, so just a draft for now. |
|
Testing locally the query in 9103 still produces a crash. Not convinced it's the same stack crash (we're investigating what might be another). |
|
@mjibson did you test in release mode? In debug mode, |
|
@philip-stoev perhaps you could take a stab at tweaking the tests or the recursion limit constant here? I'm not going to have a chance to look at this for a bit. |
benesch
force-pushed
the
stack-guards
branch
from
f5a0097
to
1499132
Compare
|
Nope, was in debug. |
In the SQL planner, use the same technique as the SQL parser for automatically growing the stack when presented with a deeply nested AST. The stack-checking code is generalized and moved to a new `ore::stack` module. Fix MaterializeInc#9103.
benesch
force-pushed
the
stack-guards
branch
from
1499132
to
394e19d
Compare
|
Ok, great, this should be ready to go now. @philip-stoev the limits test now passes, and if you like you could introduce an even larger number of additions and verify that you get a smooth error message rather than a crash. I'm also pretty confident that the infrastructure in |
umanwizard
approved these changes
Nov 17, 2021
benesch
force-pushed
the
stack-guards
branch
from
394e19d
to
c8c5069
Compare
|
With 1000 items I get an actual result out of this query. With 1100 items I get a stack overflow elsewhere. I am unable to get the "recursion limit exceeded" error. I am going to open a separate ticket for this (as I have for several other stack overflows elsewhere). |
philip-stoev
approved these changes
Nov 17, 2021
|
For the optimizer, I had filed #9000 to use stacker. I added a comment in that ticket to use the newly added stack management utilities. |
aalexandrov
approved these changes
Nov 17, 2021
There was a problem hiding this comment.
LGTM, I will then it to fix similar issues in the optimizer (see #9000).
| /// | ||
| /// If the recursion limit for the recursion guard returned by | ||
| /// [`CheckedRecursion::recursion_guard`] has been reached, returns a | ||
| /// `RecursionLimitError`. Otherwise, it will call `f`, possibly growing the |
There was a problem hiding this comment.
nit: strictly speaking, returns a RecursionLimitError converted to E.
|
Thanks for the reviews, everyone! Let the games of whack a mole begin. |
14 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the SQL planner, use the same technique as the SQL parser for
automatically growing the stack when presented with a deeply nested AST.
The stack-checking code is generalized and moved to a new
ore::stackmodule.
Fix #9103.
Motivation
This PR fixes a recognized bug.
Checklist
user-facing behavior changes.