sql: Fix Seg Faults when parsing complex queries#4153
Merged
Conversation
kacpermuda
reviewed
Nov 18, 2025
Resolves segmentation faults that occurred when parsing certain queries with 1 MB stack size. The upstream v0.59.0 fixes the inefficient stack memory usage that caused these issues. Signed-off-by: kchledowski <github@chledowski.com>
bef6530 to
10fdb21
Compare
mobuchowski
approved these changes
Nov 18, 2025
Contributor
|
Would it be possible to add some more complex example in tests that fails now and is fixed with this PR? |
Contributor
Author
To make such test reliable, it would have to be forced to run with 1 MB stack size somehow. I guess it should be possible to spawn a new thread with 1 MB stack size and run a test in it, I'll look into it |
Member
|
IDK if it's worth spending time doing that. Bump of dependency is good enough reason for me to merge that :) |
Contributor
|
That's totally fine. I was rather asking for it hoping it's a small task to do. |
JDarDagran
approved these changes
Nov 19, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
SQL Parser is not able to parse some of the queries, mainly more complex ones. Apparently it runs out of stack memory.
Given a troublesome SQL query:
Closes: #3973
Solution
OpenLineage is currently using sqlparser (https://docs.rs/sqlparser) v0.50.0 from a custom fork.
Testing showed that starting with v0.52.0, the issue is gone.
The solution is to migrate to the latest version of sqlparser, which is v0.59.0
One-line summary:
sql: Fix Seg Faults when parsing complex queries
Checklist
SPDX-License-Identifier: Apache-2.0
Copyright 2018-2025 contributors to the OpenLineage project