Skip to content

sql: Fix Seg Faults when parsing complex queries#4153

Merged
mobuchowski merged 1 commit into
OpenLineage:mainfrom
kchledowski:fix-seg-faults
Nov 19, 2025
Merged

sql: Fix Seg Faults when parsing complex queries#4153
mobuchowski merged 1 commit into
OpenLineage:mainfrom
kchledowski:fix-seg-faults

Conversation

@kchledowski
Copy link
Copy Markdown
Contributor

Problem

SQL Parser is not able to parse some of the queries, mainly more complex ones. Apparently it runs out of stack memory.

Given a troublesome SQL query:

  • With 1 MB stack size (default on Linux x64) the parsing fails with 139 error code.
  • With 2 MB stack size (default on macOS ARM64) the parsing is successful.

Closes: #3973

Solution

OpenLineage is currently using sqlparser (https://docs.rs/sqlparser) v0.50.0 from a custom fork.
Testing showed that starting with v0.52.0, the issue is gone.
The solution is to migrate to the latest version of sqlparser, which is v0.59.0

One-line summary:

sql: Fix Seg Faults when parsing complex queries

Checklist

  • You've signed-off your work
  • Your pull request title follows our guidelines
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • Your comment includes a one-liner for the changelog about the specific purpose of the change (not required for changes to tests, docs, or CI config)
  • You've versioned the core OpenLineage model or facets according to SchemaVer (if relevant)
  • You've added a header to source files (if relevant)

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2025 contributors to the OpenLineage project

@kchledowski kchledowski requested a review from a team as a code owner November 18, 2025 16:01
@boring-cyborg boring-cyborg Bot added area:integration/sql area:tests Testing code language:rust Uses Rust programming language labels Nov 18, 2025
Comment thread integration/sql/impl/Cargo.toml
Resolves segmentation faults that occurred when parsing certain queries
with 1 MB stack size.
The upstream v0.59.0 fixes the inefficient stack memory usage
that caused these issues.

Signed-off-by: kchledowski <github@chledowski.com>
@JDarDagran
Copy link
Copy Markdown
Contributor

Would it be possible to add some more complex example in tests that fails now and is fixed with this PR?

@kchledowski
Copy link
Copy Markdown
Contributor Author

Would it be possible to add some more complex example in tests that fails now and is fixed with this PR?

To make such test reliable, it would have to be forced to run with 1 MB stack size somehow. I guess it should be possible to spawn a new thread with 1 MB stack size and run a test in it, I'll look into it

@mobuchowski
Copy link
Copy Markdown
Member

IDK if it's worth spending time doing that. Bump of dependency is good enough reason for me to merge that :)

@JDarDagran
Copy link
Copy Markdown
Contributor

That's totally fine. I was rather asking for it hoping it's a small task to do.

@mobuchowski mobuchowski merged commit 3f1d597 into OpenLineage:main Nov 19, 2025
57 checks passed
@kchledowski kchledowski deleted the fix-seg-faults branch November 19, 2025 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:integration/sql area:tests Testing code language:rust Uses Rust programming language

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] SQL Lineage parse causes SigInt

4 participants