Skip to content

sql: include WHERE subquery inputs in lineage#4752

Merged
kacpermuda merged 1 commit into
OpenLineage:mainfrom
mattfaltyn:agent/sql-predicate-subquery-inputs
Jul 22, 2026
Merged

sql: include WHERE subquery inputs in lineage#4752
kacpermuda merged 1 commit into
OpenLineage:mainfrom
mattfaltyn:agent/sql-predicate-subquery-inputs

Conversation

@mattfaltyn

Copy link
Copy Markdown
Contributor

closes: #4751

One-line summary for changelog:

SQL parser: include input tables referenced by WHERE predicate subqueries in table lineage.

Meaningful description

Problem

Select::visit traverses FROM relations, joins, and projections, but did not traverse the SELECT selection expression. As a result, queries such as:

SELECT 1 WHERE EXISTS (SELECT 1 FROM customers)

silently omitted customers from in_tables.

Solution

Visit the selection expression in its own context frame, then collect its table dependencies and aliases without merging predicate-only column ancestry into projected column lineage.

Add a focused table-lineage regression test for a WHERE EXISTS subquery. The existing nested-subquery column-lineage test continues to pass and guards against predicate columns leaking into output column lineage.

Impact

SQL parser consumers now receive complete input-table lineage for tables read only through WHERE predicate subqueries.

Validation

  • cargo test from integration/sql: 143 passed, 3 ignored
  • cargo fmt --all -- --check
  • Exact Python-binding reproduction repeated twice for EXISTS and IN subqueries, with projection-subquery controls
  • All configured hooks applicable to the changed Rust files
  • Repository-wide hooks passed except Docker-backed ShellCheck and Go lint, which could not run because Docker is unavailable; the transient integration/common mypy setup failure passed on rerun

Checklist

  • AI was used in creating this PR

@boring-cyborg boring-cyborg Bot added area:integration/sql area:tests Testing code language:rust Uses Rust programming language labels Jul 21, 2026
@mattfaltyn
mattfaltyn marked this pull request as ready for review July 21, 2026 18:43
@mattfaltyn
mattfaltyn requested a review from a team as a code owner July 21, 2026 18:43
Co-Authored-By: OpenAI Codex <noreply@openai.com>
Signed-off-by: Matt Faltyn <faltyn.matthew@gmail.com>
@kacpermuda
kacpermuda force-pushed the agent/sql-predicate-subquery-inputs branch from ea303d2 to b93c6e9 Compare July 22, 2026 13:39
@kacpermuda
kacpermuda merged commit 4943ead into OpenLineage:main Jul 22, 2026
61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:integration/sql area:tests Testing code language:rust Uses Rust programming language

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] SQL parser omits input tables referenced by SELECT predicate subqueries

2 participants