fix(optimizer): keep columns a recursive CTE reads from itself [CLAUDE] - #7899
Merged
georgesittas merged 1 commit intoJul 20, 2026
Merged
Conversation
pushdown_projections records column usage of a recursive CTE's self-reference against the throwaway scope that _traverse_ctes creates for the recursive term, so that usage is lost and projections the CTE still joins on get pruned. The corrupted CTE then crashes merge_subqueries with a KeyError (or silently generates invalid SQL). Guard self-referencing recursive CTEs with SELECT_ALL, mirroring the existing DISTINCT guard; merge_subqueries and pushdown_predicates already special-case recursive CTEs the same way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
georgesittas
approved these changes
Jul 20, 2026
georgesittas
left a comment
Collaborator
There was a problem hiding this comment.
Thank you, LGTM, but I think we can improve this by checking what projections are actually referenced to avoid the over-conservative block.
I'll take it to the finish line myself.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes
optimizer.optimizecrashing (or silently producing invalid SQL) on a self-referencing recursive CTE when the enclosing query doesn't select every CTE column.Repro on current main:
Root cause:
_traverse_ctesgives the CTE body a throwaway branch scope to resolve the recursive term's self-reference.pushdown_projectionsrecords the self-reference's column usage (t.linkabove) against that throwaway scope, so it is lost, and the projections the recursive term still joins on get pruned from the CTE.merge_subqueriesthen crashes on the missing projection; depending on the query the pruning can also survive silently as invalid SQL. Remapping the usage to the real scope wouldn't help on its own, because the reversed-postorder pass visits the union before the recursive term, making the self-reference a back-edge.Fix: treat self-referencing recursive CTEs conservatively and skip pruning their projections, mirroring the existing
DISTINCTguard in the same loop.merge_subqueries._is_recursiveandpushdown_predicatesalready special-case recursive CTEs the same way;pushdown_projectionswas the one rule missing the guard.Added a regression test that asserts the optimized CTE keeps the column its recursive term joins on (it fails with a
KeyErrorwithout the fix).🤖 Generated with Claude Code