Skip to content

Fix issue #24701 Better error when the query contains reserved names - #24715

Open
Ruchirtripathi wants to merge 4 commits into
apache:mainfrom
Ruchirtripathi:fix-issue-24701
Open

Fix issue #24701 Better error when the query contains reserved names #24715
Ruchirtripathi wants to merge 4 commits into
apache:mainfrom
Ruchirtripathi:fix-issue-24701

Conversation

@Ruchirtripathi

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #24701

Rationale for this change

When a query renames a column using a DataFusion reserved name (specifically names starting with __common_expr, which are used internally by the Common
Subexpression Elimination optimizer pass), it currently fails during execution with obscure Arrow errors (e.g., Arrow error: Invalid argument error:
Invalid comparison operation: Float64 > Boolean). This happens because the user-provided alias collides with the internal optimizer logic.

This change ensures we fail earlier and return a clear, descriptive planning error to the user indicating that the column name is reserved.

What changes are included in this PR?

• Added a check_plan validation step inside SqlToRel::sql_statement_to_plan_with_context in datafusion/sql/src/statement.rs.
• The unoptimized LogicalPlan is traversed, and if any field in any node's schema starts with the reserved prefix __common_expr, we immediately throw a
plan_err!.
• Doing this validation on the raw unoptimized plan ensures that we correctly flag user-provided reserved aliases without incorrectly flagging the
__common_expr aliases legitimately added later by the CSE optimizer pass.

Are these changes tested?

Yes, they have been verified against the reproducer provided in the issue to return the correct error (__common_expr_2 is a reserved DataFusion column
name, please use another name). It was also verified that this does not break existing logic where CSE natively inserts these aliases (e.g.,
list_view_agg_test passes successfully).

Are there any user-facing changes?

Yes. Users who alias their columns with the __common_expr prefix will now receive a clear, actionable planning error instead of a confusing Arrow
execution error.

@github-actions github-actions Bot added the sql SQL Planner label Aug 27, 2026
@codecov-commenter

codecov-commenter commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.51%. Comparing base (dd77ffc) to head (3457bcf).

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #24715   +/-   ##
=======================================
  Coverage   81.51%   81.51%           
=======================================
  Files        1123     1123           
  Lines      405139   405151   +12     
  Branches   405139   405151   +12     
=======================================
+ Hits       330242   330263   +21     
+ Misses      55568    55560    -8     
+ Partials    19329    19328    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@appletreeisyellow

Copy link
Copy Markdown
Contributor

@Ruchirtripathi Thank you for the fix! Is it possible to add a test to show the new error message?

@Ruchirtripathi

Copy link
Copy Markdown
Contributor Author

@appletreeisyellow Thank you for the review! I've just pushed a commit that adds a test for this in sql_integration.rs. It verifies that attempting to alias a column to __common_expr_1 correctly throws the reserved name error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Better error when the query contains reserved names

3 participants