[VL] Fix CheckOverflowTransformer using wrong child type for cast decision#12261
Open
Xtpacz wants to merge 3 commits into
Open
[VL] Fix CheckOverflowTransformer using wrong child type for cast decision#12261Xtpacz wants to merge 3 commits into
Xtpacz wants to merge 3 commits into
Conversation
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
1 similar comment
|
Run Gluten Clickhouse CI on x86 |
Contributor
Author
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix: #12260
What changes are proposed in this pull request?
CheckOverflowTransformerreadsoriginal.child.dataTypeto decide whether to insert a cast. ForBinaryArithmetic, Spark's.dataTypereturnsleft.dataTyperather than the arithmetic result type. After child transformers apply rescale optimizations, the actual output type may differ from the Spark-declared type, and the cast is wrongly skipped.The resulting substrait plan has decimal types that mismatch function signatures. Velox's SimpleFunction validation rejects it, and
ColumnarPartialProjectRulefalls the entire Project back to JVM. Result is correct (via fallback) but native acceleration is lost.Reproducer
Root cause:
gluten/gluten-substrait/src/main/scala/org/apache/gluten/expression/UnaryExpressionTransformer.scala
Line 90 in fc90a79
this read the Spark expression's declared type instead of the transformer's actual output type.
Fix
How was this patch tested?
Before fix — Project falls back to JVM:
After fix — Project runs natively in Velox:
Key differences:
Project(JVM,*= codegen) toProjectExecTransformer(Velox native,^= transformer)VeloxColumnarToRowmoves from before Project (forced conversion to feed JVM) to after Project (deferred until output)