-
Notifications
You must be signed in to change notification settings - Fork 411
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GLUTEN-4772][VL] Support empty map/array literal #4771
Changes from all commits
e7ed77d
32053b0
a7cff81
e218a77
112fedb
2e3889e
5b83029
5800076
4d5bda4
e7efd20
3a28733
5bdd9fa
2c630ae
28dc0ef
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -21,6 +21,7 @@ | |
|
||
import io.substrait.proto.Expression; | ||
import io.substrait.proto.Expression.Literal.Builder; | ||
import io.substrait.proto.Type; | ||
import org.apache.spark.sql.catalyst.util.ArrayData; | ||
|
||
public class ListLiteralNode extends LiteralNodeWithValue<ArrayData> { | ||
|
@@ -33,13 +34,18 @@ protected void updateLiteralBuilder(Builder literalBuilder, ArrayData array) { | |
Object[] elements = array.array(); | ||
TypeNode elementType = ((ListNode) getTypeNode()).getNestedType(); | ||
|
||
Expression.Literal.List.Builder listBuilder = Expression.Literal.List.newBuilder(); | ||
for (Object element : elements) { | ||
LiteralNode elementNode = ExpressionBuilder.makeLiteral(element, elementType); | ||
Expression.Literal elementExpr = elementNode.getLiteral(); | ||
listBuilder.addValues(elementExpr); | ||
if (elements.length > 0) { | ||
Expression.Literal.List.Builder listBuilder = Expression.Literal.List.newBuilder(); | ||
for (Object element : elements) { | ||
LiteralNode elementNode = ExpressionBuilder.makeLiteral(element, elementType); | ||
Expression.Literal elementExpr = elementNode.getLiteral(); | ||
listBuilder.addValues(elementExpr); | ||
} | ||
literalBuilder.setList(listBuilder.build()); | ||
} else { | ||
Type.List.Builder listTypeBuilder = Type.List.newBuilder(); | ||
listTypeBuilder.setType(elementType.toProtobuf()); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. if if element type is NullType, we will still fallback it right ? e.g., There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's related to #2996. It will not fallback once NullType is supported |
||
literalBuilder.setEmptyList(listTypeBuilder.build()); | ||
} | ||
|
||
literalBuilder.setList(listBuilder.build()); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you know why does it fallback ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, it's not related to empty literal, but NullType support.
I'll fix it in another PR, otherwise this PR is too big.