feat(spark): implement Spark elt function
#17729
Conversation
davidlghellin
changed the title
elt function
| continue; | ||
| }; | ||
|
|
||
| let ansi_enable: bool = false; |
There was a problem hiding this comment.
this field seems is not changed?
There was a problem hiding this comment.
I will see to put this comments
There was a problem hiding this comment.
we can put these comments
To know and don´t forget
There was a problem hiding this comment.
We should remove this code itself but leave a TODO comment for ANSI support, otherwise it can be confusing why we have a static boolean variable here
comphead
left a comment
comphead
left a comment
There was a problem hiding this comment.
Thanks @davidlghellin it is good overall, just some minor comments
Jefffrey
left a comment
Jefffrey
left a comment
There was a problem hiding this comment.
Functionally looks good, have some further suggestions for refactoring
| for a in args.iter().skip(1) { | ||
| let casted = cast(a, &Utf8)?; | ||
| let sa = casted.as_string::<i32>().clone(); | ||
| cols.push(Arc::new(sa)); | ||
| } |
There was a problem hiding this comment.
Technically coerce_types() should have done this casting for us by now
| continue; | ||
| }; | ||
|
|
||
| let ansi_enable: bool = false; |
There was a problem hiding this comment.
We should remove this code itself but leave a TODO comment for ANSI support, otherwise it can be confusing why we have a static boolean variable here
|
Thank you for the contribution @davidlghellin and the review @Jefffrey We are picking up steam now on the spark functions 🚂 |
* feat(spark): implement Spark elt function * feat(spark): implement Spark elt function license * feat(spark): implement Spark elt function license complettly * test slt * doc * wip * change error * fmt * some commts * some changes * coerce basic <- udf * coerce basic drop 32 * change n,k,j names * first value int64 or casteable * change revision * change revision test * change revision test fmt
4 participants
Which issue does this PR close?
part of #15914
Rationale for this change
Migrate spark functions from https://github.com/lakehq/sail/ to datafusion engine to unify codebase
What changes are included in this PR?
implement spark udf elt
https://spark.apache.org/docs/latest/api/sql/index.html#elt
Are these changes tested?
unit-tests and sqllogictests added
Are there any user-facing changes?
now can be called in queries