New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-3813][SQL] Support "case when" conditional functions in Spark SQL. #2678
Conversation
Can one of the admins verify this patch? |
@@ -333,6 +338,24 @@ class SqlParser extends StandardTokenParsers with PackratParsers { | |||
IF ~> "(" ~> expression ~ "," ~ expression ~ "," ~ expression <~ ")" ^^ { | |||
case c ~ "," ~ t ~ "," ~ f => If(c,t,f) | |||
} | | |||
CASE ~> opt(expression) ~ (WHEN ~ expression ~ THEN ~ expression).* ~ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd prefer expression.?
to opt(expression)
since you're already using postfix style like .*
.
CASE ~> expression.? ~ (WHEN ~> expression ~ (THEN ~> expression)).* ~ | ||
(ELSE ~> expression).? <~ END ^^ { | ||
case casePart ~ altPart ~ elsePart => | ||
val altExprs = altPart.flatMap{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Space missing before {
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK. Updated
QA tests have started for PR 2678 at commit
|
case casePart ~ altPart ~ elsePart => | ||
val altExprs = altPart.flatMap{ | ||
case we ~ te => | ||
Seq(casePart.fold(we)(EqualTo(_, we)),te) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Space missing after ,
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK. Updated
Thank you for reviewing it. I have updated the code as per your comments. Please review it |
1 similar comment
Thank you for reviewing it. I have updated the code as per your comments. Please review it |
QA tests have finished for PR 2678 at commit
|
LGTM. Thanks for working on this! |
@marmbrus Can you also verify this PR. Thank you. |
Can one of the admins verify this patch? |
Thanks! I've merged to master. |
"case when" conditional function is already supported in Spark SQL but there is no support in SqlParser. So added parser support to it.
Author : ravipesala ravindra.pesala@huawei.com