New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-19518][SQL] IGNORE NULLS in first / last in SQL #17566
Conversation
@hvanhovell, could you check if it makes sense please? |
* Create a [[First]] expression. | ||
*/ | ||
override def visitFirst(ctx: FirstContext): Expression = withOrigin(ctx) { | ||
val ignoreNullsExpr = ctx.IGNORE != null && ctx.NULLS != null |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NIT: you don't have to check both (the grammar enforces that they both need to be present), either one suffices.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, sure. Thank you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM - one minor comment
Test build #75606 has finished for PR 17566 at commit
|
Test build #75607 has finished for PR 17566 at commit
|
LGTM - merging to master. Thanks! |
Thank you @hvanhovell. |
What changes were proposed in this pull request?
This PR proposes to add
IGNORE NULLS
keyword infirst
/last
in Spark's parser likewise http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions057.htm. This simply maps the keywords to existingignoreNullsExpr
.Before
After
How was this patch tested?
Unit tests in
ExpressionParserSuite
.