-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-8810] [SQL] Added several UDF unit tests for Spark SQL #7207
Conversation
testData.sqlContext.udf.register("havingFilter", (n:Long) => { n > 5 }) | ||
|
||
val result = | ||
testData.sqlContext.sql("SELECT g, SUM(v) as s FROM groupData GROUP BY g HAVING havingFilter(s)") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we now prefer just having the dataset closer to the test case rather than putting them in TestData.
You can do something like this easily in each test case itself
val df = Seq(("red", 1), ("red", 2), ("blue", 10), ("green", 100), ("green", 200)).toDF("g", "v")
df.registerTempTable("groupData")
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line exceeds 100 characters and the last test failure is due to this. @spirom , Could you add proper indentation here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
Thanks - looks pretty good for the first patch! There are just some minor style issues. |
Jenkins, ok to test. |
ok to test. |
Test build #36484 has finished for PR 7207 at commit
|
Test build #36512 has finished for PR 7207 at commit
|
Thanks - I've merged this in master. |
One test for each of the GROUP BY, WHERE and HAVING clauses, and one that combines all three with an additional UDF in the SELECT.
(Since this is my first attempt at contributing to SPARK, meta-level guidance on anything I've screwed up would be greatly appreciated, whether important or minor.)