-
Notifications
You must be signed in to change notification settings - Fork 28.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-17347][SQL][Examples]Encoder in Dataset example has incorrect type #14901
Conversation
Test build #64730 has finished for PR 14901 at commit
|
Test build #64731 has finished for PR 14901 at commit
|
@@ -203,7 +203,7 @@ object SparkSQLExample { | |||
// No pre-defined encoders for Dataset[Map[K,V]], define explicitly | |||
implicit val mapEncoder = org.apache.spark.sql.Encoders.kryo[Map[String, Any]] | |||
// Primitive types and case classes can be also defined as | |||
implicit val stringIntMapEncoder: Encoder[Map[String, Int]] = ExpressionEncoder() | |||
// implicit val stringIntMapEncoder: Encoder[Map[String, Any]] = ExpressionEncoder() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This compiled before though, right? you're saying it's unnecessary because of the implicit in the line above? that seems fine but let's delete these 2 lines then.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am saying that the comments (L205) seem to indicate that, besides L204, the encoder can be defined as L206 as well. Actually, L206 has the incorrect type (you can find that by removing L204 and compile with the original L206)
so, I fixed the type and comment the line for successful compilation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it, LGTM
merged to master/2.0 |
… type ## What changes were proposed in this pull request? We propose to fix the Encoder type in the Dataset example ## How was this patch tested? The PR will be tested with the current unit test cases Author: CodingCat <zhunansjtu@gmail.com> Closes #14901 from CodingCat/SPARK-17347. (cherry picked from commit 97da410) Signed-off-by: Sean Owen <sowen@cloudera.com>
What changes were proposed in this pull request?
We propose to fix the Encoder type in the Dataset example
How was this patch tested?
The PR will be tested with the current unit test cases