-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-6475][SQL] recognize array types when infer data types from JavaBeans #5146
Conversation
Test build #29035 has started for PR 5146 at commit
|
Test build #29035 has finished for PR 5146 at commit
|
Test PASSed. |
DataFrame df = context.createDataFrame(rdd, Bean.class); | ||
Row first = df.select("a", "b").first(); | ||
Assert.assertEquals(bean.getA(), first.getDouble(0), 0.0); | ||
Assert.assertArrayEquals(bean.getB(), first.<Integer[]>getAs(1)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be better to add assertions for schema field data types.
LGTM except for a minor comment about the test case. |
Test build #29058 has started for PR 5146 at commit
|
Test build #29058 has finished for PR 5146 at commit
|
Test PASSed. |
Merged into master. |
liancheng mengxr this is similar to #5146. Author: Punya Biswal <pbiswal@palantir.com> Closes #5578 from punya/feature/SPARK-6996 and squashes the following commits: d56c3e0 [Punya Biswal] Fix imports c7e308b [Punya Biswal] Support java iterable types in POJOs 5e00685 [Punya Biswal] Support map types in java beans
liancheng mengxr this is similar to apache#5146. Author: Punya Biswal <pbiswal@palantir.com> Closes apache#5578 from punya/feature/SPARK-6996 and squashes the following commits: d56c3e0 [Punya Biswal] Fix imports c7e308b [Punya Biswal] Support java iterable types in POJOs 5e00685 [Punya Biswal] Support map types in java beans
Right now if there is a array field in a JavaBean, the user wold see an exception in
createDataFrame
. @liancheng