Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-12198] [SparkR] SparkR support read.parquet and deprecate parquetFile #10191

Closed
wants to merge 2 commits into from
Closed

[SPARK-12198] [SparkR] SparkR support read.parquet and deprecate parquetFile #10191

wants to merge 2 commits into from

Conversation

yanboliang
Copy link
Contributor

SparkR support read.parquet and deprecate parquetFile. This change is similar with #10145 for jsonFile.

@sun-rui
Copy link
Contributor

sun-rui commented Dec 8, 2015

LGTM

@yanboliang
Copy link
Contributor Author

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Dec 8, 2015

Test build #47321 has finished for PR 10191 at commit e507c31.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

felixcheung
felixcheung reviewed Dec 8, 2015
View reviewed changes
R/pkg/NAMESPACE
@@ -269,6 +269,7 @@ export("as.DataFrame",
"jsonFile",
"loadDF",
"parquetFile",
"read.parquet",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: sort - this should go after read.df

@felixcheung
Copy link
Member

looks good otherwise. are we adding a generic read() method?

@sun-rui
Copy link
Contributor

sun-rui commented Dec 9, 2015

@felixcheung, what do you mean "a generic read() method"? we have read.df() yet?

@felixcheung
Copy link
Member

sure - read.df doesn't match DataFrameReader so there are certain sources that it couldn't handle

@sun-rui
Copy link
Contributor

sun-rui commented Dec 9, 2015

@felixcheung, could you detail more about "read.df doesn't match DataFrameReader"? The only exception is JDBC source?

@SparkQA
Copy link

SparkQA commented Dec 9, 2015

Test build #47401 has finished for PR 10191 at commit 9e0fd63.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@yanboliang
Copy link
Contributor Author

+1 @sun-rui Only jdbc can not fit in read.df.

@felixcheung
Copy link
Member

sorry that's unrelated to this PR. Opened SPARK-12224 and SPARK-12232

@felixcheung
Copy link
Member

looks good.

@shivaram
Copy link
Contributor

LGTM. Thanks @yanboliang -- Merging this to master and branch-1.6

@SparkQA
Copy link

SparkQA commented Dec 10, 2015

Test build #47518 has finished for PR 10191 at commit 9e0fd63.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@asfgit asfgit closed this in eeb5872 Dec 10, 2015
asfgit pushed a commit that referenced this pull request Dec 10, 2015
@yanboliang @shivaram
[SPARK-12198][SPARKR] SparkR support read.parquet and deprecate parqu…
b7b9f77 
…etFile

SparkR support ```read.parquet``` and deprecate ```parquetFile```. This change is similar with #10145 for ```jsonFile```.

Author: Yanbo Liang <ybliang8@gmail.com>

Closes #10191 from yanboliang/spark-12198.

(cherry picked from commit eeb5872)
Signed-off-by: Shivaram Venkataraman <shivaram@cs.berkeley.edu>
@yanboliang yanboliang deleted the spark-12198 branch December 11, 2015 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
5 participants
@yanboliang @sun-rui @SparkQA @felixcheung @shivaram