-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-20550][SPARKR] R wrapper for Dataset.alias #17825
Changes from 17 commits
944a3ec
5e9f8da
73133f9
848eeef
05c0781
22d7cf6
22e1292
6bb3d91
b3c1a41
40fedcb
1e1ad44
2d5ace2
5fe5495
09f9cca
f1c74f3
43c02bc
505561a
1f1e72b
2b8f288
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3745,3 +3745,26 @@ setMethod("hint", | |
jdf <- callJMethod(x@sdf, "hint", name, parameters) | ||
dataFrame(jdf) | ||
}) | ||
|
||
#' alias | ||
#' | ||
#' @aliases alias,SparkDataFrame-method | ||
#' @family SparkDataFrame functions | ||
#' @rdname alias | ||
#' @name alias | ||
#' @examples | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. add There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done, but do we actually need this? We don't use roxygen to maintain There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. true, it's more for tracking it manually |
||
#' \dontrun{ | ||
#' df <- alias(createDataFrame(mtcars), "mtcars") | ||
#' avg_mpg <- alias(agg(groupBy(df, df$cyl), avg(df$mpg)), "avg_mpg") | ||
#' | ||
#' head(select(df, column("mtcars.mpg"))) | ||
#' head(join(df, avg_mpg, column("mtcars.cyl") == column("avg_mpg.cyl"))) | ||
#' } | ||
#' @note alias(SparkDataFrame) since 2.3.0 | ||
setMethod("alias", | ||
signature(object = "SparkDataFrame"), | ||
function(object, data) { | ||
stopifnot(is.character(data)) | ||
sdf <- callJMethod(object@sdf, "alias", data) | ||
dataFrame(sdf) | ||
}) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -130,19 +130,19 @@ createMethods <- function() { | |
|
||
createMethods() | ||
|
||
#' alias | ||
#' | ||
#' Set a new name for a column | ||
#' | ||
#' @param object Column to rename | ||
#' @param data new name to use | ||
#' | ||
#' @rdname alias | ||
#' @name alias | ||
#' @aliases alias,Column-method | ||
#' @family colum_func | ||
#' @export | ||
#' @note alias since 1.4.0 | ||
#' @examples \dontrun{ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. think generally we put \dontrun on the next line |
||
#' df <- createDataFrame(iris) | ||
#' | ||
#' head(select( | ||
#' df, alias(df$Sepal_Length, "slength"), alias(df$Petal_Length, "plength") | ||
#' )) | ||
#' } | ||
#' @note alias(Column) since 1.4.0 | ||
setMethod("alias", | ||
signature(object = "Column"), | ||
function(object, data) { | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -387,6 +387,17 @@ setGeneric("value", function(bcast) { standardGeneric("value") }) | |
#' @export | ||
setGeneric("agg", function (x, ...) { standardGeneric("agg") }) | ||
|
||
#' alias | ||
#' | ||
#' Returns a new SparkDataFrame or Column with an alias set. Equivalent to SQL "AS" keyword. | ||
#' | ||
#' @name alias | ||
#' @rdname alias | ||
#' @param object x a Column or a SparkDataFrame | ||
#' @param data new name to use | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. shouldn't we have a
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. that we did, at one point. I think the feedback is we could have one line for parameter ( There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To be honest I find both equally confusing, so if you think that a single annotation is better, I am happy to oblige. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. that's true actually. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On the bright side it looks like matching #' alias
#'
#' @aliases alias,SparkDataFrame-method
#' @family SparkDataFrame functions
#' @rdname alias,SparkDataFrame-method
#' @name alias
... and #' alias
#'
#' @aliases alias,SparkDataFrame-method
#' @family SparkDataFrame functions
#' @rdname alias,SparkDataFrame-method
#' @name alias
... (I hope this is what you mean) indeed solves SPARK-18825. But it doesn't generate any docs for these two and makes CRAN checker unhappy:
Docs for generic are created but it doesn't help us here. Even if we bring Theres is also my favorite I opened to suggestions, but personally I am out ideas. I've been digging trough There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. sigh, sadly I think you have captured all the constraints we are working with here. let's get the 3 lines in the same order
to
|
||
#' @return a Column or a SparkDataFrame | ||
NULL | ||
|
||
#' @rdname arrange | ||
#' @export | ||
setGeneric("arrange", function(x, col, ...) { standardGeneric("arrange") }) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add
@family SparkDataFrame functions
I think we should probably review all these
@family
at one point...There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I general it would nice to sweep all the files to make it more consistent. Capitalization, punctuation, examples. return and such.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes!