Skip to content

Commit

Permalink
ARROW-7796: [R] write_* functions should invisibly return their inputs
Browse files Browse the repository at this point in the history
I had some trouble building from source on Windows but I thought the changes were simple enough that I just went ahead. Tests and documentation also updated.

Closes #6387 from boshek/write_return and squashes the following commits:

d4d1ead <Sam Albers> Update r/NEWS.md
5f0e316 <Sam Albers> Merge branch 'master' into write_return
5d32579 <Sam Albers> update NEWS
53c3ab9 <Sam Albers> Update r/tests/testthat/test-read-write.R
673de49 <Sam Albers> Update r/tests/testthat/test-feather.R
11db34c <Sam Albers> Update r/tests/testthat/test-parquet.R
12222d6 <Sam Albers> update documentation
3768d0f <Sam Albers> return invisibly
a595854 <Sam Albers> write_arrow returns input
cac8db8 <Sam Albers> write_feather returns input
f6b79de <Sam Albers> write_parquet returns input

Lead-authored-by: Sam Albers <sam.albers@gov.bc.ca>
Co-authored-by: Sam Albers <sam.albers@gmail.com>
Signed-off-by: Neal Richardson <neal.p.richardson@gmail.com>
  • Loading branch information
2 people authored and nealrichardson committed Feb 7, 2020
1 parent cb686b3 commit a76e277
Show file tree
Hide file tree
Showing 10 changed files with 46 additions and 2 deletions.
3 changes: 3 additions & 0 deletions r/NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@

# arrow 0.16.0.9000


* `write_feather`, `write_arrow` and `write_parquet` now return their input
similar to `write_*` functions from `readr` (#6387, @boshek)
* Dataset filtering is now correctly supported for all Arrow date/time/timestamp column types.

# arrow 0.16.0
Expand Down
6 changes: 6 additions & 0 deletions r/R/feather.R
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
#' @param x `data.frame` or RecordBatch
#' @param sink A file path or an OutputStream
#'
#' @return the input `x` invisibly.
#'
#' @export
#' @examples
#' \donttest{
Expand All @@ -29,6 +31,8 @@
#' }
#' @include arrow-package.R
write_feather <- function(x, sink) {
x_out <- x

if (is.data.frame(x)) {
x <- record_batch(x)
}
Expand All @@ -42,6 +46,8 @@ write_feather <- function(x, sink) {

writer <- FeatherTableWriter$create(sink)
ipc___TableWriter__RecordBatch__WriteFeather(writer, x)

invisible(x_out)
}

#' @title FeatherTableWriter class
Expand Down
5 changes: 4 additions & 1 deletion r/R/parquet.R
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ read_parquet <- function(file,
#' disable compression, set `compression = "uncompressed"`.
#' Note that "uncompressed" columns may still have dictionary encoding.
#'
#' @return `NULL`, invisibly
#' @return the input `x` invisibly.
#'
#' @examples
#' \donttest{
Expand Down Expand Up @@ -145,6 +145,7 @@ write_parquet <- function(x,
allow_truncated_timestamps = allow_truncated_timestamps
)
) {
x_out <- x
x <- to_arrow(x)

if (is.character(sink)) {
Expand All @@ -164,6 +165,8 @@ write_parquet <- function(x,
writer <- ParquetFileWriter$create(schema, sink, properties = properties, arrow_properties = arrow_properties)
writer$WriteTable(x, chunk_size = chunk_size %||% x$num_rows)
writer$Close()

invisible(x_out)
}


Expand Down
4 changes: 4 additions & 0 deletions r/R/write-arrow.R
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ to_arrow.data.frame <- function(x) Table$create(!!!x)
#' `write_arrow` is a convenience function, the classes [arrow::RecordBatchFileWriter][RecordBatchFileWriter]
#' and [arrow::RecordBatchStreamWriter][RecordBatchStreamWriter] can be used for more flexibility.
#'
#' @return the input `x` invisibly.
#' @export
write_arrow <- function(x, sink, ...) {
UseMethod("write_arrow", sink)
Expand All @@ -61,6 +62,7 @@ write_arrow.RecordBatchWriter <- function(x, sink, ...){

#' @export
write_arrow.character <- function(x, sink, ...) {
x_out <- x
assert_that(length(sink) == 1L)
x <- to_arrow(x)
file_stream <- FileOutputStream$create(sink)
Expand All @@ -74,6 +76,8 @@ write_arrow.character <- function(x, sink, ...) {
# Available on R >= 3.5
# on.exit(file_writer$close(), add = TRUE, after = FALSE)
write_arrow(x, file_writer, ...)

invisible(x_out)
}

#' @export
Expand Down
3 changes: 3 additions & 0 deletions r/man/write_arrow.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions r/man/write_feather.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion r/man/write_parquet.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions r/tests/testthat/test-feather.R
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ test_that("Write a feather file", {
expect_true(file.exists(feather_file))
})

test_that("write_feather() returns its input", {
tib_out <- write_feather(tib, feather_file)
expect_identical(tib_out, tib)
})

test_that("feather read/write round trip", {
tf2 <- normalizePath(tempfile(), mustWork = FALSE)
write_feather(tib, tf2)
Expand Down
8 changes: 8 additions & 0 deletions r/tests/testthat/test-parquet.R
Original file line number Diff line number Diff line change
Expand Up @@ -130,3 +130,11 @@ test_that("write_parquet() to stream", {
con$close()
expect_equal(read_parquet(tf), df)
})

test_that("write_parquet() returns its input", {
df <- tibble::tibble(x = 1:5)
tf <- tempfile()
on.exit(unlink(tf))
df_out <- write_parquet(df, tf)
expect_identical(df, df_out)
})
9 changes: 9 additions & 0 deletions r/tests/testthat/test-read-write.R
Original file line number Diff line number Diff line change
Expand Up @@ -112,3 +112,12 @@ test_that("table round trip handles NA in integer and numeric", {
expect_true(is.na(res$dbl[10]))
unlink(tf)
})


test_that("write_arrow() returns its input", {
df <- tibble::tibble(x = 1:5)
tf <- tempfile()
on.exit(unlink(tf))
df_out <- write_arrow(df, tf)
expect_identical(df, df_out)
})

0 comments on commit a76e277

Please sign in to comment.